Your machine learning model distinguishes between breeds of dogs with higher rates of accuracy than seasoned human professionals. Training it required tens of thousands of dog photos, which, though pre-normalized, still required extensive manipulation before becoming useful for supervised training. Now you need to put the model into production where it will see thousands of random photos rife with stray artifacts like people, landscapes… and cats. Many may contain no images of dogs at all, causing problems for your model. In other words, you are moving from the clean-room of the lab to the messy, real world.
Where do you start?
Transitioning from a controlled, lab environment to dynamic, transactional production system involves the same steps with new twists. Data must still be prepared, normalized and transformed but the transformations must happen automatically and at fire-hose scale. There is still an interface to your model, but now, rather than loading pre-normalized images from a filesystem or database, it involves potentially invalid input streaming through an authenticated ReST API and dozens of GPU-enabled, cloud-based virtual machines which hungrily consume data from distributed message queues. Worker processes that seize up during processing need to be gracefully removed from service, failures need to be reported via well instrumented dashboards, performance trends in terms of failures, model accuracy and overall system latencies need to be tracked for both immediate systems scaling and long-term planning.
S3 performs these roles so that your internal team can remain focused on machine and deep learning.