apply() - The Only Truly Hard Problem in MLOps

Speakers: Todd Underwood, Google and Niall Martin, Microsoft

MLOps solutions are often presented as addressing particularly challenging problems. This is mostly untrue. The majority of the problems solved by MLOps solutions have their origins in pre-ML data processing systems and are well addressed by the solutions that we devised to those problems. Data ingestion, feature storage, model serving and even model management and training are all relatively well addressed by traditional data processing approaches. And all of these can be solved with little or no understanding of the modeling challenge your system is designed to solve.

The only truly ML-centric, hard problem in MLOps is in the data. There is no automated, generalized way to mitigate the impact of subtle changes in the distribution of training data or of undetected changes in the semantics of that data. A general solution to this problem would unlock more usability and trust in ML than any other improvement we can make.