Technical Debt in Machine Learning

Great post exploring the often neglected problem of technical debt in machine learning systems. The author presents three types of technical debt:

  • Feedback Loops: you're ML system is fed data that it generated itself, improving performance metrics without actually improving the system. Fix lies in proper exploitation / exploration calibration.
  • Correction Cascades: If you apply too many fixes and heuristic corrections to your ML system as a result you are no linger able to properly train your system as a whole.
  • Hobo-Features: useless features that are hard to get rid of, e.g. a feature that gave a minor performance boost but has become neutral once more data was collected.


Want to receive more content like this in your inbox?