ML in Production

ML in Production

https://vickiboykis.com/2024/01/15/whats-new-with-ml-in-production/

Machine Learning: the high-interest credit card of technical debt https://research.google/pubs/machine-learning-the-high-interest-credit-card-of-technical-debt/

“Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning.”

Making GPUs go BRRR https://horace.io/brrr_intro.html

Patterns for Building LLM-based Systems & Products https://eugeneyan.com/writing/llm-patterns/

ML Engineering

• https://github.com/stas00/ml-engineering https://github.com/stas00/ml-engineering/blob/master/transformers/make-tiny-models.md

Quick hosting and running

• https://together.ai/

• https://replicate.com/

• https://anyscale.com/

• https://baseten.co/

• https://modal.com/

• https://banana.dev/

• https://runpod.io/

• https://bentoml.com/

• https://brev.dev/

• https://octo.ai/

• https://cerebrium.ai/

• https://fly.io

• https://www.paperspace.com/notebooks