Design Thinking–Driven: From ML Ops to AI Engineering
How ML Teams can thrive in the Age of Plug-and-Play AI
🕰 A Lesson from History
The emergence of electricity in the late 19th and early 20th centuries forced many lamp-making companies into bankruptcy. But Edward Miller & Co., a prominent oil-lamp manufacturer based in Meriden, Connecticut, faced this pivotal moment differently. As electricity became more accessible, oil-lamp seems became more and more irrelevant.
Edward Miller & Co. didn’t resist the change. Instead they adapted. The company retrained its artisans, engineers, and factory workers to master electrical bulb design and production. The transition from handcrafted oil lamps to modern electric fixtures.
This wasn’t just a technological shift in their company.
It was an organizational transformation!
⚡ The Foundation Model Shift
The release of foundation models like GPT, Claude, and open-source versions started a pivotal shift in the AI landscape.
Initially, you needed elite research teams and millions to setup an infrastructure for AI. Now you can do the same thing via a simple API call.
AI became cheaper to implement!
Everyone wanted in. Many companies tried integrated AI into their products.
But beneath the surface, a surprising number still struggle to truly adapt. Ironically, even teams with established MLOps teams. How could these be?
If your ML Ops team is one of those, don’t worry. You’re not alone. Even companies with top-tier data science talent are facing growing pains. But the ones who act now have a window of opportunity to leap ahead.
🤖 Why Traditional ML Ops Teams Struggle to Adapt?
Historically, ML Ops revolves around supervised learning.
What is supervised learning?
You collect labeled data, train a model, deploy it, monitor it, and retrain.
Examples of supervised learning are:
Fraud Detection: They train models on years of labeled transaction data to identify patterns indicative of fraudulent activity. Is this fraud or not?
Product Recommendation: Teams use historical user behavior and purchase data to train models that suggest relevant products. Is this a relevant recommendation or not?
Email Spam Detection: ML systems are trained on user-labeled emails to classify and filter out spam messages. Is this spam or not?
But foundation models flip this script. They’re trained on huge and generalized data. They don’t need your labeled dataset to get started.
The challenge now with accessible foundation model is: how to prompt, fine-tune, or steer the model to deliver value. This demands a new mindset, one where your team needs to understand how to use models initially without direct training and with minimal data
🏓 Ending the AI Ping-Pong
Given how easy it is to start building AI feature, ML teams and product teams fall into this trap:
CEO: Let’s integrate AI!
ML team ask product managers: “What do you want AI to do?”
Product team ask ML engineers: “What can AI do?”
This loop goes nowhere. Without a shared workflow, nobody moves forward. This gap is exactly what AI Engineering aims to bridge.
Start by solving something real. Then ask, “Can AI make this faster, easier, smarter, or more scalable?”
🧠 Rethinking ML Pipelines with AI Engineering + Design Thinking
“The success of an ML system depends less on the fanciness of the model and more on how well the problem is defined and how well the solution is integrated into the product.”
Chip Huyen, Designing Machine Learning Systems (2022)
The way we integrate AI into products is shifting fundamentally. No longer do we start with a dataset and model. Instead, we begin with users: their pains, needs, and behavior.
🤖 AI Engineering (Design Thinking–Driven)
Inspired by Chip Huyen’s AI Engineering book, I’ve adapted the workflow to start with customers, not the dataset. From my learnings of building and working with startups (many of which didn’t quite take off 😅), the rise of accessible foundation models should give us a chance to shift our focus from “What can AI do?” to “How can AI make our customers’ lives easier?”
❌ Traditional ML Engineering:
Data ➡ Model ➡ Product ➡ Customer
Data: A team collects years of historical transaction records, each labeled as “fraud” or “not fraud.”
Model: They train a supervised learning model to recognize patterns that indicate potential fraud.
Product: The model is deployed into the company’s payment processing system to flag or block suspicious transactions in real-time.
Customer: Customers benefit indirectly through enhanced security and fewer fraudulent charges on their accounts.
It’s a proven system, but it assumes you already have structured data and a clear pattern to learn.
✅ AI Engineering (Design Thinking–Driven):
Customer ➡ Product ➡ Data ➡ Model
Customer: A product team notices that users often struggle to write effective product descriptions. This is a real, high-friction user problem.
Product: They respond by designing a “smart suggestion” feature directly within the editor to support users as they write.
Data: As users begin interacting with this feature, the team collects anonymized drafts and completions to understand usage patterns and pain points.
Model: With this data, they evaluate what foundation models can assist with (e.g., tone, clarity, SEO) and fine-tune or chain prompts to deliver helpful, personalized writing suggestions. If this is not enough, you can then later decide to augment the model with retrieval (RAG), integrate external tools (like grammar checkers or SEO analyzers), or design workflows with human-in-the-loop for higher quality control.
The process starts by solving a real user problem instead of building a model and looking for where to apply it. After identifying the problem, strategically bring in AI where it creates value.
You can see companies like Notion, Canva, and Shopify are already embracing this approach. Embedding AI seamlessly into their products that it feels like a natural extension of the user experience. From Notion’s AI that enhances productivity without disrupting workflow, to Canva’s Magic Studio that empowers non-designers to create effortlessly, to Shopify’s Sidekick that supports merchants like a real teammate. These platforms prove that AI can be invisible in interface, intuitive in use, and deeply human in intent.
🎯 Takeaway
We’re early in this transformation. But like Edward Miller & Co., the companies who adapt now will be the ones who lead.
Don’t wait for ML teams to be told what to build.
Don’t wait for product teams to "get AI."
Start by solving something real. Then ask, “Can AI make this faster, easier, smarter, or more scalable?”
AI Engineering isn’t a rebrand. It’s a reorganization.
Here’s a photo of me trying to reorg myself to learn what is possible with AI through the Master of Artificial Intelligence program at the University of Waikato, home of Weka.
References
Historic New England. (n.d.). Electric lamp factory workers [Photograph]. https://images.historicnewengland.org/rs/138533/scr/default.jpg
Huyen, C. (2024). The AI engineering book. Claypot AI. https://huyenchip.com/ai-engineering/book/
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models (Stanford CRFM Report). https://crfm.stanford.edu/report.html
Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. (2021). Dive into deep learning (Version 1.0.0). https://d2l.ai
Anthropic. (2024). How Notion uses Claude to help people work faster and smarter. https://www.anthropic.com/customers/notion ↩
Canva. (2023). Canva launches Magic Studio, a suite of AI tools for everyday creators. https://www.canva.com/newsroom/news/canva-ai-launches ↩
Upskillist. (2024). Hiring AI over humans? Decoding Shopify’s bold new strategy. https://www.upskillist.com/blog/hiring-ai-over-humans-decoding-shopifys-bold-new-strategy ↩