Should you become an ML/AI engineer?

Across industries, companies are hiring machine learning engineers to build everything from customer-facing copilots to behind-the-scenes infrastructure. LinkedIn named AI engineer the fastest-growing role in the U.S. this year. Salaries are high. Job descriptions are everywhere.

But job openings don’t always mean it’s the right move.

Yes, the demand is real. But so is the complexity. These roles aren’t plug-and-play. The learning curve is steep. The work is different from traditional engineering. Even experienced developers are asking themselves, "Is this worth it?"

That’s what this post explores — beyond the market opportunity, what work in AI/ML actually looks like, what skills it takes to succeed, and how to figure out whether this is the right next step in your career.

The market is hot, but focused

AI hiring is growing fast, but it’s not uniform. It’s concentrated in specific sectors: tech giants building foundational models, AI-native startups building on top of them, and enterprise teams investing in narrow use cases like personalization, forecasting, or workflow automation.

According to the World Economic Forum’s Future of Jobs Report, the field is expected to grow by 40% (resulting in approximately 1 million new jobs) over the next five years. Companies want engineers with both ML depth and strong software foundations. And they often want them yesterday.

The compensation reflects that. Levels.fyi reports that the median total comp for ML engineers at top companies sits at $248,000. In startups, it varies more, but well-funded teams are competing hard for talent.

So yes — the market is real. But it rewards depth, not dabbling.

What ML/AI engineers actually do

You’ll find different job titles — machine learning engineer, AI engineer, applied scientist, ML platform engineer — but most of them share a few common building blocks:

• Model development: Training and tuning models using PyTorch, TensorFlow, or JAX. It could be anything from classic regression models to custom transformer architectures.
• Data work: Writing pipelines, cleaning raw inputs, and analyzing failure modes. A lot of time is spent on infrastructure, not just algorithms.
• Deployment and monitoring: Putting models in production, setting up evaluation metrics, catching drift or degradation.
• Collaboration: Working closely with product, data science, and infrastructure teams to make sure models are solving real problems, not just optimizing test accuracy.

Some engineers work on net-new models. Others focus on fine-tuning open-source LLMs. Many work on integrating models into products, like improving search, ranking, recommendations, or real-time decision-making.

There’s also a growing category of engineers working in AI-adjacent roles — people building scaffolding, tools, or infra for model teams rather than the models themselves.

The learning curve is real and worth understanding

If you don’t already have a background in data science or applied ML, becoming a true ML engineer is a major investment.

The technical skills go beyond most full-stack or backend roles. You’ll need:

• Strong Python, plus exposure to libraries like NumPy, pandas, and PyTorch
• A working understanding of linear algebra, probability, and optimization
• Comfort with model training, evaluation, and debugging
• Familiarity with MLOps tooling (like Docker, MLflow, SageMaker)
• And increasingly, experience fine-tuning LLMs or working with frameworks like Hugging Face Transformers or LangChain

You don’t need a PhD. But you do need time and motivation. Most engineers who make the switch successfully spend 6–12+ months upskilling through coursework, side projects, and structured support. It’s often easiest to break into ML/AI roles by doing an internal transfer as a SWE onto a more AI-focused team, and then moving into that space and learning from others on the team.

That effort pays off. But only if you actually enjoy the work.

When it’s a great move

This is a great path for developers who like deep systems work, love experimentation, and want to stay close to the core of where tech is headed. It’s a fit if:

• You’re curious about how models work
• You enjoy debugging complex systems and working with messy data
• You’re excited to spend time learning and iterating
• You want to work on products that learn, adapt, and evolve

There’s also upside. ML engineers with strong fundamentals are in high demand, and that demand isn’t slowing down. AI-native products are becoming the norm, not the exception. And teams that know how to ship working ML systems have leverage.

When it might not be

The role also isn’t for everyone. There’s a reason many smart engineers choose not to specialize in ML, even if they use it.

It might not be the right fit if:

• You prefer clearly scoped work over experimentation and iteration
• You like fast feedback loops and user-facing wins — ML cycles can be long and invisible
• You want to move fast and break things — ML often requires slow, careful tuning
• You don’t enjoy working with data (which is often messy, slow, and brittle)

And critically: if you’re interested in AI but don’t want to learn how models work from the inside out, that’s okay. There are other roles that still let you participate.

The middle path: working with AI, not on AI

You don’t have to become a machine learning engineer to build an AI career. There’s a huge opportunity in roles that bridge traditional engineering with model integration, application logic, and tooling.

Think:

• AI product engineer: Frontend/backend engineers integrating LLMs or embeddings into real products.
• ML platform engineer: Infra specialists building tools for model training, versioning, or deployment.
• Prompt engineer or LLM integrator: Engineers who design retrieval pipelines, construct prompts, or fine-tune foundation models for specific use cases.

These roles often require less math and modeling background, but a strong understanding of how AI fits into the bigger product picture. They’re perfect for engineers who want to stay close to the impact without getting deep into model internals.

And many of them are easier to pivot into from existing engineering experience.

The real question: Does the work excite you?

The title matters less than the work. Do you want to tune models? Build workflows? Create infrastructure that makes learning systems safer and faster?

Or do you want to build great products that use AI, even if you don’t build the models themselves?

The people who thrive in AI engineering roles aren’t chasing the market. They’re following a genuine interest. They enjoy the ambiguity. They like learning new tools. They’re willing to wrestle with the uncertainty because they believe in what they’re building.

If that sounds like you, this might be a great time to start the shift.

Get holistic interview prep with Formation

The Formation Fellowship gives mid-level and senior engineering job seekers everything they need to land their dream roles — including personalized skill brush-ups, resume help, unlimited mock interviews with experienced software engineers and hiring managers from top-tier tech companies, career and negotiation support, and more. 

If you’re having trouble navigating your job search on your own, apply here and get unconditional support from a team of engineering mentors, technical recruiters, career coaches, and more.