What to Look For When Hiring a Specialist for a Complex, Long-Term AI Project

The market for AI specialists today is arranged so that resumes look almost identical, while the results come from people with completely different profiles. Skills, credentials, experience, education — all of this certainly provides some confidence, but that confidence is, alas, deceptive: a resume shows the path traveled, not the ability to travel the next one. Let's break down what to actually look at when you're hiring for a complex, long-term AI project, where a wrong choice costs not one rewrite but months and reputation.

Why classic screening doesn't work in AI

Traditional hiring is built on the principle of "verify what can be verified." Formal markers — education, certificates, years in the profession, the list of technologies on a resume. In most engineering fields this works, at least because the stack changes slowly: if someone wrote Java for ten years, they'll probably write your service too.

In AI development this logic breaks. The field changes so fast that experience from three years ago can be almost irrelevant. Models, frameworks, practices — all of it updates faster than it can settle into a resume. A person with a perfect list of technologies can be helpless before a real task, while a person without trendy buzzwords on their resume can assemble a working solution in two weeks.

There's a second reason too: AI projects are rarely typical. Almost every one is a research effort at the boundary of what a model, data, and product can do. Here a specialist's value isn't in what they know but in how they search when they don't know. And that isn't measured by formal markers.

What actually says something about a specialist

Let's start with the fact that you should look at traces of real work. Not a list of projects, but how the person talks about them. A good sign is when they talk not about technologies but about problems: what was hard, where they got stuck, what they redid, what result they got. If the story is only stack and generalities, most likely the real involvement was limited.

The second sign is the ability to admit they don't know. That sounds strange, but in AI it's critical. The field is so vast that nobody knows everything. A specialist who honestly says "I haven't done this, but here's how I'd approach it" is worth more than one who answers everything confidently. The second type is usually either superficial or unaware of the boundaries of their competence — and both are dangerous on a long project.

The third sign is the ability to explain complex things simply. If a person can't explain, without a slide and a diagram, the difference between RAG and fine-tuning and when to use which — they probably don't fully understand it themselves. Depth usually shows up in clarity of expression, not in the number of terms.

The fourth sign is having a position. A good specialist doesn't just know technologies but has an opinion: what works, what doesn't, what's overrated, what's underrated. If the answer to every question is "it depends on the task," perhaps the person simply hasn't faced a choice.

Experience: which kind actually matters

Experience comes in different kinds, and not all of it is relevant. Experience at a large company with ready-made infrastructure doesn't always transfer to a startup where everything has to be built from scratch. Research experience doesn't always convert to production. Experience in one domain may not help in another.

What really matters: experience going through the full cycle. Someone who has launched an AI system into production and seen what happens a month after release knows things that someone who only built prototypes doesn't. The difference is huge: in production you run into questions of cost, latency, quality degradation, model updates — all the things that don't exist in laboratory conditions.

Experience with failure matters too. A specialist who can tell you where they went wrong, what didn't work and why, is usually worth more than one for whom everything always worked out. The second is either untrue, or means they never left their comfort zone.

Skills: what to look at first

The list of skills on a resume is nearly useless. Everyone writes "Python, PyTorch, LLM, RAG, agents." Several deeper abilities matter far more.

The first is the ability to formulate a task. AI projects often start with a vague "we need a smart assistant." A specialist who can turn that into a concrete spec with metrics and boundaries is worth a lot. If the person immediately rushes to write code without asking clarifying questions, that's a warning sign.

The second is the capacity for systems thinking. An AI system isn't just a model. It's data, pipelines, integrations, monitoring, updates, rollbacks. A specialist who sees the whole picture, not just their layer, is more useful over the long run.

The third is the skill of working with uncertainty. AI projects rarely go according to plan. The model behaves unexpectedly, the data turns out to be dirty, the result doesn't match expectations. A specialist who works calmly in that uncertainty is a find.

The fourth is the ability to learn fast. Given how quickly the field changes, this is arguably the most important skill of all. You can test it simply: ask what the person has been studying over the past six months, what articles they've read, what experiments they've run. If there's no answer, they probably haven't been learning.

Education: what it says and what it doesn't

Education is a useful but overrated signal. A technical degree says something about basic preparation, but nothing about the ability to solve specific tasks. A degree in ML says something about immersion in theory, but nothing about production development skills. AI courses say almost nothing except interest in the topic.

It's more useful to look at traces of independent work. Pet projects, publications, talks, open-source contributions. Not as a formal list, but as an indicator that the person is genuinely interested in what they do. In AI this matters: without personal interest, it's hard to keep the pace here.

Red flags

Now for what should give you pause.

The first flag is confidence about everything. If a specialist answers "yes" to any question and doesn't acknowledge the boundaries of their knowledge, you're most likely dealing with superficial competence. In AI nobody knows everything, and anyone who demonstrates otherwise is either deceiving or deceived.

The second flag is hype orientation. If the whole conversation revolves around trendy words rather than tasks, the person is probably chasing trends, not results. In six months the trend will change and the interest will fade.

The third flag is the absence of questions about your project. A good specialist starts with clarifications: what already exists, what the constraints are, what the goals are, what you consider success. If they're ready to solve before understanding the task, they'll solve their own task, not yours.

The fourth flag is a mismatch of scale. A specialist used to working in a large team with ready-made infrastructure may get lost in conditions where everything must be built from scratch. And vice versa. That's neither good nor bad — you just need to understand who you're hiring.

How to test: practice instead of theory

Formal interviews test knowledge of terms, not the ability to work. It's like testing a swimmer by their knowledge of muscle names: theoretically correct, practically useless.

It's far more informative to give a small real piece of work. Not a test assignment with a known answer, but a fragment of your actual task — with real data, real constraints, real uncertainty. Even if it takes a week or two. It's worth the time, because the cost of a hiring mistake on a long-term AI project is far higher.

This approach gives you one more important thing: you see how the person works, not how they talk about work. How they ask questions, how they react to feedback, how they handle uncertainty, how they make decisions with insufficient information. That matters far more than the list of technologies on a resume.

Author's column

Climax: what you're actually choosing

In the end, you're choosing not a set of skills but a person you'll walk a long and almost certainly difficult road with. An AI project isn't a sprint but a marathon on a changing course. Over such a distance, what matters more is not how fast the person starts, but how they hold up at the tenth kilometer.

A good signal in hiring is when, after the conversation, you don't feel you got all the answers but do feel the person will be able to find them along the way. A bad signal is when everything was smooth in the interview, but after hiring it turns out the specialist can only work in familiar conditions.

Education, skills, and experience are landmarks, not guarantees. They help filter out the clearly unsuitable but don't help pick the best. You'll find the best through real work, not through formal checks. And if it seems expensive and slow, compare it to the cost of a mistake when three months later it turns out the person can't carry the load.

Everything is within our power. We just have to honestly admit that formal markers describe the past, while you need someone who will build the future.

Glossary of terms

• AI project — a project using machine learning or generative AI models to solve a product task.
• Production (prod) — the working environment where the system is used by real users under real load.
• Prototype — an early version of a system built to test an idea. Not intended for real users.
• RAG (Retrieval-Augmented Generation) — an approach where the model answers based on retrieved external documents, not only on its internal knowledge.
• Fine-tuning — additional training of a model on specific data to improve quality in a narrow domain.
• LLM (Large Language Model) — a large language model, the foundation of modern AI text-generation systems.
• Pipeline — a sequence of data or task processing steps, an automated chain of processes.
• Metrics — measurable indicators of system quality: accuracy, completeness, latency, cost.
• Systems thinking — the ability to see a system as a whole rather than its individual parts, and to understand the connections between components.
• Screening — initial candidate selection by formal markers: resume, experience, education.
• Technical interview — a hiring stage where a candidate's technical knowledge is tested through questions and tasks.
• Vibe coding — a development practice using AI code-generation tools heavily, where the human sets direction and the model writes much of the code.
• Pet project — a specialist's personal project, made out of interest rather than by commission. Often the best indicator of real skills.
• Open source — projects with open source code that anyone can contribute to.
• Test assignment — a practical task a candidate completes during hiring to demonstrate skills.
• Red flags — warning signs in a candidate's behavior or competence indicating possible problems.
• Infrastructure — the set of technical means and services that keep a system running: servers, databases, queues, monitoring.
• Research — investigative work aimed at producing new knowledge, as opposed to applied development.
• Hiring — the process of selecting and bringing a specialist on board, including several selection stages.
• Competence — the combination of knowledge, skills, and experience that lets a specialist solve tasks of a certain class.
• Engineering maturity — a specialist's ability to make balanced technical decisions, accounting for constraints, risks, and long-term consequences.