Tech & Development

How to Test a Developer's Skills When They Use AI Coding Tools

Ascent24 Technologies
5 min read
Cover image with the title How to Test a Developer's Skills When They Use AI Coding Tools and the Ascent24 logo

Short answer: stop testing whether a developer can write code from memory, and start testing whether they can deliver working, safe code with the tools they will actually use. Give a small, realistic task, let them use AI openly, and score how they understand the problem, check the AI's output, debug, and explain their choices.

This applies whether you hire one developer, add outside developers, or choose an outsourcing partner.

Why the old tests no longer work

Most developers now use AI at work. In the Stack Overflow 2026 Developer Survey, 66% of respondents said they use coding assistants or coding agents, and only 17.2% said they use no AI tools at all (Stack Overflow press release, 6 Oct 2026).

That changes what a coding test measures. Canva's engineering team found that AI tools solved its traditional interview questions with correct, well-documented answers in seconds (Canva Engineering, 11 Jun 2025). If a puzzle can be solved by pasting it into a chat tool, it tells you very little about the person.

Some large employers have already changed their process. Canva now expects backend, frontend and machine learning candidates to use tools like Copilot, Cursor and Claude in its technical interviews. DoorDash replaced its traditional coding rounds with a 60-minute AI-assisted working session on a realistic project (DoorDash, 19 Mar 2026).

What good looks like now

The same survey shows how careful developers work. 48% trust AI output only when they can easily verify it, and just 6.6% trust it for important work decisions. Before using an AI answer, 76.5% run it locally, 63.9% compare it with the existing codebase, and 53% check tests or security implications. Only 9.6% use answers as they are (Stack Overflow 2026, AI trust; next steps after an AI answer).

So the skill you are buying is judgement: using AI to move faster, then checking the result like a reviewer would.

A practical four-step test

1. Use a small task that looks like your real work

Pick something from your own world: add a field to an order form and save it correctly, fix a bug in a small API, or connect two systems with a simple data sync. Give starter code, not a blank file. Reading and changing existing code is most of the job.

Keep it small. DoorDash found its first task was too big for an hour, because many candidates needed 15 to 20 minutes just to explore the code.

2. Allow AI openly, and say so in advance

Tell candidates which tools are allowed and ask them to say when they use one. Banning tools tests a way of working nobody uses on the job. If you need to check fundamentals, do it in a separate short discussion rather than a timed puzzle.

3. Ask them to narrate, then review their own work

Have the candidate share their screen and talk through what they are doing and why. At the end, ask them to review their own code as if a colleague had written it. What would they test? What could break? What would they change with another day?

For a take-home task, keep it under a few hours and pay for it if it is longer. Then hold a follow-up call where they walk you through their decisions.

4. Score five signals, not just the output

Signal What to look for
Understanding Asks clarifying questions before writing code
Verification Runs the code, writes or checks tests, spots AI mistakes
Debugging Reproduces problems and fixes the cause, not the symptom
Scope Delivers a working core first instead of a half-finished everything
Communication Explains trade-offs in plain language

Finishing every part matters less than a tight loop of building, checking and fixing.

Testing an outsourcing partner, not just a person

If you are choosing a vendor or a staff augmentation or dedicated team model, apply the same thinking to the team:

  • Ask for a short paid pilot on a real piece of your backlog.
  • Ask how they review AI-generated code, and who approves a pull request.
  • Ask for their AI policy. The survey found only 24% of developers say their organisation publishes approved tools or official AI guidance.
  • Check where your code and data go. Confirm which tools may see your source code and customer data, and get it in the contract.

For more on where AI-written code goes wrong, see what not to do in vibe coding.

Where Ascent24 fits

Ascent24 provides developers who join your in-house team and dedicated teams for larger projects. If you would like to run a small pilot task with us before committing, write to info@ascent24.io or message us on WhatsApp at +91 96558 70024.

Talk to us WhatsApp us

FAQs

Should I let candidates use ChatGPT or Copilot in a coding test?

For most roles, yes, as long as they say when they use it. Then judge how well they check and explain the result.

How do I check fundamentals if AI is allowed?

Ask them to explain a piece of their own code line by line, or to predict what happens if an input changes. Real understanding shows quickly.

Sources

You May Also Like