Scoring method
Deterministic
The same recording always gives the same score. Fixed rules compute every number; no language model forms an opinion of the candidate on a bug-fix round.
Explainable
Every dimension ships with the evidence that produced it, down to the prompt or test run, so a reviewer can disagree with a specific point.
Assists, never decides
Scores support human judgment. PraxisAI never rejects anyone automatically, and a report should not be the sole basis for a decision.
Each dimension is scored 0 to 100 from the round's recording. The overall score is their weighted sum, with a letter grade (A+ from 90, A from 80, B from 70, C from 60, D from 45).
weight 35%
Did the fix work? Measured by the task's hidden tests, which the candidate never sees.
weight 15%
How the candidate directed the agent.
weight 15%
Whether the candidate checked the work instead of trusting the agent.
weight 15%
What the result cost in agent usage.
Held to what the outcome earned: thrift on work that did not land is only partly credited.
weight 10%
Time used against the time allowed.
Fully gated on the outcome. A round with nothing attempted scores zero here.
weight 10%
What happened after something went wrong.
Held to what the outcome earned, down to a floor that still shows the behavior.
When the hidden tests cannot run (the task's environment failed to build), the round is marked not graded with no score, rather than a misleading number.
Using the agent is expected; it is not cheating and it is not penalized. What is measured is how it was used: whether the candidate gave it specific direction, checked what it produced, noticed when it went wrong, and what the result cost. A round solved with heavy, well-directed agent use can score at the top.
The agent runs through our proxy, so every prompt and token is recorded and capped. Help from outside the workspace (another AI tool, another person) cannot be seen directly; attention signals such as long focus losses and large pastes are flagged for a reviewer instead. After the round, candidates can write a short debrief explaining their change; it is shown to the hiring team as written after the round. It is not part of the score; it feeds the ownership and communication behavior in the hireability read.
The headline on a report (Strong hire signal, Promising, Mixed signal, Concerns) is chosen by rules from the same numbers: for example "Strong" needs the hidden tests to pass with nothing broken, an overall score of 75 or more, solid process scores, no high-severity flag, and prompts mostly in the candidate's own words. The strengths, risks and interview questions under it are templates filled from the evidence, each pointing at the moment it refers to. Candidates see their own copy of an assessment without the hiring team's verdict wording or interview questions.
At the top of every report, the hiring side's question: from an AI-coding point of view, how strong is this candidate? Seven behaviors are scored 0 to 100 by itemized rules over the same recording, each point tied to a sentence and, where there is one, a moment in the round. The hireability score is their weighted average (75%) blended with the hidden-test outcome (25%). It is a signal for the person deciding, never a decision.
The call: Strong hire from 85, Hire from 70, Lean no from 50, No hire below. Strong hire needs the issue solved and no behavior under 50; any red flag (pasted debrief answers, tests edited while failing, an unsupervised agent, tests never run) holds the call at Hire, and a high-severity one (only test files changed, an integrity flag) at Lean no. A round that could not be graded, or recorded too little, gets no call. A behavior that was not observed is left out of the average rather than scored zero.
Raised from editor and proxy events, shown with timestamps, and never applied to the score automatically except where a dimension above says so.