How many competencies should we score?

Enough to cover role-critical behaviors, few enough to keep the assessment focused. Many teams start with a weighted subset rather than an exhaustive laundry list.

Do we need a 1–5 scale?

Use a scale your reviewers can apply consistently. Behaviorally anchored levels matter more than the number of points.

How often should rubrics change?

Change when the job changes or calibration shows ambiguity-then freeze again for the next wave. Avoid mid-wave edits that break candidate equivalence.

How do rubrics relate to validation?

Anchors operationalize job-related competencies. They are central to explainability, human review, and documentation for AI hiring assessments.

Building Competency Rubrics

Pick observable competencies

Prefer ownership, adaptability, customer empathy, and judgment over vague labels like “culture fit” or undefendable traits. If two trained reviewers cannot describe what the competency looks like at work, the model will not magically make it valid. Start from the job: which behaviors distinguish strong performers?

Ownership, judgment, collaboration, customer empathy, execution, coachability
Avoid vague culture-fit proxies that invite bias
Weight competencies by role criticality
Document why each competency is job-related

Write anchors, not vibes

A score of 5 should describe concrete behaviors: personal responsibility, actions taken, outcomes measured, prevention reflected. A score of 1 should describe avoidance, blame, or unrelated examples. Anchors should be specific enough that a new reviewer-and a scoring system-can apply them consistently.

Describe observable actions, decisions, and outcomes
Include what “missing evidence” looks like, not only what excellence looks like
Call out common false positives (fluent but generic answers)
Align anchors to the question prompts candidates will see

Design questions that can elicit the evidence

A brilliant rubric fails if the question cannot elicit the behaviors you want to score. Use structured behavioral and situational prompts that invite specifics: context, actions, constraints, outcomes, and reflection. Plan follow-ups for shallow answers. Freeze the approved set so candidates are comparable.

Pair each competency with prompts designed to surface it
Include consistency probes across the assessment
Avoid leading questions that telegraph the “right” story

Calibrate with humans before you scale AI

Have hiring managers and trained raters score a sample of responses using the anchors. Measure agreement. Revise ambiguous anchors. Only then let automated scoring assist at volume. Calibration is how you keep AI tied to your standard instead of drifting into generic eloquence rewards.

Double-score a calibration set
Discuss disagreements and tighten language
Track ongoing reviewer–model disagreement after launch

Workflow

Keep this loop in mind

Guides go deeper—but the product motion stays the same.

Assessment loop

Select

Role · Customer Success Manager

Customer empathy

30%

Ownership

25%

Communication

20%

Adaptability

15%

Conflict resolution

10%

Integrity gate

Threats get intercepted. Genuine candidates pass through.

A four-scene integrity loop: live session, anomaly detection, layered interception, and verified pass—built for AI-era cheating.

Device & session
Behavior & language
AI assistance

Honrly integrity

Session live

Candidate sessionLive

“When a renewal was at risk, I called the customer the same day and owned the save through Q3…”

Response · 00:14:22

Integrity watch

Identity checkok
Device postureok
Response cadenceok

Comparison

Traditional vs Honrly

Side by side

Primary signal

Traditional

Self-report / black-box score

Honrly

Demonstrated evidence + integrity

Freeze and version

Once approved, freeze the question set and rubrics for a hiring wave so candidates are comparable. Log versions for auditability. If you change anchors mid-funnel, you break equivalence and complicate adverse-impact analysis. Treat freezes as a fairness feature.

Score evidence dimensions explicitly

Beyond a single competency score, it helps to evaluate evidence quality dimensions: Was there a real example? Ownership? Appropriate judgment? Specificity? Reflection? Cross-question consistency? These dimensions make reports explainable and help interviewers know what to probe next.

Evidence, ownership, judgment, specificity, reflection
Competency alignment and cross-response consistency
Integrity observations kept separate from content quality

What not to put in a rubric

Do not encode accent, appearance, “energy on camera,” or facial expressions as competency evidence. Do not smuggle personality stereotypes into anchors. Rubrics should stay tied to job-related, observable workplace behaviors. That boundary protects fairness and scientific credibility.

Connect rubrics to integrity and decisions

A high content score with integrity flags is not the same as a high authentic score. Show both to reviewers. Rubrics tell you what good evidence looks like; integrity tells you whether the evidence is likely the candidate’s. Humans combine those inputs to decide-Honrly does not silently finalize employment decisions.

FAQ

Frequently asked questions

AI can draft. Experts should review and approve. The approved anchors-not a free-form model opinion-should drive scoring.

Keep exploring

Next step

Building behaviorally anchored competency rubrics