Swiftscore Docs

AI Calibration

AI Calibration lets your district shape how Swiftscore's AI writes feedback, scores components, and talks about instruction — so the language it uses sounds like your evaluators, reflects your priorities, and stays consistent across every observer in your district.

Without calibration, Swiftscore's AI uses sensible defaults. With calibration, you're telling it: how formal or warm to sound, how long feedback should be, what instructional priorities to emphasize, which topics to steer around, and which research-based frameworks should inform its language. Every evaluation generated afterward — for every observer using your settings — reflects those choices.

This matters most for districts juggling multiple observers, frameworks, or schools: calibration is what keeps AI-generated feedback consistent and on-message, instead of each evaluator getting a slightly different "voice" from the AI by default.

Two ways to calibrate

Swiftscore gives you two complementary paths to shape the AI:

  1. AI Calibration settings — a settings page where you (or your district admin) directly configure tone, length, focus areas, and override rules. Good for quick, direct control.
  2. The calibration survey — a guided, conversational walkthrough that builds a fuller picture of your district's evaluation philosophy by asking you questions, showing you AI-drafted examples, and refining its understanding through a short chat. Good for a deeper, one-time setup that the settings page can then build on.

Both feed the same underlying calibration profile, and both apply automatically to every AI-generated evaluation, component score, and narrative summary going forward.

Calibration settings

Found under Settings → AI Calibration, this is where day-to-day adjustments live.

What you can configure

  • Tone and voice — how the AI's written feedback should sound (e.g., warmer and encouraging vs. direct and concise).
  • Feedback length — whether component feedback should be brief or more detailed.
  • Focus areas — instructional priorities you want the AI to emphasize when it writes feedback (for example, questioning technique, pacing, or student engagement).
  • Topics to avoid — areas you'd rather the AI not editorialize on.
  • Citations — whether AI feedback should cite supporting evidence or examples, and how those citations are styled.

District-level overrides

District administrators have access to a deeper override panel for fine-tuning how the AI behaves at scale:

  • Toggle whether AI feedback prefers bulleted lists over prose.
  • Toggle whether schools and individual evaluators can further customize calibration on top of district defaults, or whether the district-level settings are locked in.
  • Toggle whether the AI should prioritize your district's stated goals when generating feedback and narrative summaries.
  • Edit style and content preferences directly for advanced, fine-grained control over AI output.

Each setting saves independently, with a clear saved/saving/error indicator so you always know your change has taken effect.

Settings apply by tier

Calibration settings can be set at the organization (district), school, or individual user level. More specific settings take priority — an individual evaluator's personal preferences will be respected over school-level defaults, and school-level defaults will be respected over district-level defaults, unless your district has locked settings at a higher tier.

Carried-over preferences

If your district was using Swiftscore's evaluation feedback before AI Calibration existed, any general feedback preferences you'd already set continue to apply automatically — you don't need to re-enter anything to keep your existing experience.

The calibration survey

The calibration survey, found at Calibration, is a guided nine-step conversation that builds a complete picture of how your district wants AI feedback to sound and what it should prioritize. It's designed to feel like a short onboarding conversation rather than a settings form, and every step you complete is saved automatically as you go — so you can pause and pick back up later from your calibration settings page.

Here's what each step covers:

1. Welcome

A short introduction explaining what the calibration survey does and what to expect, with a one-click way to begin.

2. Documents

Upload reference materials — rubrics, district letters, instructional plans, or other documents that reflect how your district thinks about good instruction. Swiftscore reads these and uses them to ground the feedback it generates. You can drop in multiple files at once.

As part of this step, you can also preview document targeting: a per-teacher view showing which uploaded documents Swiftscore would actually draw from for a given teacher's evaluation, along with a relevance score and the reasons it matched. This helps you confirm your reference materials are being used the way you expect before you finish calibrating.

3. Priorities

Two short, open-ended questions: what's the single biggest instructional priority you want feedback to reinforce, and what kind of feedback actually helps your teachers grow? Your answers ground everything that follows.

4. Orientation

A couple of quick multiple-choice questions about your preferred feedback style — for example, how you like feedback to open, and how long you generally want it to run.

5. Research lenses

Choose from a set of well-known instructional research frameworks and methodologies (drawn from researchers and practitioners widely used in K-12 evaluation) that you want the AI's language and emphasis to be informed by. You can select up to five.

6. Pairwise comparisons

Swiftscore generates a handful of side-by-side feedback examples based on your answers so far, and asks you to pick which version better matches what you're looking for. This is one of the most direct ways to steer the AI's voice — your picks are used to refine the tone and substance of future feedback.

7. Sample sign-off

Swiftscore shows you a few generated feedback examples and asks you to confirm whether each one "looks right" or needs more work. This gives you a final sanity check on the AI's output style before it goes live for your evaluators.

8. Edith conversation

Chat directly with Edith, Swiftscore's AI assistant, to refine your calibration in plain language. You can describe adjustments you want — more specific, less formal, more focused on a particular practice — and Edith responds and helps tighten the profile based on the conversation.

9. Your profile

A final summary of the calibration profile that's been built from your answers — a readable recap of what shapes the AI's feedback for your district, plus an option to keep refining it later.

Where calibration shows up

Once configured, your calibration settings apply automatically and consistently:

  • Every component score the AI generates during an evaluation reflects your tone, length, and focus-area preferences.
  • Domain summaries and overall narrative feedback on evaluations are shaped by the same settings.
  • Settings apply across every observer using your district's account, so feedback stays consistent district-wide rather than varying observer to observer.

You don't need to do anything special when generating an evaluation — calibration runs in the background every time.

Revisiting and refining

Calibration isn't a one-time setup. You can return to Settings → AI Calibration any time to adjust tone, focus areas, or override rules, or return to the calibration survey to refine your profile further — the call-to-action there automatically changes to "Refine your calibration" once you've completed it once, so it's easy to pick back up.

Who can configure what

  • District administrators can configure organization-wide defaults, including the override panel that controls bulleted-list formatting, downstream customization permissions, and goal-prioritization behavior.
  • School-level administrators and individual evaluators can adjust calibration within the bounds their district allows, unless the district has chosen to lock settings centrally.

Getting started

  1. Go to Settings → AI Calibration to set quick defaults, or go to Calibration to take the full guided survey.
  2. If you're starting fresh, we recommend the survey — it walks you through priorities, examples, and a conversation with Edith, and produces a much richer calibration profile than the settings page alone.
  3. Once you've completed either path, generate an evaluation as you normally would. You'll notice the AI's feedback reflects the tone, priorities, and style you configured.
  4. Return any time to refine — calibration is meant to evolve as your district's priorities do.