Turn quality failures into fixes
Connect
SPIRITT
BraintrustBraintrust
Make AI quality a repeatable operation
Connect Braintrust logs, datasets, experiments, and prompt versions to an evidence-backed release and improvement process. SPIRITT brings the projects, log events, datasets, experiments, and prompts into one working operation, coordinates the next steps across your connected tools, and keeps the outcome in view.
What you can do with Braintrust.
Logs become reproducible examples
Query bounded Braintrust log data and preserve verified failures as dataset examples, with stable identifiers and source context.
Measured results reach reviewers
Create experiments, store actual result events from your test harness, and compare returned summaries with the underlying event evidence.
Prompt configuration stays versioned
Create versioned prompt configurations and maintain their metadata, keeping configuration storage separate from model invocation and release approval.
Works with your other apps.
Get started in three steps.
01
Open your workspace
Create a SPIRITT workspace for your Braintrust operation. Describe the outcome, the records involved, and what finished work should look like.
02
Connect Braintrust
Connect Braintrust with access to the projects, log events, datasets, experiments, and prompts needed for the mission. Connect companion apps for the handoffs you want SPIRITT to run.
03
Delegate a mission
Ask SPIRITT to connect Braintrust logs, datasets, experiments, and prompt versions to an evidence-backed release and improvement process. Set the approval points and cadence, then let it carry the work through.
Hand over a mission.
Choose a mission. Make it yours.
Put Braintrust to work.
Make AI quality a repeatable operation
Questions
Braintrust, with SPIRITT.
01Can SPIRITT turn logs into evaluation datasets?+
Query bounded Braintrust log data and preserve verified failures as dataset examples, with stable identifiers and source context.
02Can it maintain evaluation results?+
Create experiments, store actual result events from your test harness, and compare returned summaries with the underlying event evidence.
03Can it manage prompt versions?+
Create versioned prompt configurations and maintain their metadata, keeping configuration storage separate from model invocation and release approval.
04Does storing a prompt or experiment execute a model?+
Braintrust prompt actions store configuration and metadata; experiment actions store result events. SPIRITT runs the model through your connected application or evaluation harness and records measured outputs. Read queries stay bounded, and stored configuration is not an execution result.
05How do I set up Braintrust with SPIRITT?+
Open a SPIRITT workspace, connect Braintrust, and describe the mission. Choose the projects, log events, datasets, experiments, and prompts it should use and the actions it can take. Add the companion connections for messages, records, or deliverables outside Braintrust.
06How does SPIRITT follow through on Braintrust work?+
Dataset and experiment event IDs connect each review decision to measured evidence. SPIRITT assigns failing cases to engineering and compares later harness results before describing a regression as resolved.