Reasoning review
Multi-step answers, explicit assumptions and factual consistency.
Protocol version
v0.1.0Evidence for the work ahead.
Explore verified customer reports and proposed evaluation protocols. Understand the dataset, execution source and limitations before comparing a score.
Verified evaluation reports
Customer-owned datasets, independently reviewed. Scores, billed USD and units are rechecked against completed platform requests. Fixture runs are explicitly labelled; these are not third-party benchmarks or universal model rankings.
Loading verified results…
The following are proposed protocols, with no published protocol results. These original concepts have not been run. A protocol is not a model score or an independently validated benchmark.
3 protocols
Structured output
Schema adherence and validity of proposed tool arguments.
Protocol version
v0.1.0Visual understanding
Grounded answers about image content, with uncertainty made visible.
Protocol version
v0.1.0PROPOSED EVALUATION · v0.1.0
Reasoning review
Assess whether an answer reaches a supported conclusion and exposes material uncertainty.
Dataset & version
Not publishedA versioned set of self-contained reasoning tasks with reference answers and documented ambiguity. No dataset has been selected or published.
Evaluator & rubric
Proposed · evaluator version not published- Conclusion agrees with the verified reference answer.
- Supporting statements are consistent with the supplied context.
- Assumptions and ambiguous cases are identified without unsupported certainty.
Proposed prompt template
Template variables must be filled from a versioned dataset. This template has not been tested or used for a published run.
Task: {{task}}
Context: {{context}}
Give your answer with a concise justification based on the supplied information. State any assumptions and uncertainty.Run configuration
Model version, route, sampling parameters and trial count are not set. Before execution:
- Freeze the dataset, prompt, model version and provider route before a run.
- Declare sampling parameters, permitted tools and repeated-trial count.
- Retain raw outputs, failures, usage units and USD costs for every request.
- Publish the evaluator version and uncertainty alongside reproducible evidence.
No published results
No evaluation run has been completed or published for this protocol. Quality, USD cost, elapsed time and sample size remain unavailable.
This is an empty publication record, not a failed request or a score of zero.
Open the comparison workbench →Protocol provenance
sandriver-ai original evaluation concept, adapted from the approved benchmark prototype.
Version 0.1.0 describes this proposed protocol only. It is not a dataset version, evaluator release or completed run.
- Source dataset
- Not published
- Run logs
- Not published
- Result publication date
- Not published
- Third-party attribution
- None — original proposal
Limitations
Reference-answer quality, prompt sensitivity and reviewer disagreement can affect findings. A proposed rubric is not a validated scoring system.
A controlled evaluation cannot establish fitness for every production workflow. Keep task-specific validation separate from usage popularity.