Customer Eval pass / fail · no rubric inside
Prompt
You are a discerning staff engineer onboarding to the QTI 3.0 Persistence Package this week. You came to this page to write a migration and a query against qti.tenant and qti.content_package without reading any source code. You've read every page leading up to this one. Benchmark: Stripe's API reference. You have live access to these three pages, one per level of the data dictionary tree: Schema: https://docs.stripe.com/api Object: https://docs.stripe.com/api/charges Field: https://docs.stripe.com/api/charges/object Is this data dictionary the same as or better than Stripe's for the job you came to do, or are there places where it is worse? Evaluate at all three levels: schema navigation, object overview, and field-level reference. Answer yes or no. Point to specific examples on this page and on the corresponding Stripe page. If no, what would have made you say yes?
Scoring Eval binary · weighted · non-overlapping · failure-named
Data structure
Use this tree to locate failures: every category below grades a specific slot.
Schema
- schema metadata (nav, search, object index)
Object
- object metadata (purpose, lifecycle, example payload)
Field
- field metadata (name, type, null/required)
- description (intent + unit + behavior)
- constraints (length, charset, range, scope)
- allowed values (enum + per-value explanation)
- cross-refs (relationship, cardinality, nullability)
- edge cases (deprecation, conditional, lifecycle)
- example
Failure categories
| # | Level | Category (failure-named) | Weight | Applied when (quote-anchored) |
|---|---|---|---|---|
| 1 | Field | C-Field-Description-Useless | 10 | A field description restates the field name or gives a dictionary-style definition instead of telling the reader what the field is for, what unit/format/convention it uses, or how it behaves. Quote one offending row and the corresponding Stripe field description that does the work. |
| 2 | Field | C-Field-Constraints-Absent | 8 | A field has length limits, charset restrictions, range bounds, or scope conditions that the reader needs to write a migration or query correctly, and the field doc does not state them. Quote the field and the constraint that's missing. |
| 3 | Field | C-Field-Values-Unexplained | 7 | An enum field lists allowed values without a one-sentence behavioral explanation per value. Quote the enum list and one value with no explanation. |
| 4 | Field | C-Field-Crossref-Untyped | 6 | A field references another object but doesn't name the
relationship, cardinality, or nullability
(e.g. customer_id without "belongs to one Customer,
nullable"). Quote one such field row. |
| 5 | Field | C-Field-Edge-Case-Silent | 5 | A field exists because of a workflow branch (deprecated, conditionally populated, lifecycle-dependent) but the field doc doesn't name the branch. Quote the field and the silent condition. |
| 6 | Field | C-Field-Example-Missing | 4 | A field has no realistic example value shown inline or in the object's example payload. Quote the field row and the absence. |
| 7 | Field | C-Field-Types-Missing | 4 | Any field row lacks an explicit data type (e.g. UUID,
TEXT, TIMESTAMPTZ, JSONB,
BOOLEAN) or a null/required indicator.
Quote one offending row. |
| 8 | Object | C-Object-Undocumented | 6 | An object has fields but no object-level description, lifecycle note, or example payload. Reader can't tell what the object is before reading its fields. Quote the object header and the absence. |
| 9 | Schema | C-Schema-Unnavigable | 4 | Reader can't find a field, jump to an object, or see the object list without scrolling. No search, no per-object anchor, no nav, no deep-link to a single field. Quote the page state. |
| 10 | Format | C-Format-Below-Stripe | 3 | The page's visual presentation, typography, density, color, spacing, alignment, or interactive affordances make this data dictionary feel less polished, less professional, or harder to read than docs.stripe.com/api. Quote one specific element on this page and the corresponding Stripe element that does it better. |