QUESTIONNAIRE 0.7 · 23 SEPTEMBER 2026

Every question.
A recorded decision.

Four model families reviewed the same 84 questions. This record distinguishes their proposed findings from the final, reasoned decisions about the wording.

Content review for descriptive reflection

The review examines wording and options. It does not establish compatibility, actual skill, relationship outcomes or empirical validity with real couples. A retained item means no supported correction remained in this procedure, not that every possible ambiguity has been excluded.

What was reviewed

Each model received the complete frozen v0.6 question bank: 60 broader scenarios and 24 focused scenarios. The same instructions required one retain-or-revise judgment per item. A proposed revision needed an exact quotation, a concrete alternative reading or counterexample, and a minimal repair. Prior findings and other model responses were withheld. The initial audit produced 336 judgments.

OpenAI ran through a fresh Codex agent context. Claude used the displayed Fable 5.1 Extra setting in an incognito conversation; Grok used Expert in a private conversation. Gemini used the API model identifier gemini-3.1-pro-preview. Browser labels are recorded as displayed settings, not independently verified backend snapshots. Exact OpenAI snapshot and browser sampling settings were unavailable.

How decisions were made

Eleven findings concerned eight questions. Two fresh adjudicators, using OpenAI and Gemini, assessed every finding with provider labels withheld. They agreed to accept eight findings and disagreed on three. Repeated findings about one question were not counted as independent defects. The coordinator resolved each finding by checking its concrete reading against the stated facts, preserving both judgments and the reasons for the final decision.

Six questions were revised. Changes preserve item IDs, option IDs, form allocation and scoring. Existing couples retain their stored questionnaire version. New couples receive v0.7; versions are not pooled.

The three initial disagreements remain explicit: whether “preserving the food budget” misleadingly implied a venue-versus-food trade-off; whether the reading-time scenario needed a prescribed number of evenings; and whether requesting a responsibility outline was possible before precise roles were settled. The first received a conservative clarification. The other two were retained because an initial proposal or information request remains meaningful under the stated uncertainty.

Revision and retesting

Four fresh targeted rounds covered six, two, two and one complete questions per family: 44 further judgments, bringing the total to 380. Thirteen additional proposals were adjudicated. Refinements clarified lunch contingencies and dinner participation. Every earlier draft and unfavorable review was retained. These were iterative development rounds, not untouched confirmatory tests.

The final round retained an explicit disagreement: Claude read “decide what to commit to today” as potentially requiring a binding commitment. OpenAI adjudication rejected that inference; Gemini accepted it. The coordinator retained the sentence because deciding today to wait is coherent and expressly available among the options. No payment or venue deadline prohibits waiting. Completion means every observed finding has a reasoned disposition and supported corrections were retested; it does not mean unanimous approval or that all possible defects have been excluded.

Targeted round 1

Complete items: DS-02-02, DS-03-01, DS-03-03, DS-06-02, DS-06-04, PSS-01-04.

  • OpenAI: 6 retained, 0 proposed revisions.
  • Gemini: 4 retained, 2 proposed revisions.
  • Claude: 5 retained, 1 proposed revisions.
  • Grok: 4 retained, 2 proposed revisions.
R01 · DS-03-01Gemini proposal

The stated budget (180 units) easily covers both the room (90) and the stated food cost (70), leaving a surplus. Describing the food outcome as a 'simple lunch' dictated by the 'remaining budget' implies a forced financial compromise or downgrade that isn't mathematically necessary, which acts as accidental coercion against this option.

Evidence quoted: use the remaining budget for a simple lunch

Proposed repair: Reserve the room now and proceed with the planned lunch.

  • OpenAI adjudication: reject. The wording does not state that reserving the room requires a cheaper menu or less food. After the 90-unit reservation, 90 units remain, which covers the stated 70-unit food cost. A 70-unit planned lunch can itself be a simple lunch; that is a concrete reading satisfying both the stem and the option without any downgrade. The finding adds a forced financial compromise that the item does not require.
  • Gemini adjudication: accept. The total budget of 180 minus the 90 room fee leaves 90 units, which easily covers the 70-unit food cost. Describing the outcome as a 'simple lunch' necessitated by the 'remaining budget' implies an unwarranted financial compromise, which may falsely deter respondents who recognize that the planned meal is fully funded.

Coordinator — clarify: Reviewers disagree whether simple lunch necessarily implies a downgrade. Preserve both judgments. Remove this unnecessary evaluative qualifier to explicitly retain the already affordable planned lunch, without changing the decision or budget.

R02 · DS-06-02Gemini proposal

This is a double-barreled option that forces respondents who want to maintain the original meal schedule to also accept an inhospitable social condition. A host could choose to hold the food and sit with the arriving guest to keep them company without necessarily eating a second full meal themselves.

Evidence quoted: without the hosts sitting with the later guest for another meal

Proposed repair: Keep the original meal time and prepare a fresh plate to serve the later guest when they arrive.

  • OpenAI adjudication: reject. The prompt asks respondents to choose an arrangement, and keep_time explicitly specifies both timing and whether the hosts sit for another meal. That is a coherent arrangement rather than an internal contradiction. Hosts can welcome and converse with the late guest while the guest eats, then spend the remaining evening together, without sitting with that guest for another meal. Calling this necessarily inhospitable imports a social judgment. A respondent preferring another combination does not establish a wording defect; the alternatives need not enumerate every possible hosting arrangement, and deleting the qualifier would remove an explicit distinction from staggered_meal.
  • Gemini adjudication: accept. The clause 'without the hosts sitting with the later guest for another meal' conflates the logistical choice of keeping the original meal time with a specific social behavior. A respondent could choose to keep the original meal time for the first guest while still sitting with the late guest to keep them company without eating a second full meal, rendering the option inappropriately double-barreled.

Coordinator — clarify: Both reviewers accept R05 scope ambiguity but disagree on the repair. Preserve original table-participation contrast rather than require hosts to eat twice. State table participation explicitly and establish nearby seating permits shared conversation. This addresses an avoidable inclusion inference without deleting the goal or reverting to overlapping options. Normative judgments that not sitting at the table necessarily means exclusion are not treated as facts.

R05 · DS-06-02Claude proposal

This clause is the only feature separating keep_time from staggered_meal ("with the hosts joining each group at the table"), and it admits two readings: (a) the hosts do not eat a second meal, or (b) the hosts do not sit with the later guest while that guest eats. A respondent who would serve on time, bring the latecomer a fresh plate, and keep them company at the table without eating again maps to keep_time under reading (a) but to staggered_meal under reading (b). Under reading (b) keep_time also reads as leaving the later guest to eat alone, which conflicts with the stem's stated aim that both friends feel included and could deter selection for a reason unrelated to the intended distinction (single meal time versus a second seated meal).

Evidence quoted: without the hosts sitting with the later guest for another meal

Proposed repair: In keep_time replace the clause with "without the hosts eating a second meal", and in staggered_meal change "with the hosts joining each group at the table" to "with the hosts eating with each group", so the two options differ only on whether the hosts take a second seated meal.

  • OpenAI adjudication: accept. The phrase "sitting with the later guest for another meal" leaves a genuine scope ambiguity: "another meal" can mean another meal eaten by the hosts or another seated occasion while only the guest eats. As a concrete counterexample, hosts who serve on time and later sit with the late guest without eating again can understand themselves as avoiding another meal while also joining each arrival group at the table. The inclusion claim is not independently established: hosts can include the guest through conversation and shared time without sitting at the table. The proposed repair would additionally require the hosts to eat twice in staggered_meal, whereas the existing option specifies joining at the table only. Clarify table participation, the distinction directly expressed by the existing options, without adding a second food-consumption requirement.
  • Gemini adjudication: accept. The finding accurately identifies a material ambiguity. The phrase 'without the hosts sitting with the later guest for another meal' in 'keep_time' can be interpreted as leaving the latecomer to eat alone, which directly conflicts with the stem's premise that the hosts want both friends to feel included. Clarifying that the distinction centers on whether the hosts actually eat a second meal resolves the issue and preserves the intended comparison.

Coordinator — clarify: Both reviewers accept R05 scope ambiguity but disagree on the repair. Preserve original table-participation contrast rather than require hosts to eat twice. State table participation explicitly and establish nearby seating permits shared conversation. This addresses an avoidable inclusion inference without deleting the goal or reverting to overlapping options. Normative judgments that not sitting at the table necessarily means exclusion are not treated as facts.

R03 · DS-03-01Grok proposal

The stem already prices food at 70 and the room at 90 against a 180 budget, so reserving the room still funds the stated food cost. Adding “simple lunch” can be read as requiring a cheaper menu. A respondent who would reserve the room and keep the 70-unit food plan cannot map that preference onto reserve_room without also accepting an unintended food downgrade.

Evidence quoted: use the remaining budget for a simple lunch

Proposed repair: Change the option to: “Reserve the room now and pay for food from the remaining budget.”

  • OpenAI adjudication: reject. Nothing identifies the original 70-unit menu as elaborate or says that simple lunch means a cheaper replacement. Reserving the room and spending the stated 70 units on that same simple lunch fits the option exactly and leaves 20 units. Thus the asserted inability to select reserve_room without accepting a food downgrade is not established by the wording. A possible stylistic association with simplicity is insufficient to demonstrate a material unintended requirement.
  • Gemini adjudication: accept. Consistent with R01, adding the 'simple lunch' modifier implies a forced downgrade to the menu despite the budget leaving a 20-unit surplus. This forces respondents to accept an unintended alteration to the food plan just to select the room reservation strategy.

Coordinator — clarify: Reviewers disagree whether simple lunch necessarily implies a downgrade. Preserve both judgments. Remove this unnecessary evaluative qualifier to explicitly retain the already affordable planned lunch, without changing the decision or budget.

R04 · DS-06-02Grok proposal

The stem states an inclusion goal while keep_time is defined as not sitting with the later guest for another meal. Choosing for those hosts can then mean discarding keep_time because it appears to violate a given goal, not because of a serving-time preference. A respondent who would keep the original meal time but still sit with the late guest over the fresh plate also cannot map that arrangement onto keep_time or onto staggered_meal, which requires separate seated meals for each arrival group.

Evidence quoted: Dev and Rowan want both friends to feel included

Proposed repair: Delete “Dev and Rowan want both friends to feel included, and” from the stem, and end keep_time at “prepare a fresh plate for the later guest.”

  • OpenAI adjudication: reject. Wanting both friends to feel included does not require the hosts to sit for a second meal. For example, the late friend can receive a fresh plate, be welcomed into conversation, and share the remaining hour with everyone; keep_time does not prohibit those actions. Whether this feels sufficiently inclusive is a legitimate contextual or preference judgment, not a contradiction. The absence of a separate option for every combination of serving time and seated company is also not a defect in a choice among specified arrangements. The proposed deletion would remove a stated motivation and an intentional behavioral distinction without proving either impossible or ambiguous.
  • Gemini adjudication: accept. The explicit premise in the stem that the hosts 'want both friends to feel included' directly conflicts with the option's stipulation that they will not sit with the later guest. This internal contradiction requires respondents to discard the 'keep_time' option simply because it violates a stated goal, rather than evaluating the scheduling approach itself.

Coordinator — clarify: Both reviewers accept R05 scope ambiguity but disagree on the repair. Preserve original table-participation contrast rather than require hosts to eat twice. State table participation explicitly and establish nearby seating permits shared conversation. This addresses an avoidable inclusion inference without deleting the goal or reverting to overlapping options. Normative judgments that not sitting at the table necessarily means exclusion are not treated as facts.

Targeted round 2

Complete items: DS-03-01, DS-06-02.

  • OpenAI: 2 retained, 0 proposed revisions.
  • Gemini: 1 retained, 1 proposed revisions.
  • Claude: 1 retained, 1 proposed revisions.
  • Grok: 0 retained, 2 proposed revisions.
S01 · DS-06-02Gemini proposal

This stipulation introduces accidental moral ranking by forcing the respondent to select an actively exclusionary or rude behavior. A respondent might logically prefer to serve the main meal at the original time but would still sit with the late guest to keep them company while they eat.

Evidence quoted: without the hosts joining the later guest at the table while that guest eats

Proposed repair: Change the option to: "Keep the original meal time and save a plate for the later guest to eat when they arrive."

  • OpenAI adjudication: reject. The wording excludes joining the late guest at the table, not conversation or company. The stem expressly places nearby seating within conversational distance. A concrete counterexample is the hosts chatting with the late guest from those seats while the guest eats; this satisfies keep_time without being actively exclusionary. Preferring hosts to sit at the table is a legitimate arrangement preference, and staggered_meal already describes hosts joining each arrival group. The proposed removal would erase an explicit distinction between arrangements. A possible ambiguity about continued company is narrower than the claimed forced rude behavior.
  • Gemini adjudication: accept. The finding correctly identifies that the restrictive clause artificially bundles meal timing with potentially exclusionary behavior, creating an unintended moral penalty for selecting the 'original meal time' choice.

Coordinator — reject_broad_claim_clarify_narrow_gap: The stated nearby seating permits conversation, so exclusion is not a necessary implication. Preserve the different hosting arrangements. The narrower S04 ambiguity is repaired rather than deleting the contrast.

S04 · DS-06-02Claude proposal

The option is defined only by a negative clause that says what the hosts do not do, not where they are. Read against the stem's goal that the hosts 'want both friends to feel included', it can be taken as the hosts leaving the later guest to eat alone, which makes it look like the socially deficient option (accidental moral ranking). The apparently intended arrangement, given the stem's statement that 'The table and nearby seating are close enough for everyone to talk together', is that the hosts and the first guest keep talking with the later guest from the nearby seating; a respondent who prefers exactly that may avoid this option and select staggered_meal or later_main instead, even though keep_time is the arrangement they would choose.

Evidence quoted: without the hosts joining the later guest at the table while that guest eats

Proposed repair: Replace the clause with: 'with the hosts and the other guest keeping the conversation going from the nearby seating rather than at the table while that guest eats.'

  • OpenAI adjudication: accept. Proximity establishes that conversation is possible, but does not establish that the hosts actually remain nearby or continue it. Both sitting nearby and chatting, and leaving the late guest alone at the table, satisfy the negative clause. Those readings materially differ in companionship while preserving the same meal timing; this is more than an ordinary unknown about food or personal preference. This does not prove the option is necessarily rude, but supports making the existing nearby-seating interpretation explicit while retaining the off-table distinction from staggered_meal.
  • Gemini adjudication: accept. The finding correctly diagnoses that the negative phrasing makes the option appear socially deficient, penalizing respondents who prefer this meal timing. The suggested fix beautifully resolves the issue by describing an inclusive action using details provided in the stem.

Coordinator — revise: State that hosts and the first guest continue conversation from nearby seating, positively specifying the arrangement while keeping it distinct from staggered table service.

S02 · DS-03-01Grok proposal

The required decision is what to commit to today, but this option can be read either as adopting that contingent policy now (garden if dry; split apartment if rain; no room reservation) or as waiting until rain occurs and only then using the apartment. The second reading can produce the same present choice as keeping the garden plan and reconsidering when the forecast is clearer, so the two options need not pick out different commitments.

Evidence quoted: Commit to the apartment alternative if rain occurs, avoiding the nonrefundable room fee.

Proposed repair: Decide today to use the garden if it is dry and the apartment in two groups if it rains, and do not reserve the indoor room.

  • OpenAI adjudication: accept. The trailing 'if rain occurs' can scope over either the apartment contingency or the act of committing. Under the first reading, the hosts adopt the apartment fallback today; under the second, they retain discretion now and commit only when rain occurs. The stem's request for today's decision favors the first reading, but does not eliminate the second because wait_forecast explicitly permits deferring commitment. Thus two respondents can select home_fallback while reporting different present commitments, a material difference for these options.
  • Gemini adjudication: accept. The phrasing 'Commit to [X] if [Y] occurs' is genuinely ambiguous regarding whether the commitment is made today (as a conditional plan) or delayed until the condition is met. The correction removes this ambiguity.

Coordinator — revise: Explicitly place the contingency commitment today, distinguishing it from deciding later.

S03 · DS-06-02Grok proposal

The option binds a timing/plating choice to a separate seating choice. A respondent who would keep the original sitting and plate the later guest, but would sit with that guest, has no matching option. The non-joining clause also collides with the stem’s statements that the hosts want both friends included and that seating allows everyone to talk together, which can change selection for reasons other than meal timing.

Evidence quoted: Keep the original meal time and prepare a fresh plate for the later guest, without the hosts joining the later guest at the table while that guest eats.

Proposed repair: Keep the original meal time for those already present and prepare a fresh plate for the later guest.

  • OpenAI adjudication: reject. The prompt asks for an arrangement, not an isolated meal-timing preference, so specifying both timing and host seating is not inherently an unintended interpretation. The claimed missing response is substantially supplied by staggered_meal: serve the first arrival group, then serve the later arrival group with the hosts joining each at the table. Nor does remaining off the table contradict inclusion: the stem explicitly permits everyone to talk from nearby seating. The proposed deletion allows keep_time to describe the same host-joining arrangement as staggered_meal and removes the stated distinction. The narrower question of whether off-table hosts remain in conversation does not establish the finding's claimed contradiction or missing-option defect.
  • Gemini adjudication: accept. The finding accurately notes that the option forces a respondent to accept a specific, potentially exclusionary seating behavior to choose a specific meal timing, which introduces a confounding double-barreled element.

Coordinator — reject_broad_claim_clarify_narrow_gap: The stated nearby seating permits conversation, so exclusion is not a necessary implication. Preserve the different hosting arrangements. The narrower S04 ambiguity is repaired rather than deleting the contrast.

Targeted round 3

Complete items: DS-03-01, DS-06-02.

  • OpenAI: 2 retained, 0 proposed revisions.
  • Gemini: 1 retained, 1 proposed revisions.
  • Claude: 1 retained, 1 proposed revisions.
  • Grok: 1 retained, 1 proposed revisions.
T01 · DS-06-02Gemini proposal

The scenario explicitly states there are exactly two friends arriving at different times. Referring to a single friend as an 'arrival group' is unnatural wording that mildly obscures how the hosts would coordinate the seating.

Evidence quoted: each arrival group

Proposed repair: Change 'each arrival group, with the hosts joining each group' to 'each guest as they arrive, with the hosts joining them'.

  • OpenAI adjudication: reject. The stem establishes two guests arriving separately, so each arrival group is readily identifiable as one guest. For example, the hosts can sit with the first guest and later sit with the second guest; nothing in the wording requires either group to contain multiple guests. The phrase is slightly unnatural but does not introduce a material alternative arrangement, impossible option, or genuinely ambiguous referent.
  • Gemini adjudication: reject. The finding critiques the phrase 'arrival group' as unnatural wording for single guests. Evidence: The text 'each arrival group' refers to the two friends arriving separately. Counterexample: Interpreting 'arrival group' as a single person does not alter the logistical meaning, introduce a contradiction, or create an impossible option. The instructions explicitly forbid rewriting for style.

Coordinator — reject: Both blinded adjudicators found a stylistic preference without a material alternate interpretation. Each separate arrival is identifiable; retain the wording.

T03 · DS-03-01Claude proposal

The option does not state where the lunch is held once the room is paid for. Reading A: the room is a nonrefundable rain backup and the garden is still used if Sunday is dry. Reading B: the lunch moves into the room regardless of weather. A respondent who wants the backup approach finds a matching option only under Reading A; under Reading B no option combines reserving the room with keeping the garden, so that respondent may fall back to 'wait_forecast' or 'context' instead. Two respondents with the same underlying preference can therefore land on different options depending on how they read this line, and two with different preferences can land on the same one.

Evidence quoted: Reserve the room now and proceed with the planned lunch.

Proposed repair: Replace with: "Reserve the room now as a rain backup and still use the garden if Sunday is dry." (If the intended meaning is instead to move the lunch indoors, replace with: "Reserve the room now and hold the lunch there whatever the weather.")

  • OpenAI adjudication: accept. Reserving the room does not specify whether it is the definite venue or only a rain backup, and proceeding with the planned lunch can retain the garden location stated in the stem. For example, someone who wants to secure an indoor alternative while using the garden if dry can select this option under the backup reading but decline it under the definite-indoor reading. The wording therefore admits a material unintended interpretation. This is the same defect as T02, not an additional independent defect. The correction below consistently uses the definite-indoor interpretation proposed in T02; the original wording alone does not establish which venue policy the author intended.
  • Gemini adjudication: accept. This finding highlights the same genuine ambiguous referent as T02. Evidence: The phrase 'proceed with the planned lunch' fails to specify the final venue. Counterexample: Two respondents with identical preferences for a rain backup could choose different options depending on whether they read this as a definitive move indoors or a backup plan, resulting in unintended material overlap with other fallback options.

Coordinator — revise: Both adjudicators accepted the same backup-versus-definite-venue ambiguity. The coordinator selects the explicit rain-backup interpretation offered in T03: it preserves the original question about paying now to secure an alternative under uncertain weather. The definite-indoor wording suggested by adjudicators would also resolve ambiguity but changes that contingency strategy. No scoring or option ID changes.

T02 · DS-03-01Grok proposal

The stem locates the planned lunch in a public garden. A reader can take this option as paying the nonrefundable indoor fee while still holding the garden lunch, or as an inconsistent instruction, and so may not map a preference for a definite indoor gathering onto reserve_room.

Evidence quoted: Reserve the room now and proceed with the planned lunch.

Proposed repair: Change the option text to: Reserve the indoor room now and hold the lunch there.

  • OpenAI adjudication: accept. The planned lunch is explicitly in the public garden, while this option only commits to reserving the room. For example, a respondent could pay 90 units to secure a rain backup and still eat in the garden in dry weather, whereas another could understand the same choice as committing to an indoor lunch. Both readings fit the words and the budget. This is a material ambiguity about the arrangement, although paying for a backup while retaining the garden plan is not itself inconsistent.
  • Gemini adjudication: accept. The finding correctly identifies a genuine ambiguous referent. Evidence: The option says to 'proceed with the planned lunch' after reserving an indoor room, but the stem defines the planned lunch as being 'in a public garden'. Counterexample: A respondent might interpret this as paying for a backup room but still using the garden, while another might interpret it as moving the lunch indoors entirely.

Coordinator — revise: Both adjudicators accepted the same backup-versus-definite-venue ambiguity. The coordinator selects the explicit rain-backup interpretation offered in T03: it preserves the original question about paying now to secure an alternative under uncertain weather. The definite-indoor wording suggested by adjudicators would also resolve ambiguity but changes that contingency strategy. No scoring or option ID changes.

Targeted round 4

Complete items: DS-03-01.

  • OpenAI: 1 retained, 0 proposed revisions.
  • Gemini: 1 retained, 0 proposed revisions.
  • Claude: 0 retained, 1 proposed revisions.
  • Grok: 1 retained, 0 proposed revisions.
U01 · DS-03-01Claude proposal

The stem presents a binding commitment today as required, yet two options are explicit deferrals: wait_forecast ("reconsider when the forecast is clearer") and conditional_invite ("then delay spending"). A respondent who reads the stem as ruling out deferral may treat those options as non-compliant with the scenario and shift a wait-and-see preference to home_fallback, reserve_room, or context; another respondent reads "Keep the garden plan for now" as itself a commitment and selects wait_forecast. The same underlying preference can therefore map to different options depending on how strictly the stem sentence is read.

Evidence quoted: They need to decide what to commit to today.

Proposed repair: Replace "They need to decide what to commit to today." with "Today they are deciding whether to make any commitment now or hold off." so that deferral options are clearly admissible under the stated decision.

  • OpenAI adjudication: reject. The stem requires a decision today about what to commit to, but does not require a binding reservation, expenditure, or final venue choice today. The explicit deferral options establish that holding off is an available response. For example, Toni and Gray can decide today to keep the garden plan provisionally and reconsider later, accepting that someone else may reserve the room; this satisfies the stated decision without contradicting any deadline. They can likewise notify guests today of possible postponement while delaying spending. The finding's claim that a binding commitment is required adds a restriction absent from the wording. The proposed clarification is optional rather than a correction of a material contradiction or unintended interpretation.
  • Gemini adjudication: accept. The phrase 'decide what to commit to today' implies a requirement to make a firm or binding choice immediately. This could lead respondents who prefer to wait to incorrectly eliminate the 'wait_forecast' and 'conditional_invite' options because they do not involve making a commitment today. The proposed minimal fix clarifies that deciding to hold off is a valid outcome of today's decision process.

Coordinator — retain_with_dissent: OpenAI rejected and Gemini accepted the proposed implication of a binding commitment. The coordinator retains the item: deciding today what to commit to does not require paying, reserving or selecting a final venue today. Both explicitly offered deferral options are coherent actions today, accepting possible loss of room availability. No scenario deadline prohibits waiting. The strict reading is recorded as reviewer dissent, not silently removed; adding the suggested sentence would be optional clarification rather than a supported correction under the stated rubric.

All 84 item dispositions

Open an item to see each family’s initial judgment and any proposed finding. “Revise” in a model’s review is a proposal, not the final adjudication. Final wording is available in the question library.

DS-01-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-01-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-01-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-01-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-01-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-02-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-02-02Revised for v0.7
  • OpenAI: retain.
  • Claude: revise.

    The stem names two technicians (the one who quoted the repair and 'Another technician' who can assess tomorrow). If 'the technician' is read as the second one, this option becomes a component of the second assessment rather than a distinct evidence source, and a respondent who wants a second opinion cannot tell whether to map to 'condition' or 'second_quote'.

    Evidence quoted: Ask the technician about the condition of other expensive components.

  • Gemini: retain.
  • Grok: retain.

F04 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

DS-02-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-02-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-02-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-03-01Revised for v0.7
  • OpenAI: retain.
  • Claude: revise.

    Under the stated numbers (budget 180, room 90, food 70) reserving the room still leaves 20 units after food, so no option puts the food budget at risk. The phrase implies a food-versus-venue tradeoff the stem rules out and may steer respondents away from 'reserve_room' on a false premise; the actual saving is the nonrefundable room fee.

    Evidence quoted: preserving the food budget

  • Gemini: retain.
  • Grok: retain.

F05 — revise: The literal phrase can be true, as reviewer A notes; the defect is the contrast implied by presenting food-budget preservation as the benefit of only the apartment option when both venues cover the stated food cost. Name the actual fee avoided. This is a conservative clarification of an existing trade-off, not a claim that the original arithmetic was impossible. Reviewer disagreement remains preserved.

DS-03-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-03-03Revised for v0.7
  • OpenAI: revise.

    The phrase "less space and coordination" can mean accepting less coordination, or accepting less space plus the burden of coordinating with another maker. Those readings reverse a relevant cost of sharing: someone willing to coordinate may select it, while someone seeking reduced coordination may select the same wording for the opposite reason.

    Evidence quoted: Ask another maker to share the stall, accepting less space and coordination.

  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

F01 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

DS-03-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-03-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-04-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-04-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-04-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-04-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-04-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-05-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-05-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-05-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-05-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-05-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-06-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-06-02Revised for v0.7
  • OpenAI: revise.

    The stem does not establish that the later arrival precedes the other guest's fixed departure by enough time to share a main meal. For example, an originally one-hour meal leaves no overlap after a one-hour delay, whereas an originally three-hour evening makes this option feasible. Thus the option may represent either a viable preference for a shared main course or an impossible arrangement under equally consistent readings.

    Evidence quoted: Offer a substantial starter first and delay the main meal until everyone is present.

  • Claude: revise.

    A host who serves the first guest at the original time and then sits with the later guest over a freshly prepared plate satisfies both this option and 'staggered_meal' ('Serve seated individual meals to each arrival group, with the hosts joining each group at the table'). The only stated difference is the hosts' presence at the second plate, which 'keep_time' leaves unspecified, so the two options overlap for that respondent.

    Evidence quoted: Keep the original meal time and prepare a fresh plate for the later guest.

  • Gemini: retain.
  • Grok: retain.

F02 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

F06 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

DS-06-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-06-04Revised for v0.7
  • OpenAI: revise.

    "Replacement route" is not assigned to a specific route in the stem. It can mean the original service with a bus change and twenty-minute delay, or the alternative direct service reached by walking. Under the second reading this overlaps the shared_route response; under the first it expresses trying the disrupted service before switching.

    Evidence quoted: Try the replacement route for a week before changing the wider routine.

  • Claude: revise.

    'Replacement route' is not defined in the stem, which names the change-of-buses route required by the road project and an 'alternative direct route'. If read as the alternative direct route, this option duplicates 'shared_route' with a review added; if read as the change-of-buses route, it is a distinct wait-and-see approach. Respondents may map differently depending on which reading they adopt.

    Evidence quoted: Try the replacement route for a week before changing the wider routine.

  • Gemini: retain.
  • Grok: revise.

    The stem never names a 'replacement route'; it names a required change of buses and three alternatives (walk to another direct route, separate services, time-shifted express). A respondent who wants to trial walking to the other direct service can map that preference onto both shared_route and trial_week, so the options overlap under one reading of the undefined label.

    Evidence quoted: Try the replacement route for a week before changing the wider routine.

F03 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

F07 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

F10 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

DS-06-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-07-01Retained from v0.6
  • OpenAI: retain.
  • Claude: revise.

    The number of evenings Ellis wants is unstated while the stem specifies Morgan wants two evenings and both have the window free on four weekdays. If Ellis wants two evenings, no conflict remains and 'alternating_evenings' is a complete solution rather than a negotiated proposal, making 'split_windows' and 'alternative_mix' unmotivated; if Ellis wants all four evenings, every proposal involves a real trade. This unstated condition changes which proposal a respondent would put forward.

    Evidence quoted: Ellis wants uninterrupted reading time in the same room

  • Gemini: retain.
  • Grok: retain.

F08 — retain: Retain: the prompt asks for an initial allocation proposal, not the only solution to an established shortage. Different evening allocations remain meaningful without inventing how many evenings Ellis desires. Unknown frequency is a legitimate reason to request context. Preserve reviewer B dissent.

DS-07-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-07-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-07-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-07-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-08-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-08-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-08-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-08-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-08-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-09-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-09-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-09-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-09-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-09-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-10-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-10-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-10-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-10-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-10-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-11-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-11-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-11-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-11-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-11-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-12-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-12-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-12-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-12-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

DS-12-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-01-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-01-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-01-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-01-04Revised for v0.7
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: revise.

    The prompt requires a first response to the host, but this option addresses a 'trusted event organizer' who is not present in the stem (only staff who secured the pieces are mentioned). A respondent who wants a delayed, mediated conversation cannot record that preference without ignoring the prompt or inventing an actor; a respondent who treats the option as unavailable is forced onto the other three host-directed moves.

    Evidence quoted: Ask the host's trusted event organizer to help arrange a quiet conversation after the speech.

F11 — revise: Both independent adjudicators identified a concrete ambiguity, feasibility gap, overlap or prompt-addressee conflict. Apply the minimal content repair; retest the complete revised item in fresh contexts.

PSS-01-05Retained from v0.6
  • OpenAI: retain.
  • Claude: revise.

    The stem states that 'the partners' precise responsibilities will not be settled for another week', so this option requests something the stem says cannot be supplied before the deadline. A respondent cannot tell whether selecting it means decline-if-not-provided, accept-anyway, or ask for a partial outline, so the same intended preference could map to this option or to 'decline_this_round'.

    Evidence quoted: Ask for a concrete responsibility outline before tomorrow's decision deadline.

  • Gemini: retain.
  • Grok: retain.

F09 — retain: Retain: requesting an outline is a feasible first action; an outline need not settle precise responsibilities. A failed request can inform a later decision. The prompt deliberately leaves those later actions open. Preserve reviewer B dissent.

PSS-01-06Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-02-06Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-03-06Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-01Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-02Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-03Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-04Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-05Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

PSS-04-06Retained from v0.6
  • OpenAI: retain.
  • Claude: retain.
  • Gemini: retain.
  • Grok: retain.

No initial reviewer proposed an actionable change to this item.

Strengths and the scope of the evidence

Explicit coverage makes omissions visible. Frozen inputs, raw responses and exact quotations make decisions traceable. Fresh contexts and different model families expose disagreements. Versioned corrections protect historical answers. Failed checks remain part of the record. These are practical strengths of the review process; they do not establish historical priority or superiority over human validation.

The preceding fictional-couple exercise supplied 1,344 initial answers and 197 actual AI interview replies, covering all 84 questions but only 119 of 420 selected question-option combinations in interviews. It is a separate phase, not a four-family replication or a human sample. Cross-model interviewer qualification had failures, and the full planned candidate/interviewer matrix remains unexecuted. Future population claims require evidence appropriate to those interpretations.

Read the full methodology and earlier phases →

Understand what your results mean →