# Clinic Shortlists and Sources Across ChatGPT Models and Reasoning Efforts

Version 1.0 · Published 2026-08-29 · Rotgar Research

This public package accompanies [Clinic Shortlists and Sources Across ChatGPT Models and Reasoning Efforts](https://rotgar.com/medical/resources/chatgpt-clinic-shortlists-model-effort-stability). It reports a descriptive
stability benchmark built from 450 successful core answers: three exact New
York City commercial prompts × five ChatGPT model/reasoning configurations ×
30 isolated executions. Fifteen technical preflight answers are excluded from
every published denominator.

## Collection surface

The observations came from Codex CLI 0.147.0 authenticated through one ChatGPT
account, with live search available. They did not come from the consumer
ChatGPT Free/Plus interface and did not use API-key billing. New York City was
named in every prompt; physical user location and IP were not controlled.

## Public-data boundary

The package contains audited aggregates, chart data, exact prompts and
configuration labels, plus two sanitized observation tables. Anonymous
`OBS-####` identifiers replace collection run IDs. The package does not contain
raw JSONL, complete answers, stderr, search queries, raw provider strings,
answer fragments, normalized URLs, local paths, authentication state, response
IDs or the private archive.

Visible-source measures use only `final_answer_markdown_link` records. URLs and
registrable domains were deduplicated inside each answer before analysis. The
audited core contains 3,757 visible markdown-link occurrences, 3,690 unique
observation × normalized-URL pairs and 3,194 unique observation ×
registrable-domain pairs. URLs themselves are withheld from this package;
registrable domains are included.

## Interpretation boundary

Presentation position is not a provider-quality ranking. A recurring visible
domain is not evidence that the domain caused a recommendation. The study did
not evaluate clinical quality, factual accuracy, patient outcomes, source
quality, traffic or conversions. Results describe one account, one Mac, one
network context and one August 18, 2026 collection window.

## Reproduction

Run `python3 reproduce_public_metrics.py` from this directory. The script uses
only sanitized public tables to re-check observation counts and selected
headline rates. Use `SHA256SUMS.txt` to verify package bytes.

## Credit and license

Author: Evgeniy Yudin, Founder and Strategy Lead. Methodology reviewer — Boris
Teplyakov, SEO Lead. The methodology-review credit is not a claim of peer
review or medical subject-matter review. Rotgar-owned derived data,
documentation and charts are licensed CC BY 4.0; third-party names and
trademarks remain the property of their owners.