USA Crawler - Equibase

Last reviewed: 2026-08-08 AEST

Identity and scope

• Source ID: us_equibase
• Discipline: Thoroughbred (T)
• Racing code: USA
• ISO country: US
• Module: src/formcrawler/sources/na/equibase.py
• FormCrawler stores raw source responses and parses only the links and identifiers needed for bounded acquisition.

Canada is deliberately separate under ca_equibase and retains its existing TVG-branded static-mirror workflow.

Official source routes

• Entries index: https://www.equibase.com/static/entry/index.html
• Summary-results index: https://www.equibase.com/static/chart/summary/index.html?SAP=TN
• Full-chart PDF index: https://www.equibase.com/static/chart/pdf/index.html?SAP=TN
• Example complete card: https://www.equibase.com/static/entry/DMR080726USA-EQB.html
• Example complete summary: https://www.equibase.com/static/chart/summary/DMR080726USA-EQB.html
• Example full-meeting PDF: https://www.equibase.com/static/chart/pdf/DMR080726USA.pdf
• Example GPS page: https://www.equibase.com/static/chart/gps/DMR080726USA1.html

Acquisition contract

The entries index discovers dated meeting entry indexes. A meeting entry index yields one complete all-races card HTML artifact. The root also traverses the official summary-results and full-chart indexes. Summary meeting indexes yield:

• one complete all-races summary HTML artifact;
• one chart PDF for every published race link;
• only the GPS HTML pages that Equibase actually links (missing GPS pages are not guessed).

The full-chart index yields the published full-meeting chart PDF. Two-level nested discovery keeps date-less control indexes traversable while excluding dated meetings outside the requested range, resolving the earlier broad-index overqueue defect.

PDF acceptance requires application/pdf, a %PDF- signature, a minimum complete-document length, and %%EOF near the end. HTML validators require the matching meeting date and phase-specific card/result/GPS markers.

Command

scripts\crawl_equibase_range.bat us_equibase 2026-08-07 2026-08-07 C:\Users\Robert\project-data

The command is resumable through the shared SQLite queue and uses the common 15-25-second same-host limiter, retries, response limits, circuit breaker, and kill switch.

Breeding and horse detail

Complete cards expose horse-country suffixes, breeder, and pedigrees in Sire - Dam, by Dam Sire form, with parent country suffixes where supplied. Horse names link to stable profile URLs. Profiles can expose exact foaling dates. Example: Warrior of Gold (KY).

Validation status and limitation

Implementation and automated fixtures are complete. The project suite passes 158 tests, including origin restrictions, official index traversal, date filtering, published-only GPS behavior, and binary PDF integrity.

A licensed interactive session validated Del Mar on 7 August 2026:

• complete pre-race card with 8 races;
• complete post-race summary with 8 races;
• 8 race-chart PDFs;
• 7 published GPS HTML pages (race 8 had no GPS link and was not inferred);
• one 10-page full-meeting PDF.

Unattended direct and approved-proxy requests still returned Imperva HTML, including on PDF URLs. Therefore no new canonical artifact or successful unattended full-day run is claimed. Configure a licensed non-interactive session/API transport before production execution; do not weaken PDF validation or treat challenge HTML as data.

Module changelog

• 2026-08-08: Replaced the USA mirror assumption with official entries, summary, full-chart, race-chart, and GPS discovery; fixed broad-index overqueue; added 158-test coverage and licensed-browser validation; recorded the remaining unattended transport requirement.
• 2026-08-08: Initial source research and mirror-based representative validation published.