USA Crawler - Equibase

Last reviewed: 2026-08-08 AEST

Identity and scope

• Source ID: us_equibase
• Discipline: Thoroughbred (T)
• Racing code: USA
• ISO country: US
• Module: src/formcrawler/sources/na/equibase.py
• FormCrawler stores raw source responses and parses only the links and identifiers needed for bounded acquisition.

Canada is deliberately separate under ca_equibase and retains its existing TVG-branded static-mirror workflow.

Official source routes

• Entries index: https://www.equibase.com/static/entry/index.html
• Summary-results index: https://www.equibase.com/static/chart/summary/index.html?SAP=TN
• Full-chart PDF index: https://www.equibase.com/static/chart/pdf/index.html?SAP=TN
• Example complete card: https://www.equibase.com/static/entry/DMR080726USA-EQB.html
• Example complete summary: https://www.equibase.com/static/chart/summary/DMR080726USA-EQB.html
• Example full-meeting PDF: https://www.equibase.com/static/chart/pdf/DMR080726USA.pdf
• Example GPS page: https://www.equibase.com/static/chart/gps/DMR080726USA1.html

Acquisition contract

The entries index discovers dated meeting entry indexes. A meeting entry index yields one complete all-races card HTML artifact. The root also traverses the official summary-results and full-chart indexes. Summary meeting indexes yield:

• one complete all-races summary HTML artifact;
• one downloaded chart PDF for every published race link;
• every published GPS HTML page (missing GPS pages are not guessed).

The full-chart index yields the downloaded full-meeting chart PDF. These PDF targets are fetched and stored as immutable binary post-race artifacts with metadata and FormParser handoffs; FormCrawler does not merely record their URLs. Two-level nested discovery keeps date-less control indexes traversable while excluding dated meetings outside the requested range, resolving the earlier broad-index overqueue defect.

PDF acceptance requires application/pdf, a %PDF- signature, a minimum complete-document length, and %%EOF near the end. HTML validators require the matching meeting date and phase-specific card/result/GPS markers.

Information observed and retained

FormCrawler retains these pages raw. The following are source fields observed during the licensed Del Mar validation and are available for later FormParser mapping; FormCrawler does not normalize them.

Pre-race complete card

• meeting/track and race date, race number and scheduled post time;
• race type and conditions, purse, distance, surface/rail and eligibility restrictions;
• program and post-position numbers;
• horse name with country/state suffix and stable horse-profile link;
• age/sex, medication, claiming price, jockey, weight, trainer and morning-line odds;
• owners, breeders, scratches and free-tool/product links;
• pedigree in Sire - Dam, by Dam Sire form, including parent country suffixes where supplied.

Post-race complete summary

• race number, off time, type, age/sex restrictions, purse, distance/surface and track condition;
• winning time;
• program number, horse, jockey and Win/Place/Show returns;
• also-ran and scratched horses;
• wager types, winning combinations and payoffs;
• winning breeder, owner and trainer;
• links to each published race chart PDF and GPS page.

Downloaded chart PDFs and GPS HTML

• one full-meeting chart PDF plus one PDF for every published race chart;
• race conditions, purse/value, distance/surface, weather/track condition, off time, start and final time;
• horse/jockey, weight, medication/equipment, post position, start, intermediate calls, finish, odds and comments;
• fractional and split times, run-up, winner pedigree/foaling/breeder/owner/trainer, claims, scratches, wagering pools/payoffs and running-line detail where supplied;
• each published GPS page is downloaded as raw HTML. GPS is publication-dependent: the Del Mar example exposed races 1-7 but no race-8 GPS link, so race 8 was not inferred.

Command

scripts\crawl_equibase_range.bat us_equibase 2026-08-07 2026-08-07 C:\Users\Robert\project-data

The command is resumable through the shared SQLite queue and uses the common 15-25-second same-host limiter, retries, response limits, circuit breaker, and kill switch.

Breeding and horse detail

Complete cards expose horse-country suffixes, breeder, and pedigrees in Sire - Dam, by Dam Sire form, with parent country suffixes where supplied. Horse names link to stable profile URLs. Profiles can expose exact foaling dates. Example: Warrior of Gold (KY).

Validation status and limitation

Implementation and automated fixtures are complete. The project suite passes 158 tests, including origin restrictions, official index traversal, date filtering, published-only GPS behavior, and binary PDF integrity.

A licensed interactive session validated Del Mar on 7 August 2026:

• complete pre-race card with 8 races;
• complete post-race summary with 8 races;
• 8 race-chart PDFs;
• 7 published GPS HTML pages (race 8 had no GPS link and was not inferred);
• one 10-page full-meeting PDF.

Unattended direct and approved-proxy requests still returned Imperva HTML, including on PDF URLs. Therefore no new canonical artifact or successful unattended full-day run is claimed. Configure a licensed non-interactive session/API transport before production execution; do not weaken PDF validation or treat challenge HTML as data.

Module changelog

• 2026-08-08: Clarified that full-meeting and per-race PDFs are downloaded binary artifacts, and documented the pre-race card, post-race summary, detailed chart, GPS, breeding, and profile information observed in the licensed Del Mar validation.
• 2026-08-08: Replaced the USA mirror assumption with official entries, summary, full-chart, race-chart, and GPS discovery; fixed broad-index overqueue; added 158-test coverage and licensed-browser validation; recorded the remaining unattended transport requirement.
• 2026-08-08: Initial source research and mirror-based representative validation published.