Scraping URL's List

<!-- repository-baseline:start -->

Repository baseline

• Thoroughbred: origin/main at 4d68b42, 25 adapters, 158 passing tests.
• Harness/Greyhound: origin/codex/harness-greyhound-jurisdictions at 490b0c9, eight adapters including LeTROT and separate T/H/G registries, 131 passing tests.
• Both branches are pushed and clean, but are not merged. Check the parent revision before starting a new branch; there is not yet one revision containing all disciplines.
• See overview for the complete country/source Pre/Post/Others matrix. <!-- repository-baseline:end -->

Purpose

This page distinguishes sources that FormCrawler is configured to access from candidate racing websites supplied for future discovery and capture work. Active sources include their executable identifiers, allowed hosts, supported entry points, and implementation status. Candidate sources are reference inventory only and are not approved crawl scope.

The source-adapter registry remains the executable authority. Update this page whenever an active source adapter, allowed host, control URL, supported page pattern, or candidate source catalogue entry is added, changed, disabled, or removed.

Active sites

Status	Source	Source ID	Country	Discipline	Racing code	Canonical site
Active	Racing Australia	au_racing_australia	Australia (AU)	Thoroughbred (T)	AUS	https://www.racingaustralia.horse/
Active; licensed transport pending	Equibase USA	us_equibase	USA (US)	Thoroughbred (T)	USA	https://www.equibase.com/
Active	Harness Racing Australia	au_harness_racing	Australia (AU)	Harness (H)	AUS	https://natsite.harness.org.au/
Active	Harness Racing New Zealand	nz_hrnz	New Zealand (NZ)	Harness (H)	NZ	https://infohorse.hrnz.co.nz/
Active	Standardbred Canada	ca_standardbred_canada	Canada (CA)	Harness (H)	CAN	https://standardbredcanada.ca/
Active	Deutscher Galopp	de_deutscher_galopp	Germany (DE)	Thoroughbred (T)	GER	https://www.deutscher-galopp.de/
Active	HVT	de_hvt_harness	Germany (DE)	Harness (H)	GER	https://www.hvtonline.de/
Active	LeTROT	fr_letrot	France (FR)	Harness (H)	FRA	https://www.letrot.com/
Active	Hipódromo Chile	cl_hipodromo_chile	Chile (CL)	Thoroughbred (T)	CHI	https://hipodromo.cl/
Active	Maroñas	uy_maronas	Uruguay (UY)	Thoroughbred (T)	URU	https://hipica.maronas.com.uy/
Active	Stud Book Argentina	ar_stud_book	Argentina (AR)	Thoroughbred (T)	ARG	https://www.studbook.org.ar/
Active	SNAI Ippica	it_snai_ippica_harness	Italy (IT)	Harness (H)	ITA	https://ippica.snai.it/
Active	Greyhound Racing Ireland	ie_gri	Ireland (IE)	Greyhound (G)	IRE	https://www.grireland.ie/
Active	Kincsem Park Greyhound	hu_kincsem_park_greyhound	Hungary (HU)	Greyhound (G)	HUN	https://mla.kincsempark.hu/racing-days/greyhound/

France LeTROT

Only public HTTPS on letrot.com and www.letrot.com is accepted and canonicalized to www.letrot.com. Supported controls are /courses/YYYY-MM-DD; supported captures are /courses/programme/YYYY-MM-DD/<meeting_id> for efficient whole-meeting lifecycle HTML and /courses/YYYY-MM-DD/<meeting_id>/<race_number> for explicit race-level capture. Qualification, replay, publication and official-programme links are documented but are outside the default crawl.

Deutscher Galopp

Allowed hosts

• www.deutscher-galopp.de
• deutscher-galopp.de

Only HTTPS is accepted. Valid URLs are canonicalized to www.deutscher-galopp.de; allowing a host does not permit unrestricted crawling of every path on that host.

Supported URL patterns

Page type	URL or pattern	What FormCrawler uses it for
Calendar/control page	https://www.deutscher-galopp.de/gr/renntage/	Captures the racing calendar page and discovers in-scope race links when they are present.
Meeting page	https://www.deutscher-galopp.de/gr/renntage/{meeting_id}/?d={YYYYMMDD}	Captures one dated meeting page and discovers its individual race links. {meeting_id} must be numeric.
Individual race page	https://www.deutscher-galopp.de/gr/renntage/rennen.php?id={race_id}&d={YYYYMMDD}&s=R	Captures a pre-race or post-race race page. For the current post-race workflow, s=R is used and the page must show that a result is available before it is accepted.

Verified example

https://www.deutscher-galopp.de/gr/renntage/rennen.php?id=1364507&d=20260802&s=R

This seed URL was used on 2026-08-07 to discover all nine races at the 2026-08-02 meeting. The queue completed with nine captured targets and no failures.

Implementation reference

The registered adapter is src\formcrawler\sources\de\deutscher_galopp.py. It validates the HTTPS scheme, allowed hosts, supported paths, numeric identifiers, date format, and required query values before any capture is attempted.

<!-- usa-equibase-active:start -->

USA Equibase

Allowed host and controls

Only HTTPS on www.equibase.com is accepted. Supported controls are the entries index, summary-results index with SAP=TN, and full-chart PDF index with SAP=TN. Supported leaves are official USA complete-card HTML, complete-summary HTML, full/race chart PDFs, and published race GPS HTML under their exact static paths.

The adapter date-filters meeting links before queue insertion and does not invent GPS URLs. The source is implemented and licensed-browser validated, but unattended direct/proxy transport is currently challenged; production use requires a licensed non-interactive session/API transport. <!-- usa-equibase-active:end -->

<!-- candidate-site-inventory:start -->

Candidate source inventory

Last updated: 2026-08-07

Purpose

This page records candidate racing source websites for FormCrawler discovery and capture work. Inclusion does not mean that a source adapter exists or that the URL has been technically validated.

Sources and conventions

• Base inventory: C:\Users\Robert\Downloads\Site Links.xlsx, supplied on 2026-08-07.
• Denmark (DEN) and Morocco links: supplied directly by Robert on 2026-08-07.
• Disciplines use the FormCrawler codes T (Thoroughbred), H (Harness), and G (Greyhound).
• Proposed source IDs use lowercase ASCII snake case.
• Reuse one source ID when the same website or feed appears in multiple jurisdictions or disciplines and one adapter can own its capture behaviour.
• Split sources when they require independent adapters, allowed-host rules, authentication, or capture behaviour.

Source identity and directories

Every implemented input source needs a stable machine-readable source_id. It is the source-name segment in the artifact path:

<discipline-root>\<year>\<race_code>\<source_id>\<event_slug>-<event_date>\

The implemented German source is recorded as:

• Germany: Deutscher Galopp (de_deutscher_galopp)
  • Directory: t-artifacts\2026\GER\de_deutscher_galopp\

In this example, GER is the controlled racing jurisdiction code and de_deutscher_galopp is the source_id. The identifier names the website, feed, or source definition—not an individual downloaded file.

The IDs below are proposed identifiers for planning. They become authoritative only when the corresponding source definition or adapter is implemented.

Thoroughbred (T)

• Australia: Racing Australia (au_racing_australia)
  • URL: https://www.racingaustralia.horse/
• New Zealand: LoveRacing NZ (nz_loveracing)
  • URL: https://loveracing.nz/Home.aspx
• United Kingdom: Racing Post (gb_racing_post)
  • URL: https://www.racingpost.com/
• Korea: Korea Racing Authority (kr_kra)
  • URL: https://race.kra.co.kr/globalEn/raceCardsSeoul.do
• Malaysia: Selangor Turf Club (my_selangor_turf_club)
  • URL: https://www.selangorturfclub.com/events-calendar/
• Qatar: Qatar Racing and Equestrian Club (qa_qrec)
  • URL: https://qrec.gov.qa/racing/race-calendar/
• Bahrain: Bahrain Horse Racing Authority (bh_bhra)
  • URL: https://bhra.gov.bh/category/season-calendar
• South Africa: Gold Circle (za_gold_circle)
  • URL: http://www.goldcircle.co.za/racing/south-african-horse-racing-calender/
• South Africa: Gallop (za_gallop)
  • URL: https://www.gallop.co.za/fixtures/fixtureIframeLink
• Turkey: Türkiye Jokey Kulübü (tr_tjk)
  • URL: https://www.tjk.org/EN/YarisSever/Info/Page/GunlukYarisProgrami
• Japan (JAR): JBIS (jp_jbis)
  • URL: https://www.jbis.jp/
• Japan (JAR): Umanity (jp_umanity)
  • URL: https://umanity.jp/en/racedata/race_6.php
• Japan (NAR): Umanity (jp_umanity)
  • URL: https://nar.umanity.jp/racedata_schedule.php?year=2017&mon=5
  • Note: the workbook URL contains a 2017 calendar parameter.
• Japan (NAR): NAR Keiba (jp_nar_keiba)
  • URL: https://www2.keiba.go.jp/KeibaWeb/TodayRaceInfo/TodayRaceInfoTop
• Germany: German Racing (de_german_racing)
  • URL: https://www.german-racing.com/
• Italy: SNAI Ippica (it_snai_ippica)
  • URL: https://ippica.snai.it/partenti
• Belgium: Belgian Galopp (be_bgalopf)
  • URL: http://www.bgalopf.be/Meetings.htm
• Argentina: Stud Book Argentina (ar_stud_book)
  • URL: https://www.studbook.org.ar/reuniones
• Chile: AMR Channel (amr_channel)
  • URL: https://www.amrchannel.com/programs
• Chile: Hipódromo Chile (cl_hipodromo_chile)
  • URL: https://hipodromo.cl/
• Czechia: Dostihy JC (cz_dostihy_jc)
  • URL: http://www.dostihyjc.cz/kun.php?ID=52279
• Spain: Jockey Club Español (es_jockey_club)
  • URL: https://jockey-club.es/jornadas/
• Uruguay: AMR Channel (amr_channel)
  • URL: https://www.amrchannel.com/programs
• Brazil: AMR Channel (amr_channel)
  • URL: https://www.amrchannel.com/programs
• Canada: Equibase (ca_equibase)
  • URL: https://www.equibase.com/static/entry/index.html?SAP=TN
• Sweden: Swedish Horse Racing (se_swedish_horse_racing)
  • URL: https://www.swedishhorseracing.com/
• Saudi Arabia: Jockey Club of Saudi Arabia (sa_jcsa)
  • URL: https://jcsa.sa/en/
• Norway: RaceBets (racebets)
  • URL: https://www.racebets.com/en/horse-racing/
• Mauritius: MTC Jockey Club (mu_mtc_jockey_club)
  • URL: https://www.mtcjockeyclub.com/
• USA: Equibase (us_equibase)
  • Entries index: https://www.equibase.com/static/entry/index.html
  • Summary results: https://www.equibase.com/static/chart/summary/index.html?SAP=TN
  • Full-chart PDFs: https://www.equibase.com/static/chart/pdf/index.html?SAP=TN
• Hungary: Kincsem Park (hu_kincsem_park)
  • URL: https://mla.kincsempark.hu/results/trotting/2025-02-08
  • Note: the workbook URL references trotting; it is retained without reclassification.
• Morocco: SOREC Galop (ma_sorec_galop)
  • URL: https://www.sorec-galop.ma/pages/course_a_venir/calendrier_course.jsf?fctID=2nQEdyraO%2Bg%3D
• Netherlands: NDR (nl_ndr)
  • URL: https://ndr.nl/selectieproeven/
• United Arab Emirates: Emirates Racing (ae_emirates_racing)
  • URL: https://emiratesracing.com/
• Hong Kong: Hong Kong Jockey Club (hk_hkjc)
  • URL: https://www.hkjc.com/english/racing/siresearch.asp
• France: France Galop (fr_france_galop)
  • URL: https://www.france-galop.com/en
• India: IndiaRace (in_indiarace)
  • URL: https://www.indiarace.com/
• India: Bangalore Races (in_bangalore_races)
  • URL: https://www.bangaloreraces.com/
• India: Hyderabad Races (in_hydraces)
  • URL: https://www.hydraces.com/
• India: Racing Pulse (in_racing_pulse)
  • URL: https://www.racingpulse.in/code/rp_archives.aspx
• India: DRC Races (in_drc_races)
  • URL: https://www.drcraces.com/
• India: RCTC (in_rctc)
  • URL: https://www.rctconline.com/index.php

Workbook jurisdictions without a supplied Thoroughbred URL: Australia (NSW), Australia (QLD), Australia (WA), Austria, Ireland, Peru, Panama, and Mexico.

Harness (H)

• Australia: Harness Racing Australia (au_harness_racing)
  • URL: https://www.harness.org.au/
• New Zealand: Harness Racing New Zealand (nz_hrnz)
  • URL: https://www.hrnz.co.nz/
• France: LeTROT (fr_letrot) — implemented
  • URL: https://www.letrot.com/
• Denmark (DEN): Paris Turf (fr_paris_turf)
  • URL: https://www.paris-turf.com/
• Denmark (DEN): Swedish Horse Racing (se_swedish_horse_racing)
  • URL: https://www.swedishhorseracing.com/races#/calendar/2022-06-29
• Belgium: Belgium Horse Racing (be_belgium_horse_racing)
  • URL: https://belgiumhorseracing.be/
• Finland: Heppa (fi_heppa)
  • URL: https://heppa.hippos.fi/mobiili/races/results
• Netherlands: NDR (nl_ndr)
  • URL: https://ndr.nl/selectieproeven/
• Norway: RaceBets (racebets)
  • URL: https://www.racebets.com/en/horse-racing/
• Norway: Swedish Horse Racing (se_swedish_horse_racing)
  • URL: http://swedishhorseracing.com/
• Spain: Federació Balear de Trot (es_federacio_balear_trot)
  • URL: https://www.federaciobaleardetrot.com/
• Hungary: Kincsem Park (hu_kincsem_park)
  • URL: https://mla.kincsempark.hu/results/trotting/2025-02-08
• Sweden: Travsport (se_travsport)
  • URL: https://sportapp.travsport.se/start
• Sweden: Swedish Horse Racing (se_swedish_horse_racing)
  • URL: http://swedishhorseracing.com/
• USA: USTA Racing (us_ustrotting)
  • URL: https://racing.ustrotting.com/
• Canada: Standardbred Canada (ca_standardbred_canada)
  • URL: https://standardbredcanada.ca/
• Germany: HVT (de_hvt_harness)
  • URL: https://www.hvtonline.de/
• Italy: SNAI Ippica (it_snai_ippica_harness)
  • URL: https://ippica.snai.it/partenti

Workbook countries without a supplied Harness URL: Austria and Malta.

Greyhound (G)

• Australia: The Dogs (au_the_dogs)
  • URL: https://www.thedogs.com.au/racing/
• Australia: Greyhound Racing Victoria (au_grv)
  • URL: https://www.grv.org.au/
• New Zealand: Greyhound Racing New Zealand (nz_grnz)
  • URL: https://www.grnz.co.nz/catch-the-action/fields.aspx
• United Kingdom: Racing Post Greyhounds (gb_racing_post_greyhounds)
  • URL: https://greyhoundbet.racingpost.com/
• Ireland: Greyhound Racing Ireland (ie_gri)
  • URL: https://www.grireland.ie/racing/upcoming-race-cards/
• Hungary: Kincsem Park Greyhound (hu_kincsem_park_greyhound)
  • Corrected URL: https://mla.kincsempark.hu/racing-days/greyhound/
  • Note: the workbook URL referenced trotting; the implemented greyhound adapter rejects that route.

Maintenance

When a source is implemented, confirm or revise its proposed source_id, then record its allowed hosts, discovery URLs, capture phases, artifact types, and page patterns in versioned non-secret FormCrawler configuration. Do not place credentials or session material in this page. <!-- candidate-site-inventory:end -->

Request controls applying to every configured site

Every discovery, pre-race, post-race, and supplementary request must pass through FormCrawler's central policy layer. Current defaults serialize requests per host and apply a 10-second base delay plus 5-15 seconds of jitter, giving a 15-25-second interval between same-host request starts.

The common policy also applies bounded retries and backoff, response-size limits, circuit breaking, duplicate URL removal, shared local rate-limit state, and the external STOP kill switch. Source-specific settings may be stricter; relaxing a default requires Robert's explicit approval.

Candidate sources are not configured

The candidate inventory above is not a list of permitted crawl targets. Promoting a candidate requires a separate source adapter or explicit source definition, a stable source ID, restricted hosts and paths, capture phase and artifact type, tests, and an update to the active-site section before live requests are enabled.

Maintenance checklist

When adding or changing a source:

1. Record the organization, source ID, country, discipline, and racing code.
2. List exact allowed hosts and require HTTPS unless an explicit exception is approved.
3. List control URLs and supported page patterns; do not describe an entire domain as crawlable when only selected paths are supported.
4. Record whether each endpoint is discovery, pre-race, post-race, or supplementary capture.
5. Add adapter validation and discovery tests before making live requests.
6. Confirm that all requests use the common queue and scraping policy.
7. Update this page and the FormCrawler changelog with the delivered source capability.

Thoroughbred source research status - 2026-08-08

Every Thoroughbred URL supplied in documents/SITE_LINKS.md is now classified as implemented, partial after bounded validation, research-only, document-only, authentication-required, transport-blocked, or outside the Thoroughbred discipline. Jurisdictions without a supplied URL are explicitly marked unstarted. Canonical review links and example horse profiles are maintained in thoroughbred-source-research-2026-08 and the source-specific country pages.