Australia Crawler - Racing Australia

Last reviewed: 2026-08-08 AEST

Module identity

• Country: Australia
• Program/source name: Racing Australia
• Adapter source ID: au_racing_australia
• Python module: src\formcrawler\sources\au\racing_australia.py
• Launchers: scripts\crawl_racing_australia_meeting.bat, scripts\crawl_racing_australia_range.bat, scripts\crawl_racing_australia_results.bat, and scripts\crawl_racing_australia_today.bat
• Discipline: Thoroughbred (T)
• Racing jurisdiction code: AUS
• ISO country code: AU
• Allowed hosts: racingaustralia.horse, www.racingaustralia.horse

Scope and key pages

The adapter discovers meetings from Racing Australia's seven-day homepage index. Every requested date maps to its Monday Home.aspx?date=D/M/YYYY index, so one control request is reused per week and meetings are filtered to the exact inclusive dates. The default lifecycle workflow captures a bounded five-page bundle for each meeting. Results-only operation applies a page-type filter before queue insertion and creates exactly one Results.aspx target per meeting. A known meeting can also be seeded from a supported FreeFields page containing the same Key value; the adapter normalizes that meeting and uses its race program as the early-published control.

Page	Public route	Phase	Artifact type
Weekly meeting index	/Home.aspx?date=D/M/YYYY	discovery	meeting_index_html
Race program	/FreeFields/RaceProgram.aspx?Key=<meeting-key>	pre_race	race_program_html
Acceptances	/FreeFields/Acceptances.aspx?Key=<meeting-key>	pre_race	acceptances_html
Gear changes	/FreeFields/GearChanges.aspx?Key=<meeting-key>	pre_race	gear_changes_html
Scratchings	/FreeFields/Scratchings.aspx?Key=<meeting-key>	pre_race	scratchings_html
Results	/FreeFields/Results.aspx?Key=<meeting-key>	post_race	results_html

The meeting key must contain date, state, and venue in the source format, for example 2026Aug07,NSW,Coffs Harbour. Supported state codes are ACT, NSW, NT, QLD, SA, TAS, VIC, and WA.

Capture workflow

For a date range, FormCrawler maps each date to the Monday weekly index, captures each distinct index once, validates the seven-day meeting list, and filters meeting keys to the exact inclusive dates. Without a page filter, every retained meeting fans out to the five lifecycle targets. With --page-type results, filtering happens before targets enter the queue, so only one result page per meeting is retained.

For a known meeting, FormCrawler validates the HTTPS host, supported /FreeFields seed route, single Key, meeting date, state, and venue; it then converts the seed to the canonical race-program control. Supported seed pages include program, acceptances, results, form, all-form, nominations, weights, gear, and scratchings routes.

All workflows deduplicate targets in the persistent queue, capture ready pages through the shared 15-25-second host limiter, store immutable HTML/metadata/handoffs, and leave unpublished lifecycle placeholders retryable. Interactive status uses the shared [HTTP], [PROXY], [DISCOVERY], [RANGE], and [QUEUE] states; permanent summaries use stable key=value records.

Source and artifact data structure

The meeting index and each lifecycle artifact are complete server-rendered HTML. An index target uses page_type=meeting_index, capture_phase=discovery, artifact_type=meeting_index_html, and the Monday date as YYYYMMDD. Each meeting target includes:

• date as YYYYMMDD;
• state;
• venue;
• meeting_key;
• one of the five page_type, capture_phase, and artifact_type combinations above.

Readiness checks require the requested venue and displayed meeting date. Program and acceptances pages must expose Race 1 and their publication marker. Gear, scratchings, and results pages must expose their corresponding published content; results additionally require finish and margin headings. A page that says results are not currently available is rejected and remains retryable.

Local Thoroughbred storage follows:

C:\Users\Robert\project-data\FormCrawler\t-artifacts\<year>\AUS\au_racing_australia\<event>\
C:\Users\Robert\project-data\FormParser\inbox\T\<year>\AUS\au_racing_australia\<artifact-id>.json

How to run

Capture every lifecycle page for every indexed meeting in an inclusive range:

.\scripts\crawl_racing_australia_range.bat 2026-08-07 2026-08-08

Capture results only for an inclusive range:

.\scripts\crawl_racing_australia_results.bat 2026-08-01 2026-08-07

The optional third value is a per-date target limit. Use 0 for discovery only, a positive value for a bounded smoke test, or omit it for a full resumable run. The equivalent generic results-only command is:

python scripts\formcrawler.py capture-date-range `
  --source au_racing_australia `
  --start 2026-08-01 `
  --end 2026-08-07 `
  --page-type results

Capture all five pages from any supported meeting URL:

.\scripts\crawl_racing_australia_meeting.bat "https://racingaustralia.horse/FreeFields/Results.aspx?Key=2026Aug07%2CNSW%2CCoffs%20Harbour"

Capture today's indexed Australian meetings:

.\scripts\crawl_racing_australia_today.bat

Rerun the same command to resume queued or not-yet-ready pages. Use python scripts\formcrawler.py queue-status to inspect the shared queue.

Direct public GET is the default. If it receives HTTP 403 or 429, FormCrawler automatically attempts Robert's Racing and Sports proxy once, retains racingandsports_proxy provenance, and still runs this adapter's validation. --use-racingandsports-proxy or FORMCRAWLER_USE_RACINGANDSORTS_PROXY=1 forces proxy-only retrieval from the first request. POST, private, signed, or credentialed targets are ineligible.

Known limitations and recovery

• This module captures source material; FormParser owns field extraction and canonical JSON mapping.
• Date-range discovery uses the source's seven-day homepage index and filters meetings to exact requested dates; it is not an unrestricted historical calendar API.
• A lifecycle page explicitly saying it is not currently available remains retryable.
• Stop safely with Ctrl+C and rerun the same date range or meeting URL.
• The external FormCrawler\STOP file blocks new requests after a hard stop.
• Proxy failure returns to the existing first-block pause and repeated-block hard-stop path; it does not relax source scope or validation.

Verification

On 2026-08-08, a results-only 2026-08-07 smoke run used an isolated queue database. One 53,994-byte weekly index discovered five meetings and inserted only five results targets. A one-meeting limit captured the 190,581-byte Coffs Harbour result with HTTP 200, zero retries, and zero failures. Its SHA-256 b7062597bc8e307b20cd34ce9c987eba26c5bbe0b3a547939fa405e88726cb72 matched metadata and its FormParser handoff identified the same artifact; four other results remain queued in the isolated resumable smoke database.

The same date's lifecycle range discovered five meetings and 25 lifecycle targets, then captured one bounded Coffs Harbour program with zero retries or failures. The supplied Coffs Harbour meeting also completed all five pages with zero retries and failures: 37,784 bytes for the program, 152,298 for acceptances, 45,614 for gear changes, 34,727 for scratchings, and 190,581 for results. Stored hashes and FormParser handoffs matched. The full project suite passes 89 tests.

Module changelog

2026-08-08 - Results-only range, shared status, and challenge proxy

• Added repeatable date-range --page-type filtering before queue insertion.
• Added crawl_racing_australia_results.bat for full, discovery-only, bounded, and resumable results ranges.
• Added crawl_racing_australia_today.bat for the local date.
• Standardized transient and permanent command-line status records.
• Added automatic proxy fallback for a challenged public GET while preserving source validation and provenance.
• Live-verified five results-only targets, one Coffs Harbour result artifact, its hash and handoff, and the 89-test suite.

2026-08-08 - Weekly index and date-range discovery

• Added one Monday Home.aspx index capture per requested week.
• Filtered discovered meetings to exact inclusive dates and fanned each meeting out to five lifecycle targets.
• Added crawl_racing_australia_range.bat with discovery-only and bounded-run support.
• Accepted Form.aspx, AllForm.aspx, Nominations.aspx, Weights.aspx, and related meeting pages as direct seeds.
• Live-verified five-meeting discovery, one bounded program capture, hashes, and handoffs.

2026-08-07 - Initial implementation

• Added strict Racing Australia meeting-key and route validation.
• Added the five-page meeting bundle with program-led discovery.
• Added readiness checks that reject unavailable placeholders.
• Added crawl_racing_australia_meeting.bat and adapter/workflow tests.
• Live-verified the Coffs Harbour program, acceptances, gear changes, scratchings, and results artifacts.