FormCrawler Changelog

Document set version: 1.0.0

Created: 2026-08-07

Last updated: 2026-08-10 02:14:13 AEST (UTC+10:00)

Maintenance rule: Keep Unreleased cumulative for the current document-set version. After every material FormCrawler change, update this timestamp and the matching timestamps in documents\README.md and handover\HANDOVER.md so the latest documents encompass all changes.

Unreleased

• Added run-day concurrent daily acquisition with one isolated source/phase subprocess, SQLite queue and retained log; configurable 1-32 worker concurrency; and source-round-robin submission while the shared host gate continues to serialize same-site requests.

• Added the loopback FormCrawler Dashboard with run controls, retained history, discipline/source filters, requested-day meeting and file metrics, queue/retry/failure visibility, and bounded worker-log viewing. Published the dedicated formcrawler-dashboard operator guide.

• Completed the first eight-worker run for 10 August 2026 pre-race and 9 August 2026 post-race: 73 requested-phase meeting captures and 91 files totalling 116,881,425 bytes. All 91 passed independent hash, byte-length, metadata identity/URI, discipline/source and FormParser-handoff validation.

• The run settled partial: nine jobs completed, six were partial because requested targets were not yet published, three correctly reported no data, four failed, and four were unavailable. Racing Australia's prior-week mismatch and two HRNZ timeouts remain visible issues; future German Galopp dated jobs are now recorded as manual-seed required instead of launching an unsupported command.

• Added dashboard and operations regression coverage, Windows SQLite cleanup, local API run serialization, requested-day queue metrics, productive-run partial aggregation and packaged assets. Commit 3319e66 compiles and the dashboard-branch suite passes 139 tests.

• Published and read back the complete-day baseline in QDG KB: overview v15, changelog v23, Thoroughbred report v9, Harness/Greyhound report v5, Germany v5, Argentina v4, Chile v5, India v5, Denmark v4, Finland v4, Netherlands v6 and both Japan pages v5. The program and URL inventory pages remain v13/v11 because the public command/route inventory did not change.

• Replaced the bounded re-examination runs with concurrent complete published-day/meeting crawls under C:\Users\Robert\project-data\formcrawler-e2e-20260808. The audited baseline now contains 313 immutable artifacts and 313 FormParser handoffs with zero missing files, byte-length mismatches, SHA-256 mismatches, URI mismatches or permanent failures.

• Complete lifecycle evidence is Germany Düsseldorf 2 August (9 pre/9 post), Argentina La Plata 6 August (one whole 11-race card/result pair), Chile 8 August program plus all 20 results from 6 August, India Kolkata 6 August (one whole meeting pair), Denmark Odense 7 August (9/9), Finland 7 August (30/30 across three meetings), Netherlands latest available Eenrum card/Emmeloord result meeting pair, Japan JRA 2 August (36/36) and Japan NAR 7 August (24/24). Control/index artifacts are included in the 313-artifact audit but not in those lifecycle counts.

• The full Germany run exposed completed s=S pages whose sibling navigation links were rewritten by the site to s=R. de_deutscher_galopp v1.2.0 now preserves the seeded lifecycle while expanding a race meeting; a regression test covers the condition and the complete 9/9 meeting rerun passed.

• Published and read back the audited crawl evidence in QDG KB: overview v14, changelog v21, Thoroughbred report v8, Harness/Greyhound report v4, Germany v4, Argentina v3, Chile v4, India v4, Denmark v3, Finland v3, Netherlands v5 and both Japan Umanity pages v4. Program and URL inventory pages remain unchanged at v13/v11 because no command or source URL changed in this verification-only pass.

• Earlier response-only and bounded smoke evidence remains useful diagnostic history, but it is superseded by the complete-day baseline above. Seven later NDR future racecards and the separate JRA 7 August negative check remain retryable because the source data is not published; they are not incomplete targets from the selected complete-day runs.

• Published and read back the new unified baseline in QDG KB: overview v13, programs v13, changelog v20, URL inventory v11, Thoroughbred report v7, Harness/Greyhound report v3, and updated Germany/Argentina/Chile/India/Denmark/Finland/Netherlands/Japan country pages. Denmark and Finland country pages were created; all page histories remain immutable.

• Re-examined Japan, Germany, Argentina, Chile, India, Denmark, Finland and the Netherlands. Japan's Umanity JRA/NAR adapters remain correct after live calendar checks; Germany, Argentina and Chile now have verified pre-race acquisition; IndiaRace and three national Harness adapters are registered.

• Extended de_deutscher_galopp to traverse rennkalender.php, nested meetings and both s=S pre-race and s=R post-race pages. Extended ar_stud_book to discover the Home/Today and next-meeting cards while retaining guarded historical result calendars.

• Extended cl_hipodromo_chile through the public upcoming-program page and first-party program API, capturing raw meeting JSON and linked official PDFs. The shared fetcher now retains anonymous same-host cookies in process memory for multi-request public workflows; no cookie is persisted in artifacts or metadata.

• Added in_indiarace for current national racecard, results and trackwork meeting HTML. Live cards expose horse links, sire/dam country suffixes and age; result pages expose incidents and ratings.

• Added dk_swedish_horse_racing, fi_heppa_harness and nl_ndr_harness. They capture raw pre/post JSON or HTML through the Swedish Horse Racing, Heppa and NDR first-party index/API contracts. Added seven repeatable launchers for the new and extended routes.

• Live readiness checks passed for Germany Krefeld pre-race HTML (94,394 bytes), Argentina La Plata pre-race HTML (367,139), Chile program JSON/PDF (15,575/5,712,665), IndiaRace pre/post HTML (342,329/409,851), Denmark Odense pre/post JSON (6,155), Finland Härmä pre/post JSON (20,287), NDR Eenrum/Emmeloord HTML (24,719/35,292), and Japan JRA/NAR calendars (87,827/43,255). The full suite passes 208 tests and all modules compile.

• Unified the Thoroughbred and Harness/Greyhound development lines on main in merge commit ab0441e and pushed it to origin/main. The discipline registry now contains all 25 Thoroughbred, six Harness, and two Greyhound adapters; source_ids() remains backward compatible and rejects cross-discipline source-ID collisions. The combined 200-test suite and Python compilation pass.

• Replaced the obsolete split-branch wording in the QDG Knowledge Base and read every changed page/history back: overview v12, changelog v19, Harness/Greyhound report v2, programs v12, URL inventory v10, and the eight H/G country pages at their next versions. Thoroughbred report remains v6.

• Reconciled the complete Thoroughbred, Harness, and Greyhound project state into documents\PROJECT_STATUS.md, with the requested country/source matrix for pre-race, post-race, and other assets. The matrix distinguishes written crawlers (✅), sources that are unavailable or unsuitable under the current contract (❌), and partial/unresolved work (?), and separates captured supplementary assets from links that were only documented.

• Added documents\HARNESS_GREYHOUND_SOURCE_REPORT.md to the main documentation set so the branch investigation, validation counts, supplementary links, and breeding/profile findings have a canonical local report alongside the Thoroughbred report.

• Verified the Git baseline rather than relying on session summaries: origin/main is 4d68b42 with 25 Thoroughbred adapters and 158 passing tests; origin/codex/harness-greyhound-jurisdictions is 490b0c9 with eight Harness/Greyhound adapters, LeTROT, independent T/H/G registries, and 131 passing tests. Both heads are pushed and were synchronized at the audit point, but they are not merged into one common starting revision. This documentation reconciliation now leaves local main with intentional uncommitted documentation changes.

• Updated README, handover, source inventory, consolidated reports, project changelog, and QDG Knowledge Base documentation to expose that two-branch baseline and prevent a new branch from being assumed to contain all three disciplines. Versioned read-back verified overview v11, changelog v18, Thoroughbred report v6, new Harness/Greyhound report v1, programs v11, URL inventory v9, and all eight Harness/Greyhound jurisdiction pages at their new versions with intact histories.

• Published and read back the detailed Equibase update in QDG Knowledge Base: USA country page v3, project changelog v14, and consolidated Thoroughbred report v5; program index remains v9 and scraping URL inventory v7. The USA page now lists the observed pre-race card, post-race summary, detailed chart, GPS, breeding, and horse-profile information and explicitly identifies meeting/race PDFs as downloaded binary artifacts. Every expected marker and immutable page history was verified through the approved versioned service with optimistic version checks.

• Replaced the USA Equibase TVG-mirror assumption with the licensed official www.equibase.com lifecycle. us_equibase now follows the entries, summary-results, and full-chart indexes; downloads and stores the complete card and all-race summary HTML, full-meeting PDF, every published per-race chart PDF, and every published GPS HTML page. The PDFs are binary post-race artifacts for FormParser, not link-only records. Canada remains on its existing separate mirror workflow.

• Removed the USA broad-index overqueue defect through two-level nested discovery and exact requested-date filtering. Date-less summary/full-chart control indexes remain traversable while dated meeting targets outside the requested range are excluded. Strict PDF validation checks MIME type, %PDF- signature, minimum length, and closing marker. The full suite passes 158 tests.

• Licensed-session browser validation for Del Mar on 7 August 2026 confirmed an eight-race complete card, an eight-race summary, eight race-chart PDFs, seven published GPS pages, and a 10-page full-meeting PDF. Unattended direct and approved-proxy HTML/PDF attempts still returned Imperva HTML, so the full canonical crawl was not run and no new downloaded artifacts are claimed; a licensed non-interactive session/API transport remains required.

• Recorded the 08:44 AEST Thoroughbred canonical-artifact audit for the 21 adapters registered at that snapshot. Real canonical data existed for every adapter checked under C:\Users\Robert\project-data\FormCrawler\t-artifacts; there were zero corrupt or missing artifacts, byte-length or SHA-256 mismatches, and missing FormParser handoffs, and all audit crawl processes had finished. The registry subsequently reached 25 Thoroughbred adapters; the four later Hungary, UAE, Bangalore, and Hyderabad adapters have separate successful artifact/handoff evidence in the consolidated report.

• Preserved the strongest complete-day evidence from that audit: GB 31 pre/post pairs, Umanity JRA 36 pairs, Umanity NAR 34 pairs, Czechia 28 pairs, Bahrain 9 pairs, Qatar 7 pairs, Korea 20 cards plus 10 results, and complete Turkey and Selangor meetings.

• Recorded audit limitations without overstating coverage: Argentina, Chile, Germany, and Uruguay were post-race only; New Zealand used a combined meeting-lifecycle page; Saudi had 12 results with racecards unavailable; Sweden captured three pairs while three remained unpublished; Spain and South Africa had valid meeting/day evidence plus unavailable targets; and Canada/USA Equibase each had one complete pair. The Equibase broad-index overqueue defect was open at that audit snapshot and has since been fixed. Unregistered candidate countries remain outside this audit.

• Consolidated Australian Thoroughbred documentation under the national Racing Australia adapter and country page. NSW, Queensland, and Western Australia meetings are handled through Racing Australia's national index, so the three redundant no-source state wiki pages were retired.

• Re-reviewed and fully live-validated the Selangor Turf Club (my_selangor_turf_club) acquisition boundary. The complete 26 July 2026 meeting captured a 12-race, 805,712-byte pre-race card and a 12-race, 545,777-byte results page on their first attempts, with captured=2, no retries/failures, matching SHA-256 metadata, and valid FormParser handoffs. All 152 tests pass with warnings promoted to errors. QDG KB read-back verified Malaysia/Selangor v3, project changelog v9, and the consolidated report v2.

• Published the consolidated Thoroughbred report and per-source/no-URL status pages to the versioned QDG Knowledge Base, then merged the overview, changelog, programs, and scraping-URL pages against their latest versions. Read-back verified all 52 expected pages and histories; central versions are overview 5, changelog 7, programs 5, and URL inventory 5.

• Completed the bounded Thoroughbred source-research run from documents\SITE_LINKS.md and added documents\THOROUGHBRED_SOURCE_RESEARCH_2026-08.md with per-source routes, transport/API analysis, supplementary-data findings, breeding/profile examples, direct review links, stopped-source reasons, and the three-attempt Saudi diagnosis.

• Added and registered raw-acquisition adapters and launchers for Hungary/Kincsem Gallop (hu_kincsem_park), UAE/Emirates Racing (ae_emirates_racing), Bangalore Turf Club (in_bangalore_races), and Hyderabad Race Club (in_hydraces). All four use compact date keys required by the shared range workflow and deterministic public HTML routes.

• Added focused regression coverage for the four new adapters. Direct assumption checks and final slow crawls verified paired pre/post HTML, hashes, metadata and FormParser handoffs. Hungary and UAE passed attempt 2 after a compact-date integration fix; Bangalore passed attempt 3 through the approved proxy; Hyderabad passed attempt 1. The full suite passes 152 tests with ResourceWarning treated as an error.

• Classified sources that could not safely provide paired historical pre/post data without speculative code: Cloudflare/proxy failure, stateful JSF/postback-only operation, expiring historical racecards, authentication, PDF-only material, outages, and wrong-discipline workbook links are now explicit report outcomes.

• Added results-only filtering to the shared date-range workflow through repeatable --page-type values, and added scripts\crawl_racing_australia_results.bat START_DATE END_DATE [MEETING_LIMIT_PER_DATE]. The Australian launcher uses the weekly homepage meeting index but queues only one validated FreeFields/Results.aspx page per meeting.

• Live-verified the results-only crawler for 2026-08-07 with a one-meeting limit and an isolated queue database. One weekly index found five meetings, only five results targets were queued, and the 190,581-byte Coffs Harbour result completed with HTTP 200, zero retries, and zero failures. Its SHA-256 metadata and FormParser handoff matched; the other four results remain resumable in the isolated smoke queue. The full suite now passes 89 tests.

• Extended scripts\crawl_stud_book_argentina_results.bat with a direct meeting-URL mode. Supplying one quoted https://www.studbook.org.ar/reuniones/detalle/... URL now captures and validates that complete result page without relying on historical calendar selection; the existing date-range and optional per-date-limit modes are unchanged.

• Live-verified the direct launcher with San Isidro meeting 21422 (2026-08-05, meeting 72). It stored 300,597 bytes with 11 listed races and 11 RESULTADOS sections as artifact 6c6e7487b3f2114b9a1d5a7fd0071beb; SHA-256 a2e42f7d363ffa8211e6e6fea3001d1422c1a795bca805398b7b23eab5c06ad0 and the FormParser handoff target matched.

• Standardized command-line reporting through the shared status.py module. Interactive work now uses consistent [HTTP], [PROXY], [DISCOVERY], [QUEUE], and [RANGE] transient states, while every command emits stable [ARTIFACT], [DATE], [RUN], [QUEUE], [DIAGNOSTIC], or [ERROR] summary records.

• Added automatic Racing and Sports proxy failover to the shared fetcher used by every current source module. A direct public GET challenged with 403 or 429 is tried once through the policy-limited proxy and still passes normal source validation before storage; POST targets are ineligible, and a failed proxy attempt returns to the existing block pause/hard-stop path. Explicit proxy-only mode remains available. The full suite now passes 89 tests.

• Added scripts\crawl_racing_australia_today.bat. It resolves the local date as YYYY-MM-DD through PowerShell, then calls the Australian range launcher with identical start and end dates to discover all indexed meetings and capture their available lifecycle pages.

• Published one detailed QDG Knowledge Base page for each implemented country crawler: Australia/Racing Australia version 2, Germany/Deutscher Galopp version 2, Chile/Hipódromo Chile version 2, Uruguay/Maroñas version 1, and Argentina/Stud Book version 1. Every page records key routes, source and artifact data structure, adapter/program identity, launch and generic commands, readiness rules, storage, recovery, live verification, and a country-module changelog.

• Updated QDG KB formcrawler/overview through version 3, formcrawler/form-crawler-programs through version 3, and formcrawler/changelog through version 5. Read-back checks verified required sections, current Racing Australia weekly-index range support, 75-test status, Unicode content, and every immutable page history. The dedicated MCP wrapper was not exposed, so publication used the same local versioned KnowledgeBaseService with expected_version conflict checks against the qdgwiki schema rather than raw SQL.

• Extended Racing Australia discovery from one known meeting to homepage-indexed date ranges. Every date maps to its Monday Home.aspx?date=D/M/YYYY index, so the shared range workflow captures one index per week and filters meeting keys to the exact inclusive dates requested.

• Added scripts\crawl_racing_australia_range.bat START_DATE END_DATE [PAGE_LIMIT_PER_DATE]. Homepage links using Form.aspx, Nominations.aspx, or Weights.aspx can also seed a single meeting directly, while every discovered meeting fans out to program, acceptances, gear changes, scratchings, and results.

• Live-verified 2026-08-07 range discovery: one 53,994-byte weekly index found five meetings and queued 25 lifecycle pages. A bounded follow-up captured one 37,785-byte Coffs Harbour program through the range workflow; both hashes and FormParser handoffs matched, with zero retries or failures. The current full suite passes 89 tests.

• Implemented and registered Racing Australia (au_racing_australia) with racing code AUS, ISO country AU, strict public FreeFields meeting-key validation, and one bounded five-page bundle per meeting: race program, acceptances, gear changes, scratchings, and results.

• Added scripts\crawl_racing_australia_meeting.bat MEETING_URL. Any supported meeting page can be supplied; the early-published race program is used as the control so prerace acquisition does not depend on results already being available. Unpublished page placeholders are rejected and remain retryable.

• Live-verified the supplied 2026-08-07 Coffs Harbour URL end to end. Five HTML artifacts (37,784; 152,298; 45,614; 34,727; and 190,581 bytes) completed with zero retries or failures; every stored SHA-256 and FormParser handoff matched.

• Independently retested Uruguay date-range operation. A discovery-only 2026-07-01 through 2026-07-31 scan found 17 published meetings; a bounded follow-up captured the newly discovered 2026-07-31 Las Piedras meeting with zero retries or failures. The 417,274-byte XML contains 11 races, 124 horses, and 124 official finishes, and its hash, metadata, and FormParser handoff were verified.

• Confirmed https://sproxy.racingandsports.com/Gateway is Robert's proxy service and added it as an explicit FormCrawler transport for public GET pages. The gateway receives POST application/json with {"url":"<public-url>","direct":true,"force":true} and returns source text in html_data with optional last_scraped metadata.

• Added --use-racingandsports-proxy and FORMCRAWLER_USE_RACINGANDSORTS_PROXY=1. The Python transport unwraps html_data as UTF-8, records racingandsports_proxy provenance, rate-limits both gateway and destination hosts, and runs the normal source validator before storage. Source POST targets are rejected because the gateway contract does not carry their original POST body. Gateway and CLI regression coverage brought the suite to 63 tests at implementation time; subsequent work brings the current suite to 89 tests.

• Live-verified the Python proxy transport at approximately 2026-08-07 22:38 AEST (UTC+10:00) with Robert's Belgium horse-profile example: HTTP 200 returned 57,178 bytes of HTML with racingandsports_proxy provenance.

• Tested the proxy against the Argentina July 2026 calendar at approximately 2026-08-07 22:30 AEST (UTC+10:00). The gateway returned HTTP 200 and 82,099 characters, but the embedded calendar still identified itself as August 2026 (data-anio="2026", data-mes="8"), so the existing historical-month mismatch protection remains correct.

• Corrected Argentina historical-calendar handling after a 2026-07-01 live run exposed that the site silently returns the current month unless its interactive verification succeeds. Calendar requests now use the form's real mes/anio parameters and reject a returned-month mismatch instead of reporting a misleading zero; FormCrawler does not automate or bypass the site's verification.

• Identified four immutable pre-fix Argentina calendar snapshots labelled 2026-07 whose bodies are actually the site's August fallback. Their artifact IDs are 82eb8fae2f74620aa3951cb0f548fff0, 890d652ac784747acfa634f8f3b72342, ed8d5aef503c4bbcf5c829cf181bfcb6, and b41f0a74e5078a8b920c7907504f82da; cleanup of their content, metadata, handoffs, and queue ledger records is pending explicit approval.

• Live-tested the Uruguay range crawler for 2026-08-01: one calendar capture discovered the Maroñas and Paysandú meetings, then two complete eight-race XML files (304,943 and 208,874 bytes) were captured with zero failures. Both SHA-256 values and FormParser handoffs were verified.

• Added synchronized Sydney-time Last updated timestamps to the changelog, README, and handover, with a rule to refresh all three after every material project change.

• Added exact-name Uruguay/Maroñas horse-profile capture. The new capture-discovered workflow stores the first-party search JSON, queues only the exact equine match, and stores one raw horse-card JSON without requiring complete breeding data.

• Added scripts\crawl_maronas_horse_profile.bat "HORSE NAME". Search and horse-card requests share the central 15-25-second host limiter, persistent queue, retries, immutable storage, and FormParser handoff.

• Live-tested Corazon Rebelde on 2026-08-07: the 200-byte search response discovered horse 72002; the 2,136-byte profile contains the available horse, sire, dam, and damsire records. Stored SHA-256 and handoff verification passed, with 23.969 seconds between the two service-host requests.

• Implemented and registered Argentina Stud Book (ar_stud_book) with racing code ARG, ISO country AR, strict same-origin calendar/meeting URL validation, and one complete HTML result artifact per published meeting.

• Added month-aware inclusive date-range discovery for calendar months the site serves without interactive verification: each distinct calendar month is captured once, meeting links are filtered to the exact requested dates, future entries without detail links are ignored, and a meeting remains retryable until every listed race has a RESULTADOS section.

• Added scripts\crawl_stud_book_argentina_results.bat START_DATE END_DATE [MEETING_LIMIT_PER_DATE] for full, discovery-only, smoke-test, and resumable runs under the shared queue, 15-25-second host limiter, retry controls, and kill switch.

• Live-tested 2026-08-06: one 78,683-byte August calendar capture discovered the La Plata meeting, then one 290,541-byte meeting capture stored all 11 published race-result sections. Both artifacts passed SHA-256 and FormParser handoff verification with zero failures.

• Added eight Argentina adapter/workflow tests, including returned-month mismatch coverage, and monthly discovery caching; the full suite now passes 60 tests with ResourceWarning treated as an error.

• Standardized FormCrawler-authored text on UTF-8 across Windows, Linux containers, CLI streams, JSON metadata, handoff manifests, launchers, and repository files; raw source responses remain byte-for-byte unchanged with their declared charset recorded.

• Changed new human-readable artifact directories and filenames from ASCII transliteration to safe NFC-normalized Unicode, preserving accented and non-Latin letters, combining marks, and numbers.

• Added UTF-8 console and Unicode filesystem regression coverage; all 49 tests pass with ResourceWarning treated as an error.

• Added Hipodromo Chile workout capture for https://hipodromo.cl/carreras-aprontes. The adapter maps the public page to its same-origin first-party JSON endpoint, validates published workout rows, and stores the full rolling response as one supplementary artifact.

• Added scripts\crawl_hipodromo_chile_workouts.bat; each run uses the shared 15-25-second host limiter and makes one request rather than one request per date or horse.

• Live-verified the workout launcher on 2026-08-07: one 127,616-byte response contained 9 dates from 2026-07-23 through 2026-08-05 and 185 workout rows; its SHA-256 metadata and FormParser handoff matched. The full suite now passes 47 tests.

• Implemented and live-verified Uruguay's Maroñas source (uy_maronas) using racing code URU, ISO country UY, the interactive calendar's first-party JSON service, and one complete result XML per published track meeting.

• Added safe persisted GET/POST request definitions to the capture target and SQLite queue. Calendar POST requests retain the shared per-host limiter, retry controls, circuit breaker, response-size limit, immutable storage, and kill switch; request bodies are not copied into artifact metadata.

• Added single-request inclusive-range discovery for capable sources. Maroñas filters the returned calendar to the exact requested dates and queues only published meetings, including Maroñas, Las Piedras, and supported interior Uruguay tracks.

• Added scripts\crawl_maronas_results.bat START_DATE END_DATE [MEETING_LIMIT_PER_DATE] and the --target-limit-per-date alias for controlled discovery-only, smoke-test, full, and resumable runs.

• Live-tested 2026-08-02: one calendar POST discovered one Maroñas meeting and one XML GET captured 471,938 bytes containing 11 races and 142 horses with official finishes. Both artifacts passed SHA-256 and FormParser handoff verification with no failures.

• Expanded the full suite to 44 passing tests, including POST transport/persistence, strict range filtering, meeting XML readiness, range-level discovery, CLI registration, and resumable queue routing.

• Added ✅ completion tracking to the scraping URL catalogue. The tick is applied only to exact implemented and verified crawler entries, currently Germany/Deutscher Galopp, Chile/Hipódromo Chile, Uruguay/Maroñas, and Argentina/Stud Book Argentina; other entries remain unmarked candidates.

• Added the implemented Deutscher Galopp results-index URL to the main Thoroughbred catalogue so both completed crawlers are visible in the planning list. Every future completed source adapter must receive the tick as part of its documentation update.

• Added resumable capture-date-range --start YYYY-MM-DD --end YYYY-MM-DD processing for date-capable sources, with inclusive iteration, a 366-day safety cap, discovery-only mode, and an optional per-date race limit for controlled smoke testing.

• Added scripts\crawl_hipodromo_chile_results.bat START_DATE END_DATE [RACE_LIMIT_PER_DATE] as a convenience launcher using the shared persistent queue and 15-25-second per-host request spacing.

• Live-tested 2026-08-06 with a one-race limit: stored the Hipodromo Chile meeting-index JSON and race 820996 JSON, recorded zero failures, and left 19 discovered race targets queued for a resumable continuation.

• Expanded date-range, CLI alias, safety-bound, and resume tests; all 36 tests pass with ResourceWarning treated as an error.

• Implemented the cl_hipodromo_chile Thoroughbred source using racing code CHI, ISO country CL, the public latest-results route as canonical input, first-party JSON meeting discovery, and complete per-race JSON result capture.

• Added safe same-origin canonical-page Referer support for API-backed capture targets; cross-origin canonical URLs are not forwarded.

• Live-verified Hipodromo Chile meeting 101140: the meeting-index endpoint returned HTTP 200 JSON and discovered all 20 race targets, beginning with race 820996 on 2026-08-06.

• Added seven initial Chile adapter tests plus same-origin and cross-origin HTTP Referer coverage; the suite passed 31 tests before date-range coverage was added.

• Added documents\SITE_LINKS.md as the local catalogue of Thoroughbred, Harness, and Greyhound candidate sources, derived from the supplied workbook and supplemented with the missing Denmark (DEN) and Morocco URLs.

• Reformatted the source catalogue as Country: Site name (source_id) entries followed by individual URL lines; after adding Deutscher Galopp and Maroñas, the main discipline lists contain 68 entries, 68 URL occurrences, and 59 unique URLs with no embedded HTML breaks or Markdown tables.

• Documented source_id as the stable website/feed/source-definition segment in <discipline-root>\<year>\<race_code>\<source_id>, with candidate identifiers remaining proposed until an adapter or source definition is implemented.

• Updated QDG Knowledge Base page formcrawler/scraping-urls-list through version 4, preserving the active Deutscher Galopp configuration and crawl controls while clearly separating candidate links from approved crawl scope.

• Published the durable catalogue summary to QDG Knowledge Base formcrawler/changelog version 3 while retaining the original German acquisition release history.

• Added non-scrolling interactive crawl status using carriage-return updates for waiting time, current retrieval URL, and captures completed during the run; final summaries and block incidents remain permanent lines while redirected logs omit transient refreshes.

• Added block-response safety: the first 403/429 is logged and pauses requests for 10 minutes; a second consecutive block creates FormCrawler\STOP, logs a hard-stop event, and terminates queue processing without draining additional targets.

• Replaced token-only artifact directories with searchable source-native event and race filenames while retaining stable race IDs and artifact-ID suffixes for collision safety.

• Migrated and SHA-256 verified all 17 existing Deutscher Galopp artifacts, their FormParser handoffs, and SQLite artifact URIs into the human-readable layout; no legacy artifact directories remain.

• Added source-native Düsseldorf/race-number/race-title label extraction strictly for operational artifact naming, without introducing canonical racing-data parsing.

• Added Deutscher Galopp results-index discovery and a resumable batch launcher for sequential policy-limited capture.

• Hard-coded the requested Chrome 151 browser user-agent in the shared HTTP client.

• Audited repository scope before the initial commit: 35 program, test, script, and documentation files were included, with no downloaded or processed data, runtime databases, logs, caches, secrets, symlinks, or oversized files in scope.

• Expanded .gitignore to exclude common local data, capture, artifact, runtime, log, output, export, inbox, partial-download, HAR, and WARC paths if they are accidentally created inside the repository.

• Created local root commit 816f590 (Add German post-race meeting acquisition workflow); it has not yet been pushed to GitHub.

• Registered the formcrawler project in QDG Knowledge Base and published overview version 1 and changelog version 2, linked to commit 816f590 with immutable page history retained.

• Implemented Deutscher Galopp calendar, meeting, and race-page discovery without parsing racing values.

• Added a backend-neutral crawl-state contract and a SQLite local adapter with target deduplication, discovery provenance, expiring leases, retry scheduling, attempt history, artifact history, and queue summaries.

• Added discover, run-queue, capture-meeting, and queue-status commands alongside direct single-page capture.

• Classified German Thoroughbred artifacts with racing code GER, ISO country DE, and discipline T.

• Migrated the FormCrawler-side runtime data into t-artifacts\2026\GER and FormParser\inbox\T\2026\GER, with backend-neutral content and metadata URIs in handoff records.

• Captured all nine races discovered from the supplied 2026-08-02 seed race page; the queue finished with 9 captured, 0 failed, and verified SHA-256 and handoff references.

• Verified eight live same-host request gaps from 15.949 to 24.660 seconds, within the required 15-25-second interval.

• Expanded the suite to 14 unit tests covering source discovery, queue state, lease expiry/ownership, workflow processing, CLI defaults, storage, handoff, policy, and HTTP controls.

• Clarified Docker/EKS as a required runtime environment rather than an optional future direction, and added FORMCRAWLER_DATA_ROOT with a /data Linux-container default.

• Accepted the Docker/EKS cloud-portability decision: core workflows remain stateless and backend-neutral, while local filesystem, SQLite, queue, and file-lock implementations are replaceable adapters.

• Changed the per-host request interval to a 10-second base delay plus 5-15 seconds of jitter, producing an effective 15-25-second gap between request starts.

• Added the initial country/discipline classification implementation; the later approved race-code layout superseded its directory design and has now been applied to FormCrawler code and runtime data.

• Added the controlled discipline field to capture targets, artifact metadata, and handoff records.

• Moved the shared directory contract into C:\Users\Robert\Projects\FormDocumentation\md and added a companion architectural design.

• Changed artifacts and handoffs to a shallower year-first hierarchy so a complete year can be archived with one directory move.

• Approved and implemented separate t-artifacts, h-artifacts, and g-artifacts roots with year/race-code/source beneath each.

• Defined race_code as the racing jurisdiction path field, distinct from ISO country codes, with examples including NZ and SAF.

• Recorded Docker/EKS compatibility as a binding design constraint, including stateless workers, storage URIs, distributed coordination, idempotency, external configuration/secrets, and S3 lifecycle archival.

• Added reusable Python modules for capture models, request-policy enforcement, HTTP acquisition, immutable artifact storage, orchestration, and source-adapter registration.

• Added the first Deutscher Galopp race-page adapter and generic capture CLI.

• Moved runtime data ownership outside both source repositories to C:\Users\Robert\project-data and documented the shared layout.

• Added cross-process per-host serialization, external kill-switch support, safe response-size enforcement, and FormParser inbox handoff records.

• Captured Deutscher Galopp race 1364507 as a verified 111,203-byte post-race HTML artifact with matching SHA-256 metadata.

• Added the initial seven unit tests covering source validation, storage, handoff creation, shared rate-limit state, default request spacing, portable data-root configuration, and the external kill switch.

• Initialized the local Git repository on main and linked origin to https://github.com/RQDG/FormCrawler.git.

• Added the mandatory friendly-scraping policy covering per-host concurrency, request delays, jitter, retries, backoff, caching, circuit breaking, request scoping, logging, and an immediate kill switch.

• Required all source adapters and capture phases to share the central request queue and rate limiter.

• Selected Deutscher Galopp as the first planned discovery-to-capture vertical slice.

• Added source-control exclusions for local captures, generated output, secrets, runtime databases, caches, and logs.

• Aligned project documentation, handover, versioning, and future UI conventions with applicable Global Memory guidance.

v1.0.0 - 2026-08-06

• Added the standard project directory structure.
• Documented FormCrawler as the page and source acquisition component.
• Documented FormParser as a separate sibling human-assisted parsing project.
• Defined the initial responsibility boundary and a preliminary captured-artifact contract.
• Recorded that legacy converter/scraper code is clarification-only reference material.
• Added project version-control rules.
• Verification: inspected the empty workspace and verified the created documentation files.