FormCrawler Changelog

2026-08-07 ? Initial German post-race acquisition workflow

Source commit: 816f590 ? Add German post-race meeting acquisition workflow

Added

• Reusable Python acquisition package with shared models, HTTP retrieval, request policy, immutable storage, orchestration, and source-adapter registration.
• Deutscher Galopp adapter for individual race URLs and race-target discovery from supported calendar, meeting, and race pages.
• Complete-meeting capture starting from one race URL.
• Backend-neutral crawl-state interface.
• SQLite development adapter with target deduplication, discovery provenance, capture attempts, artifact history, retry scheduling, expiring leases, stale-worker protection, and queue summaries.
• CLI commands: capture, discover, run-queue, capture-meeting, and queue-status.
• FormParser inbox handoffs using content and metadata URIs.
• Mandatory scraping safeguards: serialized same-host access, 10-second base delay, 5-15 seconds of jitter, retry/backoff limits, response-size enforcement, circuit breaking, shared local rate-limit state, and an external kill switch.
• External runtime-data ownership under C:\Users\Robert\project-data.
• Docker/EKS portability decision with replaceable storage, queue/state, and rate-limit adapters.

Data classification

• German Thoroughbred artifacts use racing jurisdiction code GER.
• ISO country metadata remains DE.
• Discipline is T and artifacts are stored beneath t-artifacts.
• Runtime data and downloaded content are excluded from Git; only program source, tests, scripts, and documentation are versioned.

Verification

• All source modules compiled successfully.
• 14 unit tests passed.
• Live seed race 1364507 discovered 9 meeting targets.
• Queue result: 9 captured, 0 failed.
• All stored content hashes and FormParser URI references were verified.
• Eight live same-host request gaps fell between 15.949 and 24.660 seconds.

Known next work

• Add formal FormParser validation for the 1.0.0 handoff and artifact contracts.
• Define retention and repeated-snapshot content-deduplication behavior.
• Add conditional HTTP requests.
• Agree and implement scheduled calendar discovery.
• Implement shared cloud storage, queue/state, and rate-limit adapters before EKS deployment.