Hi folks,
We are currently building a data pipeline to pull Jira org events into our environment via Airflow (DAG running on a 5-minute schedule).
While verifying the ingested records, we observed data discrepancy caused by late-arriving events:
- Current Implementation: For a run at
24/09/2026 05:50 PM, the DAG queries the window 05:45 PM – 05:50 PM. - Issue: Re-triggering the API request for that exact same time window (
05:45 PM – 05:50 PM) several hours later (or the following day) yields more records than the initial run captured.
We checked the official documentation but couldn't find details regarding event indexing latency or data consistency SLAs for this endpoint. Could anyone please clarify:
- Data Consistency SLA: When is a given event time window guaranteed to be complete and static?
- API Indexing Lag: Is there a standard propagation delay between when an event occurs and when it becomes queryable via the API?
- Recommended Buffer: What time offset/lag should we apply to our DAG execution window to ensure 100% event capture?
Thanks!