Replace Jira API data with Atlassian Data Lake data

[eric_jo]

Hello, I am currently using the Jira API to convert data into CSV files, which are then connected to Power BI. I am asking these questions because I encountered an issue while changing the data source for this project, which involves connecting the Jira API to Atlassian Data Lake and Fabric via Delta Sharing.

1. It appears there is a difference in data structure between the Jira API and Data Lake. How can I verify the mapping?

2. To what extent can I switch from Jira API to Data Lake data?

3. I am using data in the format `customfield_xxxxxxx` within the Jira API. Is this data also available in Data Lake?

[Mohsin Shaikh]

Hello Eric,

These are a common scenario when migrating from a Jira REST API–based pipeline to Data Lake / Data Shares via Delta Sharing. Let me address each one.

---

1. Verifying the mapping between Jira API and Data Lake

You're right that the data structure differs between the Jira REST API and the Atlassian Data Lake. The Data Lake uses a pre-modeled relational schema with its own naming conventions, for example, the API field resolutiondate maps to jira_issue.resolution_at in the Data Lake. There is no 1:1 field-name mapping provided out of the box.

How to verify the mapping:

• Schema documentation — Review the official Schema for Jira family of apps and the Data share schema for Jira family of apps. These list every table, column, data type, and foreign key relationship available in the Data Lake and Data Shares respectively.

• Schema browser in Atlassian Analytics — If you have access to Analytics, navigate to Data → [Your Data Lake connection] → Schema to interactively browse available tables and columns.

• Important: The Data Share model is slightly different from the Data Lake model used inside Atlassian Analytics. Data Shares provide access to pre-transformation tables, so you may need to perform some additional transformations on your side to recreate the views you had via the API.

There is an open suggestion on JAC https://jira.atlassian.com/browse/ANALYTICS-409 requesting that Data Lake field names align more closely with Jira/API field names. You can watch/vote on that if it's important to your workflow.

---

2. To what extent can you switch from Jira API to Data Lake data?

For bulk analytics and reporting workloads the Data Lake / Data Shares are the recommended replacement for API-based extraction. Key advantages:

• No API rate-limiting or pagination concerns

• Pre-modeled tables with primary/foreign key relationships

• Continuous updates (most tables within ~90 minutes; some enhanced tables up to 5–8 hours)

What you can cover: Issue data, projects, sprints, users, comments, worklogs, issue history/changelogs, custom fields, components, versions, labels, linked issues, SLAs, and more are all available.

Known gaps: The Data Lake schema does not yet cover every entity available via the API. Some site-level configuration objects (e.g., all filters, all agile boards, remote links) and certain Marketplace/Forge app fields are not included. If your current API pipeline extracts data that is not present in the Data Lake schema, you may need to retain a hybrid approach for those specific entities.

Review the Available data in the Atlassian Data Lake documentation for a comprehensive list of what is currently supported.

---

3. Custom fields (customfield_xxxxxxx) in the Data Lake

Yes, Jira custom fields are available in the Data Lake. However, they are not stored as individual columns named customfield_xxxxx. Instead, all custom field data lives in the "Issue field" table (SQL name: jira_issue_field) with the following structure:

Column	Description
name	The human-readable name of the custom field (e.g., "Story Points")
value	The field's value for that issue
field_id	The field identifier
issue_id	Links back to the issue

To query a specific custom field in SQL mode:

SELECT issue_id, name, value FROM jira_issue_field WHERE name = 'Your Custom Field Name'

In Visual mode, Atlassian Analytics also provides a virtual "Issue custom fields" table that pivots your custom fields into individual columns — searchable by their display name.

See: https://confluence.atlassian.com/analyticskb/query-jira-custom-fields-1188411074.html

Limitations to note:

• Custom fields from third-party Marketplace apps or Forge apps are not included in the Data Lake or Data Shares.

• Fields are referenced by their display name, not by customfield_xxxxx IDs. You'll need to map your customfield_xxxxx references to their display names when rebuilding queries.

You can also create Query Jira custom fields with custom columns or custom tables in your Data Lake connection to make frequently-used custom fields easier to query repeatedly.

I hope this helps.

Regards,

Mohsin