Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,293,854
Community Members
 
Community Events
165
Community Groups

Introducing Atlassian Data Lake for Jira Software - Early Access [NO LONGER AVAILABLE]

Introducing Data Lake for Jira Software

a263a427-bd94-433e-b95d-7cae16615322.png

We’re excited to introduce our data lake, now available in early access. It is available to preview for Jira Software Cloud Premium and Enterprise customers.

Many of our customers regularly turn to Jira reports and dashboards to learn from and improve how they work. However, some customers also require more advanced reporting capabilities that are best supported by their own BI tools. For example, some customers may need the ability to more deeply customize what they report on or they may need to combine data sets across multiple Jira sites, or with data from 3rd party tools. 

Our data lake makes it much easier to connect data from Jira Software with your BI tools so you can do just that. With the data lake, you get access to a structured data set, which is purpose built to be simple to query and simple to plug into your own BI tools. We believe this new feature will make it more accessible than ever to unlock additional value from your Jira data in your own BI tools with reports and insights tailored to your needs.

Our data lake supports popular BI tools such as Tableau, PowerBI, Looker, Qlik, Databricks, Mulesoft and SQL work bench (if the BI tool is not on this list, please note that in our sign-up form). While today our data lake is only available with Jira Software, in the future we also expect to support additional Atlassian products.

UPDATE March 2022
Thank you all for your interest in this feature. We have decided not to productionise this feature as originally designed and will be taking the architecture in a new direction to support this. It will be rolled into a more comprehensive Analytics offering included in Enterprise Edition.

More details will be provided at the Atlassian Team 22 event.

38 comments

Hi Ben - Your Google form needs to have it's permissions adjusted. Getting "This form can only be viewed by users in the owner's organization.".

Looking forward to playing with this.

Ben Jackson Atlassian Team Jan 19, 2021

Ooops done thanks @Neeraj 

Like Dave Liao likes this

Can you comment at all on how permissions will be applied to the data in the data lake?

Like WPG likes this
Ben Jackson Atlassian Team Jan 19, 2021

Boris initially with EAP permissions are only applied on the 'access' level not at the data level. In talking to customers many said they would apply permissions to the reports and dashboards created from the data and additional granularity wasn't immediately required for some use cases.

We will look to providing more permission options over the lake data in the future but we have some internal engineering dependencies we need to wait for.

Like # people like this

I look forward to playing with this. Do you have any scoping on what data goes into the data lake in the first place? For example "exclude projects in the sensitive category".

Like # people like this
Darryl Lee Community Leader Jan 20, 2021

One type of request that occasionally comes up is for manager/executives to see issue/ticket counts, without wanting (or needing) to dive into individual tickets. Or with siloed teams, there's been requests to actively prevent visibility into details while still offering a real-time view of high-level numbers.

Has there ever been any thoughts about how to do deal with a request like that? Would the data lake offer any opportunities to have more finely grained permissions?

Like # people like this

Is it on the Atlassian roadmap to make it available on Jira data center?

Like # people like this

Hi @Ben Jackson

We are marketplace partners but we don't have Jira Enterprise or premium, just the standard Atlassian provides to marketplace partners without badge. We are really interested in joining the program, can we?

Regards,

Like # people like this
Dave Liao Community Leader Jan 26, 2021

@Ben Jackson - will Community Leaders be able to experiment with Data Lake, so we can help others on Atlassian Community who have questions about it? Hopefully someone's already requested the EAP for our sandbox. 🙏

Like Ryan Haney likes this
Josh Frank Atlassian Team Jan 27, 2021

@Jaswanth P A similar feature will be available for Data Center called "data pipeline". We've included it on our public roadmap.

Like # people like this
Ben Jackson Atlassian Team Jan 27, 2021

@Boris Berenberg - Atlas Authority for EAP all data from the instance will flow into the lake including any sensitive projects. Customers would then need to gate access to the BI tool using that token.

We know there are some customers that will need more granular control on the Lake data and we are looking at a number of options for that. We hope to gather feedback during the EAP to help us determine the best path forward for future replication and access controls.

@Darryl Lee as per above, we plan to add more controls as we learn from our customers. One of the options I think might be relevant is replicating all data but redacting certain data for sensitive projects/issues so its still possible to roll up the full picture.

@Dave Liao there is a Community Leader Premium instance. We will arrange for it to be provisioned into the EAP and let you know.

Any other q's let me know.

Like # people like this
Ben Magro Atlassian Team Jan 27, 2021

@Jaswanth P we do have a similar feature in our Data Center offering call Data Pipeline. It's currently live for JIRA Software in 8.14.X.

Whilst Data Pipeline is not a hosted solution, it will export the relevant data to a CSV file for importation into a best of breed data hosting solution.

We are also working on bringing this to other Data Center products in the portfolio.

@Josh Frank Thanks for jumping on this one.

Ben Magro
Data Center - Product Manager

Like # people like this
Ben Jackson Atlassian Team Jan 28, 2021

@Gorka Puente _Appfire_ I'll email you to discuss.

Like Gorka Puente _Appfire_ likes this
Renee Zhang Atlassian Team Jan 28, 2021

@Dave Liao Absolutely, Community Leaders will be able to experiment with our data lake. We ask that you sign-up here with your premium sandbox instance and we will let you know once you're in. 

Like Dave Liao likes this

Hello, any plans to add this for Jira Service Management Cloud?

Like # people like this

@Ben Jackson @Ben Magro @Josh Frank et. al.

I don't know if this is the direction we are moving yet.  Is there a webinar or any other introduction medium that my team can experience?

Like Dave Liao likes this
Josh Frank Atlassian Team Feb 10, 2021

@Carla Pasquali Yes, there are plans for Jira Service Management to follow shortly after Jira Software although we can't provide an exact timeline. The data lake is part of the Atlassian platform so will work across multiple Atlassian products. Jira Software is just the first product to be supported.

Like # people like this
Josh Frank Atlassian Team Feb 10, 2021

@Gerard BrownThere's no webinar available to reference just yet, although if you're interested in participating in the early access program, we'd be happy to schedule a call with you to better understand your needs. Email us at datalakeeap@atlassian.com to connect. 

Like # people like this

Can you share what the details of the data in the data lake?  Do you have a data dictionary?

What I am looking for is whether the data lake will have calculated data that is presented in jira reports but not available using JQL.

Examples:

Sprint velocity: sum of issues, sum of points committed and completed in sprint

Control chart: Time in status

Like Dave Liao likes this

@Ben Jackson, we are eagerly waiting for being a part of this community, we feel we have a very strong case of using this tool in near future. Would like to know if there is any way to expedite participation process. 


Our team would be happy to get on a call with you and take some advise on how we can proceed on this.

 

cc @Jake Meisenbach 

Ben Jackson Atlassian Team Feb 22, 2021

@Dorothy Krajewski the initial lake schema does not have any preset calculations but that should be trivial for which ever BI tool you would like to connect to perform such a roll up.

 

@Ankur Bhargava Takeda we would be more than happy to get on a call and discuss this with you. If you'd like to send us a message via datalakeEAP@atlassian.com we can organise it.

For everyone who has registered thank you all for your interest. We have had such a big response. Provisioning is rolling out in stages so if you don't hear from our team don't be concerned we will be contacting you in the coming days.

Ben

@Ben JacksonWe at Manhattan associates would also like an initial call to go over what the lake contains. Are their any webinars planned? 

Ben Jackson Atlassian Team Feb 24, 2021

Hi Aneu yes happy to set up a call check your emails there should be a link in there to be able to book a call.

Hey @Ben Jackson - I sent an email on 2 March 2021.  I have yet to get a response.  Can someone help?

Thanks

Lisa

Renee Zhang Atlassian Team Mar 31, 2021

@Lisa Schaffer Hi Lisa, I've just sent you a response via email - apologies for the delay! 

Comment

Log in or Sign up to comment
TAGS
Community showcase
Published in Jira Software

Upcoming changes to epic fields in company-managed projects

👋 Hi there Jira Community! A few months ago we shared with you plans around renaming epics in your company-managed projects. As part of these changes, we highlighted upcoming changes to epics on...

14,135 views 34 44
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you