How can I integrate JIRA into Snowflake?

Question

So I basically want to create an automation where newly created JIRA issues are listed in Snowflake, in some format. I've read something about using AWS lambda and AWS S3 bucket + kinesis and firehose, but is there any other way? An easier way? I would also like to know how to exactly do this with the AWS tools.

PS. I'm quite new at this stuff 😃

SilkeS · Answer

This topic has been locked because the thread has become a bit dated. If you’d like to keep the conversation going or have additional questions, we encourage you to start a new topic. You can read more about necro posting (“raising threads from the dead”) in our  Community Guidelines .

Thanks!

MobilityStream · Answer

Hi @Meena Bhujang ,

We have several ways in which data can be exported:

* Real-time: when an issue is updated in Jira it is almost instantly sent to Snowflake. This is great when you need up-to-date data. We have instances with thousands of users, constantly making updates to issues, that run in this mode. The lag is minimal. You can also configure what event type (create, update, comment, etc) triggers an update.
* Scheduled: at a specific time a full update or a subset of an update is run. We often see clients grouping them depending on their requirements (e.g, issues hourly, worklogs daily, change history weekly, assets bi-weekly, etc). There is a lot of flexibility how this can be organized. Obviously smaller updates complete faster. When an update gets kicked off it starts exporting right away in the background.
* One-time: a one-time export doing a one-time load

All the initial exports will take some time to complete. We provide an indication on percentage complete on the App Page and also provide a table within Snowflake with some telemetry.

The app can handle any amount of data. it will just take more time to complete if you have millions of issues vs just a few thousands.

You can add any field in Jira including all custom fields. In addition you can add additional dimension and fact tables. The schema that is created is described here and should be comprehensive and easy to query: https://docs.mobilitystream.com/si/export-database-schema

If you have more detailed questions, please reach out to support@mobilitystream.com. We are happy to provide more answers.

Kind regards,

The Mobility Team

Meena Bhujang_lulour · Answer

@MobilityStream - is there a lag in data pull from jira to snowflake? How scalable is the data mapping between jira and snowflake? Is there a threshold on how  much data can be pulled before performance issues start showing?