New feature: Import and refresh Insight data from a URL

[#1]

Hello Community!

We’re excited to announce the launch of a new feature that allows Insight admins who import CSV or JSON files to pull their data directly from a URL, instead of manually creating and uploading files. Once linked, data can be refreshed with the click of a button, without the need to set up a new import.

For detailed information and step by step instructions on how to set up an import from a URL, please refer to our documentation on Atlassian Support.

To find out more about imports for Insight in general, refer to this article.

PLEASE NOTE:

As a first iteration of this feature we only support public and secure web addresses (https). We are looking to expand this functionality further in the future by adding authentication.

In the meantime, exposure of URLs can be restricting by granting access to the 'Atlassian domains' only as outlined here, and in more granular shape here.

[Kalin U]

Hi. Excited to try this out. I have a couple of questions:

1. Will the import duplicate objects if the CVS file contains previously imported data? In our case, we are interested in importing users.
2. Do you have any suggestions as to how this CSV/JSON import from a URL can be scheduled? I understand that this works manually, with the touch of a single button, but it still requires a person to do it :)

Regards!

[Dirk Ronsmans]

@Dorothea Linneweber ,

Excellent first step! I think the showstopper might be that it has to be a public (unauthenticated) url for now. Maybe a next iteration could be something with a OAuth2 authorization or token of some kind?

Looking forward to both iterations of this feature (we like to use Sharepoint to host the import files or putting them on OneDrive with a share link?  so using a O365 token or something would be great!), scheduling of course..  and future import sources ( *cough* Azure AD and on premise SQL ;-) *cough*) but liking the progress being made!

[Cristian Iorio]

This is great, the feature has been long-awaited !

[Gary Blower (Clearvision)]

The recent API changes mean you can also trigger the import on a schedule using Automation.

You'll need to create a scheduled automation and the Web Request action.

Steps

1. You need to encode your credentials "<EMAIL>:<API_TOKEN>" with base64. You can make this through any online tool like Base64 encode.
2. You'll need the Workspace ID and Configuration ID from Insight.
3. The Configuration ID can be found on the Import Configuration Details (schema > config > import > import history > details).
4. The Workspace id can be found using this URL 
   https:// <JSM Premium Site Name> .atlassian.net/rest/servicedeskapi/insight/workspace
5. Finally you can create the Automation WebRequest action using a POST on /import/start:

• Web request URL: https://api.atlassian.com/jsm/insight/workspace/<workspace id>/v1/import/start/<configuration id>
• Authorization: Basic <encoded creds>
• Content-Type: application/json
• Accept: application/json

 

[Dan C]

Assuming this is cloud only?

It'd be really helpful if all Atlassian posts could clearly state if something is data center, cloud, or both

[Kalin U]

@Dan C , the article has been tagged cloud.

[Dorothea Linneweber]

@Kalin U to answer your questions further above:

1. Duplication of data
   • Data won't be duplicated as long as the identifier remains the same. It will be overwritten with any changed information.
2. Scheduling of URL import using automation
   • As @Gary Blower (Clearvision) mentioned, there is a way to do this using Automation.
   • We do however want to call out that this solution has the api token stored in an automation rule which is not encrypted, only a plain text which is a security vulnerability. Depending on your use case this may not be too much of a concern but please be aware.
   • We are absolutely aiming to make scheduling available for imports moving forward, but are still working through some dependencies before we can commit to a timeline. Please bear with us.

@Dirk Ronsmans absolutely hear your concern around the risk of a public URL. As mentioned, adding authentication will be our next iteration on this feature, although we can't commit to an ETA just yet. In the meantime, exposure of URLs can be limited by restricting access to allow only the 'Atlassian domains' as outlined here, and in more granular shape here.

[Patrick]

That's a great step.  Obviously getting the security right will be critically important and I look forward to the next update, especially in regard to automation and scheduled calls.

[Ste404]

Is this likely to be extended with the ability to export a schema to JSON and then import that into a fresh instance, i.e. to move from a Dev to Prod instance.

[Maor Levinas]

Hi @Dorothea Linneweber

Any update on adding authentication?

Kind regards,
Maor

[Dorothea Linneweber]

Hi all,

Thanks for your continued interest! Please follow https://jira.atlassian.com/browse/JSDCLOUD-11482 to receive any updates on future iterations of this feature (API-Token based authentication + Oauth2.0).

Thanks,

Doro

[Michal Štoppl]

Hello @Dorothea Linneweber 

few notes related to loading the data from web server (URL):

- if we restrict incoming requests only to Atlassian IP/domains, we still make the data available to all Atlassian customers. It would be awesome if we could limit it only to our atlassian instance, but I am afraid that current solution does not work like that

- on 4th April you wrote that you will provide revised ETA "next week", which did not happen. Would you provide the update, please?

Thank you

Michal

[Prathim Mukherjee]

How are we going with adding the authentication level of security to this way of importing data. This can be really useful as the flow of asset or data to jira asset has to be continuous and not one time as that beats the purpose of having an effected CMDB. I got it working as well but the level of security added will be freaking awesome.

[Prathim Mukherjee]

How are we going with adding the authentication level of security to this way of importing data. This can be really useful as the flow of asset or data to jira asset has to be continuous and not one time as that beats the purpose of having an effected CMDB. I got it working as well but the level of security added will be freaking awesome.

[Ste404]

Is there a way to leverage this to pull Intune data in for example, would be amazing to just be able to control that easily in this manner.

[Jason Chan]

Is it possible this web address can refer to an attachment of Confluence or from an issue? 

[Rodney Estrada]

Anyone know of documentation or tutorials specifically for creating the "Send web request" using a Jamf token?

[Yash Bhandarkar]

Hi Team,

I have a CSV file through which I import the data in Asset DB cloud regularly.

I want to have a functionality such that I want to make changes to this CSV file & when the changes are done, the data should get updated in Asset DB Automatically.

Is this possible using Importing the data through web address where the file is kept in One Drive and I am adding the Web Address of the file location of the sharepoint ? Please confirm over this ?

Or is this possible using Automation or Rest API ? Please confirm over this ?

Regards,

Yash

[Dirk Ronsmans]

@Yash Bhandarkar ,

Neither of those are possible I'm afraid. 

A linked file (to a web address) can only be if there is no authentication in front of it (so a public file) and I doubt you'd want to make your internal information which goes in to your assets publicly available on the internet..

Automation can trigger an import but not update the file it should read from.

You could use the external import but that requires development on your side.

https://developer.atlassian.com/cloud/assets/imports-rest-api-guide/workflow/

Next year the Assets Data Manager should become available and that should allow what you want to do, but for now either develop it yourself or periodically upload the file manually.