insight external import - how to patch import mapping with reference attributes ...

[ian]

Hello. I'm struggling my way through trying to create an external import using the info here:

Insight external imports workflow (atlassian.com)

External Imports - Schema and mapping (atlassian.com)

Finally got the import working until I introduced reference attributes. I've updated the schema JSON but when I PATCH using the updated JSON schema I get this error:

{"jsonValidationErrors":[],"dataValidationErrors":[{"code":"REFERENCED_OBJECT_TYPE_CHANGED","location":"testEmployee.Office"},{"code":"REFERENCED_OBJECT_TYPE_CHANGED","location":"testEmployee.Manager"}],"status":"error"}

This is the JSON schema definition I'm using - if I remove the reference attributes then PATCH works:

{
"$schema": "https://api.stg.atlassian.com/jsm/insight/imports/external/schema/versions/2021_09_15",
"schema": {
"objectSchema": {
"name": "Testing",
"description": "Testing schema",
"objectTypes": [
{
"name": "testEmployee",
"description": "my test employee",
"attributes": [
{
"name": "Name",
"description": "DisplayName of the Employee",
"type": "text",
"label": true
}, {
"name": "FirstName",
"description": "First name of employee",
"type": "text"
}, {
"name": "LastName",
"description": "Last name / Surname of employee",
"type": "text"
}, {
"name": "Office",
"description": "Primary office of employee",
"type": "referenced_object",
"referenceObjectTypeName": "testOffice"
}, {
"name": "Manager",
"description": "Manager of the employee",
"type": "referenced_object",
"referenceObjectTypeName": "testEmployee"
} 
]
},
{
"name": "testOffice",
"description": "my test office",
"attributes": [
{
"name": "Name",
"description": "Name of the office",
"type": "text",
"label": true
}
]
}
]
}
},
"mapping": {
"objectTypeMappings": [
{
"objectTypeName": "testEmployee",
"selector": "",
"description": "Import mapping for testEmployee",
"attributesMapping": [
{
"attributeName": "Name",
"attributeLocators": [
"DisplayName"
],
"externalIdPart": true
}, {
"attributeName": "FirstName",
"attributeLocators": [
"FirstName"
]
}, {
"attributeName": "LastName",
"attributeLocators": [
"LastName"
]
}, {
"attributeName": "Office",
"attributeLocators": [
"Office"
],
"objectMappingIQL": "Name = ${Office}"
}, {
"attributeName": "Manager",
"attributeLocators": [
"Manager_DisplayName"
],
"objectMappingIQL": "Name = ${Manager_DisplayName}"
}
]
}
] 
}
}

The attributes are defined like this:

I expect I'm not understanding something fundamental but cannot see it at present. Hoping someone else can point me in the right direction.

For reference, I'm using Microsoft Flow to update Insight with information from a SharePoint (Microsoft 365) list. This list consolidates information from various LOB apps. The plan is to schedule this Flow to update Insight regularly.  

Thanks.

Ian.

[ian · Accepted solution]

Thanks to Support for figuring out that the JSON I was submitting to the mapping was wrong!

If the reference attributes already exist in the schema objectType, then DO NOT include them in the objectSchema section of the JSON but ONLY include them in the mapping section.

If the reference attributes do not yet exist in the schema objectType, then DO include them in the objectSchema section of the JSON.

Here is the JSON file that finally worked for me. Notice that the Office and Manager reference attributes are only referred to in the mapping section:

{
"$schema": "https://api.stg.atlassian.com/jsm/insight/imports/external/schema/versions/2021_09_15",
"schema": {
"objectSchema": {
"name": "Testing",
"description": "Testing schema",
"objectTypes": [
{
"name": "testEmployee",
"description": "my test employee",
"attributes": [
{
"name": "Name",
"description": "DisplayName of the Employee",
"type": "text",
"label": true
}, {
"name": "FirstName",
"description": "First name of employee",
"type": "text"
}, {
"name": "LastName",
"description": "Last name / Surname of employee",
"type": "text"
} 
]
},
{
"name": "testOffice",
"description": "my test office",
"attributes": [
{
"name": "Name",
"description": "Name of the office",
"type": "text",
"label": true
}
]
}
]
}
},
"mapping": {
"objectTypeMappings": [
{
"objectTypeName": "testEmployee",
"selector": "",
"description": "Import mapping for testEmployee",
"attributesMapping": [
{
"attributeName": "Name",
"attributeLocators": [
"DisplayName"
],
"externalIdPart": true
}, {
"attributeName": "FirstName",
"attributeLocators": [
"FirstName"
]
}, {
"attributeName": "LastName",
"attributeLocators": [
"LastName"
]
}, {
"attributeName": "Office",
"attributeLocators": [
"myOffice"
],
"objectMappingIQL": "Name = ${myOffice}"
}, {
"attributeName": "Manager",
"attributeLocators": [
"myManager"
],
"objectMappingIQL": "Name = ${myManager}"
}
]
}
] 
}
}

For reference:

This was my objectType before creating the import mapping:

And this was the import mapping created using the above JSON:

[Chris Thomas]

Hi Ian,

Your post helped me out a lot and got me over my roadblock. Thanks for posting the detailed response with the screenshots!

[Adam Repaský]

Thank you Jan, I had exactly this problem and now thanks to your detailed post It is solved!

[ian]

I've asked Support to have a look at this and point out what I'm doing wrong. Will add their answers here.

In the meantime ...

1. Wondering if the schema reference attributes will work when first creating the external import rather than when patching. Will try this soon. I've been using patch as I add different attribute types to the external import and then test.

Forgot to mention in my original comment: the Office reference attribute was successfully created the first time i patched the external import with it included in the schema. However, every time since I get the error when I patch.   

2. Import data into non-reference attributes - type = text, number etc. Then use a scheduled global automation rule to take these attribute values and update the reference attribute. Or maybe better to trigger the rule from my Flow using the 'Incoming webhook' trigger.

[ian]

Created new external import, generated new token and using Postman, tried to create the mapping using the json schema in my original question/comment. Same error:

So that leaves me the second option - non-reference attributes and global automation.

Of course it's also very possible that my JSON schema is wrong as I'm quite new to this and cannot claim to understand the docs completely. :)

[ian]

Option 2 works: importing non-reference attributes + webhook trigger JSM automation that updates the reference attributes

Would prefer to use reference attributes directly in the import so will wait for a bit to see if someone can help me understand what I'm doing wrong before I suggest the above as the answer. 

[Mickey_Corriveau]

i'm having the same issue, my schema mapping worked before changing instance, I don't get why it's happening 

[ian]

@Mickey.Corriveau I assume that you looked at the accepted answer above and it was not helpful. As I'm still pretty new at this, I'm sorry but not sure how much I can help you. I would certainly need more info.

The fact that your schema worked before "changing instance" (not sure exactly what "instance" means) suggests to me that the cause of your problem may be different to mine.   

You may also get more traction/response by lodging your own request with more info like your schema, errors and what you have tried so far. Or seek help from the Support team. I certainly should have sought help earlier than I did as I wouldn't have wasted as much time!