Automation smart variable from BitBucket web request fails to format properly

[Fede B.]

Hi,

I'm trying to build a variable (that I'll later send to a slack message) with all an issue PR's data. To retrieve the data, I use a webhook that calls api1 issue/detail with stash appType:

rest/dev-status/1.0/issue/detail?issueId={{id}}&applicationType=stash&dataType=pullrequest

The call to the webhook is successful (I get 200) and returns as expected a big json with all PR's data in this variable:

{{webResponse.body.detail.pullRequests}}

I have a Story which is linked in BB to 2 open PR's. My problem begins when I try to use smart values list handling methods, to format a message that includes both PR's relevant data:

{{#webResponse.body.detail.pullRequests)} • {{author.name}}'s {{status.toLowerCase()}} <{{url}}PR with id {{id}}>{{#if(equals(status,"OPEN"))}}, which need to be reviewed by {{#reviewers}}{{name}}{{/}}{{/}}\n{{/}}

The expected result should be a text like this:

   • John Doe's open PR with id #1, which needs to be reviewed by Mark Twain, Virginia Woolf

   • Jane Smith's open PR with id #2, which needs to be reviewed by Ernest Hemingway

, while instead I get a text like this:

   • John Doe, Jane Smith's open, open PR with id #1, #2

I would expect the '#' operator at the beginning of the webResponse variable, to make the subsequent methods iterate all entries for pullRequests array one by one, but instead they're somehow flattened.

The same problem (I bet) prevents the {{#if}} for status OPEN to be successful.

Any hints about what I'm doing wrong?

Best regards,

Federico

[Bill Sheboy · Accepted solution]

Hi @Fede B. -- Welcome to the Atlassian Community!

I see a typo in your outermost iterator, with a parenthesis instead of a curly bracket.

Substituting that bracket, and adding the comma handling for the reviewer delimiting, is below.  I added extra spacing to see it better, so remove that after you have it tested and working as desired.

{{#webResponse.body.detail.pullRequests}}
  • {{author.name}}'s {{status.toLowerCase()}} <{{url}}PR with id {{id}}> 
    {{#if(equals(status,"OPEN"))}}, which need to be reviewed by 
      {{#reviewers}}{{name}}{{^last}}, {{/}}{{/}}
    {{/}}
    \n
{{/}}

Also, have you confirmed the conditional check on status is correct?  I recall from similar posts that the if() could fail if the case does not match for the status.

For expressions like this with nested iterators, I find it easier to write them in another tool to help see the matching operators / tokens, and then copy-and-paste into the rule.

Kind regards,
Bill

[Fede B.]

Hi @Bill Sheboy,

Thanks for your hint, the parenthesis was just a typo due to autocorrection when copying&pasting, but in my automation rule the code was ok.

I used 

{{#webResponse.body.detail.pullRequests}}
  • {{author.name}}'s {{status.toLowerCase()}} <{{url}}PR with id {{id}}> 
    {{#if(equals(status,"OPEN"))}}, which need to be reviewed by 
      {{#reviewers}}{{name}}{{^last}}, {{/}}{{/}}
    {{/}}
    \n
{{/}}

as you suggested, but got the same result:

  • John Doe, Jane Smith's open, open PR with id #1, #2

This is the (sanitized) payload that I get from my web request:

[
  {
    "author": {
      "name": "John Doe",
      "avatar": "https://bitbucket.example.com/users/jdoe/avatar.png?s=1&v=0000000000001"
    },
    "id": "#1",
    "name": "Feature/XXX-1 Test PR 1",
    "commentCount": "0",
    "source": {
      "branch": "feature/XXX-1-test-pr-1",
      "repository": {
        "name": "myRepo",
        "avatar": "https://bitbucket.example.com/projects/TEST/avatar.png?s=2&v=0000000000002",
        "avatarDescription": "TEST",
        "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/browse"
      },
      "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/commits?until=refs%2Fheads%2Ffeature%2FXXX-1-test-pr-1"
    },
    "destination": {
      "branch": "master",
      "repository": {
        "name": "myRepo",
        "avatar": "https://bitbucket.example.com/projects/TEST/avatar.png?s=3&v=0000000000003",
        "avatarDescription": "TEST",
        "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/browse"
      },
      "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/commits?until=refs%2Fheads%2Fmaster"
    },
    "reviewers": [
      {
        "name": "Mark Twain",
        "avatar": "https://bitbucket.example.com/users/mtwain/avatar.png?s=4&v=0000000000004",
        "approved": "false"
      },
      {
        "name": "Virginia Woolf",
        "avatar": "https://bitbucket.example.com/users/vwoolf/avatar.png?s=5&v=0000000000005",
        "approved": "false"
      }
    ],
    "status": "OPEN",
    "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/pull-requests/1",
    "lastUpdate": "2024-01-01T18:47:09.000+0200"
  },
  {
    "author": {
      "name": "Jane Smith",
      "avatar": "https://bitbucket.example.com/users/jsmith/avatar.png?s=6&v=0000000000006"
    },
    "id": "#2",
    "name": "Feature/XXX-1 Test PR 2",
    "commentCount": "0",
    "source": {
      "branch": "XXX-1-test-pr-2",
      "repository": {
        "name": "myRepo",
        "avatar": "https://bitbucket.example.com/projects/TEST/avatar.png?s=7&v=0000000000007",
        "avatarDescription": "TEST",
        "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/browse"
      },
      "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/commits?until=refs%2Fheads%2FXXX-1-test-pr-2"
    },
    "destination": {
      "branch": "master",
      "repository": {
        "name": "myRepo",
        "avatar": "https://bitbucket.example.com/projects/TEST/avatar.png?s=8&v=0000000000008",
        "avatarDescription": "TEST",
        "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/browse"
      },
      "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/commits?until=refs%2Fheads%2Fmaster"
    },
    "reviewers": [
      {
        "name": "Ernest Hemingway",
        "avatar": "https://bitbucket.example.com/users/ehemingway/avatar.png?s=9&v=0000000000009",
        "approved": "false"
      }
    ],
    "status": "OPEN",
    "url": "https://bitbucket.example.com/projects/TEST/repos/myRepo/pull-requests/2",
    "lastUpdate": "2024-01-01T18:47:17.000+0200"
  }
]

Best regards,

Federico

[Fede B.]

At the moment the only (ugly/unsustainable) solution that I found, is to store the above json in an Automation variable, which converts the json to string, and then use split/replace/substring methods to format the result. 

[Fede B.]

I also tried a simplified version of the format:

{{#webResponse.body.detail.pullRequests}}
• {{author.name}}'s
{{/}}

The above method returns: 

  • John Doe, Jane Smith's

Instead of

  • John Doe's

  • Jane Smith's

There must be something wrong with the format of the webresponse itself.

Because I have tried to use a similar method for a lookupissue variable, and it worked fine:

{{#lookupIssues}}
• {{key}},
{{/}}

, returns:

  • XXX-1,

  • XXX-2,

  • XXX-3,

[Bill Sheboy]

Based on your experiment with just the author element, it appears this call's webresponse is returning arrays of values (with some as plain text) rather than nested lists.  Or...that is how the rule is getting them in the Send Web Request action's response.

You may be stuck doing the text handling, as you tried earlier.

I am curious...what do you see if you writ this to the audit log:

PR count: {{webResponse.body.detail.pullRequests.size|0}}

That should only return a non-zero count if the rule interprets it as a list.

[Fede B.]

Hi @Bill Sheboy ,

nice hint indeed.

I tried it and I see:

PR count: 1

So we have a list. But the count isn't correct, as I have 2 pull requests linked to the ticket (as it can be seen in the above json output).

I'm wondering, whether smart value has a method similar to Java's getClass(), so that I can use it to understand which type Im trying to manipulate.

[Bill Sheboy]

It appears the entire array is being interpreted as one item.  You could make the text parsing a little easier by initially splitting on a known attribute, such as "author", and then using match() and split again to get at the reviewers as a list.

Regarding the conversion function, that is not available yet.  I recall seeing a suggestion to add an automation rule, text function which can interpret well-formed JSON as text so it can be parsed with attributes.

[Fede B.]

Yep, that's indeed the only solution I found so far, which is ugly and took me a lot of time of work.

I initially use separators to identify the portions of text I'm interested in:

{{#stashDetail.substring(1).split("\{author=\{name=")}}{{replaceAll("\, avatar(.*)\, id\=","ENDAUTHORBEGINID").replaceAll("\, name\=(.*)\, reviewers\=\[(.*)\]","ENDIDBEGINREVIEWERS$2").replaceAll("\, avatar\=([a-zA-Z0-9\:\/\.\?\&\%\=\-]*)","").replaceAll("\, status\=(.*)","ENDREVIEWERSBEGINSTATUS $1").replaceAll("\, url\=(.*)","ENDSTATUSBEGINURL$1").replaceAll("\, lastUpdate\=(.*)","ENDURLENDPRBEGINPRBEGINAUTHOR")}}FEDERICO{{/}}

And then I write a second variable looking for those separators:

{{#prFormattedData.split("FEDERICO")}}
• author: {{substring(0,indexOf("ENDAUTHOR"))}}
• id: {{substring(indexOf("BEGINID"),indexOf("ENDID")).replaceAll("BEGINID","")}}
• url: {{substring(indexOf("BEGINURL"),indexOf("ENDURL")).replaceAll("BEGINURL","")}}
• reviewers: {{substring(indexOf("BEGINREVIEWERS"),indexOf("ENDREVIEWERS")).replaceAll("BEGINREVIEWERS","")}}
• status: {{substring(indexOf("BEGINSTATUS"),indexOf("ENDSTATUS")).replaceAll("BEGINSTATUS","")}}
{{/}}

In this way I managed to write (for instance) slacks URLs in the form <httplink|text>, and so on.

I remember that when I was working with Datacenter + Groovy, and retrieved stash pr data, I had more or less the same problem, and I was forced to parse the got result as a String. Maybe this is just the way that BB spits out the PR data, or possibly a bug in BB itself.

For now I'll close this thread and accept your answer, thanks for the support so far :)