Parse ARN from message field

[Brian Costello]

Hi All,

I am receiving an alert from AWS that goes into the message field in OpsGenie and looks like the below:

{"version":"0","id":"random_string","detail-type":"Alarm State","time":"2020-02-09T17:41:15Z","resources":["arn:....."]}

From the message field in OpsGenie I want to grab the arn and place it into the alias field. I've tried using string processing and regex methods to extract the arn but nothing seems to be working. Please see the attached image for the regex method I am using.

I'm assuming that my regex is incorrect but could anyone advise where or even suggest an alternative way to extract the arn?

Thanks!

[Brian Costello]

Also, I’ve just noticed how many views this post got so I thought I would give a little more in depth info about how I ended up resolving (and how I continue to resolve) any type of parsing with any OpsGenie integration. 

Please note that each integration can vary greatly and the steps laid out below are for the CloudWatchEvents Integration within OpsGenie. However, the steps should be able to be extrapolated to other integrations as well.

Lastly the following steps assume that you are in the OpsGenie UI, using an advanced integration and creating an alert (or some type of output) with that integration.

1. Set up an advanced integration and send a test event into OpsGenie. For now your create alert filter should be set to "Match all alerts" so it will catch anything coming into your integration. For this integration the field that contains all of the info I want is located in the {{detail}} dynamic field. Take notice that the dynamic field you get your information from may vary depending on the integration you are using. Whatever your dynamic field name ends up being be sure to include it in one of the Alert Fields. We initially place it in either the "Note" or "Description" field.

The reason we do this first is because sometimes OpsGenie parses things differently than what you would expect so I would always recommend doing this first to see the result of the OpsGenie parsing magic.

2. Once the event is received by your integration an alert should be created and in that alert you can take a look at how your event is being parsed when it comes into OpsGenie. This is important because that will affect the regex expression you create to get the desired human readable information.
3. Once you have an idea of what's coming in you can create a regex expression to suit your needs. An example of one for this integration and purpose looks like the following:

                                     {{ detail.extract(/.*affectedResource=(.*?),/, 1) }}

(Please note that this regex will change depending on the fields in your integration! If you look above in this post you see that I use findings.extract for SecurityHub and detail.extract for CloudWatchEvents. Also, be sure to confirm the casing of the dynamic field in OpsGenie before using the dynamic expression as I found that can affect whether the expression returns anything or not. You can do this by just dragging and dropping the dynamic fields into an alert field. As you can see some fields are camelCase and other fields aren't).

Place the above regex expression into a separate alert field than where you placed your initial dynamic field, in my case {{detail}}. This will make it easier to see what OpsGenie is spitting out, if anything.

(Also, please note I am not a regex guru so there is definitely a better expression out there but hey this works so that's good enough for me). I found that this website was helpful for visualizing what will be returned by the regex: https://regex101.com/. Make sure you’re using the Java flavor of regex on that site as I found that one closely aligns to OpsGenie’s “Java-like” regex).

4. Once you think you got the regex expression down send another test event and see if OpsGenie parses the information you were expecting. If not see what it did grab, or didn’t, and then repeat step 3 until you get the information you are looking for.

Hope this helps some people!

[John M]

Hi Brian,

The substringBetween string processing method may be easier to use in this case. You could try something like:

{{message.substringBetween("resources\":[\"arn:","\"")}}

[Brian Costello]

Hi @John M 

Tried the above and unfortunately it is still not placing the arn in the alias field. Whenever an alert is created it is still placing a random string as the alias instead of the arn which I believe is due to the fact that the string processing methods we are using above did not find a match and as a result are returning a None value to the field.

Fiddled around with your response to see if I could get anything to appear in the alias field but unfortunately nothing was working.

[John M]

Hi @Brian Costello,

Before I posted that, I tested the string processing code I gave you with the exact message field you posted (below) and it worked for me:

{"version":"0","id":"random_string","detail-type":"Alarm State","time":"2020-02-09T17:41:15Z","resources":["arn:....."]}

Can you please post screenshots of the integration settings as well as the exact message you're processing and the string processing code you are using?

Alternatively, you could open a ticket at support.atlassian.com and we can request access to your account so we can take a look at your setup.

[Radzhiv Apasov]

Hello @John M , I'm trying to get some info from 

Sample Payload from Amazon Security Hub

link is : Integrate Opsgenie with Amazon Security Hub | Opsgenie | Atlassian Support

My goal is to extract information about findings' resources from the "Message" field and present it in a human-readable format. Specifically, I want to retrieve details from the "findings" array, specifically the "Resources" or Resources--> Id object within it.

Any assistance or insights you can provide would be greatly appreciated.

[Brian Costello]

@Radzhiv Apasov 

I have no idea what your infrastructure on the AWS side looks like but assuming you are using the SecurityHub Integration from OpsGenie the following may work:

{{findings.extract(/.*Resources.*Id=(.*?),/, 1)}}

I'm assuming that you are using the advanced SecurityHub Integration within OpsGenie and are creating an alert in OpsGenie from the SecurityHub event. I would just paste that expression into one of the alert fields in your integration and see if you have any hits. 

This could be completely wrong however as I've never directly used the SecurityHub Integration from OpsGenie. That being said it should be pretty close to something that should work based upon the link you gave in your post above. 

[Radzhiv Apasov]

@Brian Costello ,

Thank you for your help. I tried the following expression yesterday: {{findings.extract(/.*Resources.*Id=(.*?),/, 1)}}, and this is what I got:

"Message: pb-08945410d7969a5af"

I found out that it gave me this portion of the findings:

"Resources=[{Partition=aws, Type=AwsEc2Instance, Region=us-east-2, Id=arn:aws:ec2:us-east-2:954375135148:instance/i-0a2b4cf7b32cf5670}], PatchSummary={MissingCount=189, OperationStartTime=2023-08-22T07:00:31Z, InstalledOtherCount=179, OperationEndTime=2023-08-22T07:01:37Z, RebootOption=RebootIfNeeded, FailedCount=0, Operation=SCAN, Id=pb-08945410d7969a5af, InstalledCount=158, InstalledPendingReboot=0, InstalledRejectedCount=0}, ProcessedAt=2023-08-22T07:01:45.931Z}"

I was looking to get the first Id: Id=arn:aws:ec2:us-east-2:954375135148:instance/i-0a2b4cf7b32cf5670

By the way, I tried

{{_parsedData.findings.substringBetween("Resources=[{","}]") }}

and that's what it gave me:

"Resource: Partition=aws, Type=AwsEc2Instance, Region=us-east-2, Id=arn:aws:ec2:us-east-2:688942428108:instance/i-03b16360928f280asg"

Perhaps I should look for 'Resource' only? Because if I use {{_parsedData.findings.substringBetween("Resources=[{","}]") }}, it doesn't always give me what I wanted. I mean, there are a lot of findings, and they generate different formats.

For example, I used {{_parsedData.findings.substringBetween("Resources=[{","}]") }}, and new findings generated this:

Partition=aws, Type=AwsRdsDbCluster, Details={AwsRdsDbCluster={StorageEncrypted=true, ClusterCreateTime=2023-08-22T12:53:42.289Z, ActivityStreamStatus=stopped, HttpEndpointEnabled=false, EngineMode=provisioned, Port=5432, DbClusterResourceId=cluster-AQU3YTQJG3JCRSVZF6AJWUHAKU, VpcSecurityGroups=[{Status=active, VpcSecurityGroupId=sg-08b88225600c32e22

So, there are many brackets and curly brackets in the data.

Do you have any ideas here?

Thank you once again for your help.

[Brian Costello]

@Radzhiv Apasov 

The following worked for me:

{{findings.extract(/.*Resources.*Id=(.*?)}],/, 1)}}

Problem with the first expression I gave you is that it didn't have the closing brackets. This may be a fragile expression though depending on the order of the keys in the returned dictionary but I suppose no need to worry about that until it happens.

However, if you are absolutely certain that the id will always be an ARN you can do the following which should be a bit more robust:

{{findings.extract(/.*Resources.*Id=(arn.*?)}],/, 1)}}

[Radzhiv Apasov]

@Brian Costello , thanks for your help. {{findings.extract(/.*Resources.*Id=(arn.*?)}],/, 1)}} worked for!!!!!! Thank you again.))

[Brian Costello]

@Radzhiv Apasov no problem glad it worked!

Lastly, not sure if you have done so already but you may want to remove or alter the aws account number from the arn you posted previously.

Don't want that info floating around.

[Radzhiv Apasov]

@Brian Costello , I changed account number and etc. before posting it. Thank you!!!