jemh regexp field processor difficulties

[Vesa Halkka]

I'm more like asking: how can I turn on more debug so that I could follow the decision path when the test case succeeds.. but result is not what I have expected.

So we have Jira 5.2.8 and JEMH 1.3.19. I have been trying to parse Oracle enterprise manager mails, but the luck varies wildly. We have also nagios and vertygo SLA plugins.

There is only one regexp match config block, I have removed the example.

Format of emails is like this:

Self Update name=Self Update
Self Update owner=SYSMAN
Message=An update for Test Data Management templates is available
Severity=Informational
Event reported time=Apr 24, 2013 9:07:47 AM EEST
Event Type=Self Update
Event name=Test Data Management templates

Description=Application Data Model Template for Oracle E-Business Suite 12.1.3(Standard Model)
Version=12.1.0.1.0
Status=Available
Rule Name=EM_UPDATE_RULESET,EM_UPDATE_RULE_NEW
Rule Owner=

I have been only using test cases as our test jira doesn't have email. Test cases pass OK, but the resulting tickets are less perfect:

- the tag for event grouping is visible at ticket, but it won't ever get filled at first round

- sometimes the grouping still succeeds if the field has a matching value, but still fails to get to be filled with a new value. ( tag field is Host.. field might have come with nagios plugin )

- sometimes also the matching fails and a new ticket is always created ( tag field a fresh custom field, type 255 char string )

Also filling values with aliases behaves oddly. So I have defined aliases that should match the EM rule with ticket type:

DEFAULT_RULESET_FOR_ALL_TARGETS,TARGET_AVAIL_DOWN_INCIDENT_CREATION
is aliaset to be issueType Incident

EM_UPDATE_RULESET,EM_UPDATE_RULE_NEWis aliased to be issueType RFC

This *usually* works, but sometimes picks a wrong choice.. still no errors at test case.

[Andy Brook [Plugin People] · Accepted solution]

Hmm, regexp FP supports multiple entries, so in theory you should be able to wrap all those scenarios into a single profile.

If you want to have multiple profiles, that are pointing at a common mailbox, things get interesting. In order for a Profile to 'not touch' an email, it must be blacklisted in that profile, so, If you put the OEM address in the Whitelisting> blacklist section of profileA, then profileB should be able to process the message. Be sure to set Email > Message Filtering > Non Catchmail Match Action : Ignore (leave in mailbox)

This should be used with a bit of caution, as if the OEM consumer stops processing for any reason, then the mail queue will be blocked.

[Vesa Halkka]

Great!

So I'll have them in same profile unless I really need to separate.. I was mostly hoping to keep the Alias lists separate.. same words are used in different systems, and mappings will collide some day.

[Andy Brook [Plugin People]]

Yes, the aliasing was a useful measure that fixed a problem at the time. I didn't want to rewrite all the aliasing support again just for one field proc, when it was already used in another. At the point you get collisions, a new profile will be needed.

[Andy Brook [Plugin People]]

if the abonve answers your question, please mark it so?

[Somu Ramanathan]

I am using JEMH as Incoming Mail handler. I need to digest based on tag and create a ticket in JIRA.

[Description] This is a description [/Description].

I have written RegEx to extract based on fields and their tags. It works for other fields but it doesn't works for "Description" field alone. Instead whole email gets written to "Description" field.

Is there any way to do it.

[Vesa Halkka]

To be exact the current setup has four sources, all with a single sender email address and different domain:

- nagios ( using nagios )

- a test input queue ( using something basic )

- another Jira ( using regex )

- Oracle enterprise manager ( using regex )

So if I have nagios & Jira at same profile ( 'nagios') apparently they both work, and if I configure OEM regex inside 'nagios' profile it works too, BUT if I have OEM in it's own profile ( oem ) the test cases pass ( If I pick oem profile ) but it never gets the emails as 'nagios' profile decides that this doesn't match any project and the email is rejected.. instead of passing it to next profile.

[Andy Brook [Plugin People]]

The current regexp fieldproc needs an exact match on the sender, as that was my original thought on how it would be used, however, if its the case where you have multiple senders, it wont (currently) work, I have a patch in the next 1.3.23 release that will allow a regular expression to be used, so .* will then match all senders.

Soo, guessing thats why the regepx match failed, you should be able to prove correct outcome with a Test Case, updating the from: address as needed to prove aliases are being applied as expected too?

Some other trailing thoughts:

- be sure that your REgexp FP embedded test content is actually the content taken from an issue creation.

- check yourcontent types, if you have different email sources, some may be text, some may be HTML. The conversion from HTML to text is not the same, may break regexp. This is one reason why the from: address was a direct match, so that you work with a known consistent source. Introducing variables will likely end up with a non-match when you'd hoped for one. Converting mails to test cases via Audit History is the best way to track these things down...

[Vesa Halkka]

Hi,

Yep.. It was picking default and when I removed defaults from everywhere else it finally used 'bug', but apparently the email was matching as there was only one regex config, and parts of that were successfully executed. So I'd suspect my aliases, though I couldn't see any typos there.

So another FAQ.. how could I use multiple profiles ? It looks like the system executes only one profile and if it doesn't match then project mapping fails.. the 'leave email to the box if it's not for you' -setting doesn't seem to help here.. the mailbox is empty.. hmm.. this might be an email server thing as well.

[Andy Brook [Plugin People]]

Hi Vesa, Its possible, but possible also, is that if the sender address doesnt match, then the default settings would be used, that could confusing if the 'default' is the 'second' issueType you wanted to select?

[Vesa Halkka]

Thanks, sorry for the delay. I'll turn on the debug, I still have some mysteries to solve, but probably I just don't understand all dependencies well enough yet. The fix for most random problems at field behaviour was simply re-index... I don't really understand the connection, but maybe it kinda fixed the caches after my experiments ?

[Andy Brook [Plugin People]]

Hi Vesa,

Re: :logging, you can get a detailed JEMH-specific logfile by following instructions at:

- https://thepluginpeople.atlassian.net/wiki/display/JEMH/Installation%2C+Setup+and+Logging

I can't really help on the forum, the Regexp Field Proc is quite complex, to advise, please create a support ticket on JEMH Jira , and attach your Profile XML export, and an example TestCase.

Thanks!