Automated API tests in Bamboo

[argentina23]

Good morning, Everyone.

I am Natasha Kramarenko, QA manager at Salmat ltd.

My team is working on API test automation and part of CI.

We're creating JMeter scripts to check functionality of API and setting the tests up running on Bamboo.

There are a couple of options available for Bamboo builds to run JMeter scripts. All of them are rather limited.

Option 1. JMeter aggregator plugin for Bamboo. Works ok, but unsupported, hence, won't even get installed by our IT team.

Option2. Blazemeter test trigger plugin for Bamboo. It is ok, but

a) Blazemeter as such won't allow you to connect to local DB as part of your test.

b) plugin is not commonly used and you need to regularly contact support to resolve setup issues

c)  plugin is updated once a year, byt request, less often than Bamboo itself.

Option 3. Your option .....

Please share your experience in this area!

I am surprised how Atlassian still is not supporting API tests on Bamboo, as they specify this as an important part of CI.

https://www.atlassian.com/agile/continuous-integration

Thank you,

Natasha

[jim weaver]

Do you really need a plugin?  I've not used JMeter, but it looks like it can be run from Ant, Maven, or Java.  https://www.blazemeter.com/blog/5-ways-launch-jmeter-test-without-using-jmeter-gui

Bamboo can launch all those things with built-in builders already.

I do see a fair amount of API testing through Bamboo, but generally I see it done from unit tests that hit a deployed API - so they are independent of any API testing framework like Postman, etc.

[argentina23]

Hi Jim,

Thank you for the feedback.

It's indeed not a problem to run JMeter on Bamboo. It can be done simple from the script task.

"sh apache-jmeter-3.1/bin/jmeter \

jmx file detailss \

jtl file details \"

Question is how to link JMeter test failure with build failure. 

I've tried com.lazerycode.jmeter 
solution locally on my PC, it generated HTML report, same as Ant task. 

What I need is to link results to the Test tab in Bamboo.

[jim weaver]

Hi Natasha,

I got ya.  Again I haven't used JMeter, so I should probably shut up ;-), but I have had success in the past wiring test tools into Bamboo by using xUnit - in other words, if it is a Java app or you have someone comfortable with Java, if you can drive the JMeter test from Junit, or you can configure JMeter's runner itself to output an xUnit XML report format, Bamboo already knows how to parse and report xUnit reports, and will fail a build if it finds a failed test inside an XML xUnit report format.  You can use the JUnit Parser task to process such a file in your working directory.

Hmm, here's a stack overflow thread with someone asking that question:  https://stackoverflow.com/questions/43477077/convert-jmeter-reports-to-junit-reports

[argentina23]

The solution was https://github.com/tguzik/m2u.