It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Elastic Experiment Executor

I am a developer of Awesome Graphs from the StiltSoft team.

We are in the process of making the app compatible with the new Data Center program. For this purpose, we used the Performance Testing Framework for Bitbucket to run an experiment in our own environment. We managed to set up the framework and run the workload, but we have some trouble with analyzing the data (the Analyzing Your Experiment Data step).

I am running e3 experiment, and having problems with parsing some of the metrics.

2018-02-15 13:08:48,991 - MainThread - INFO - Graphing node 'cluster-node-001' for stage 'stage-000' of thread 'thread-001' of run 'first-blood'
2018-02-15 13:08:49,719 - MainThread - INFO - Graphing node 'worker-node-001' for stage 'stage-000' of thread 'thread-001' of run 'first-blood'
2018-02-15 13:08:49,719 - MainThread - INFO - Graphing node 'file-server-001' for stage 'stage-000' of thread 'thread-001' of run 'first-blood'
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/gantt-thread-001-stage-000.gnuplot
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/times-thread-001-stage-000.gnuplot
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000.gnuplot
line 0: Non-numeric string found where a numeric expression was expected
cat: stdout: Broken pipe
line 0: Non-numeric string found where a numeric expression was expected
cat: stdout: Broken pipe
line 0: Non-numeric string found where a numeric expression was expected
cat: stdout: Broken pipe
line 0: Non-numeric string found where a numeric expression was expected
cat: stdout: Broken pipe
gnuplot> plot "<(cat /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/thread-001/cluster-node-001/collectd-csv/localhost/GenericJMX-com.atlassian.bitbucket_HostingTickets/gauge-QueuedRequests-????-??-??)" using ($1-start/1000):2 with filledcurves x1 fs transparent solid 0.5 noborder lc 1 title "cluster-node-001 GenericJMX-com.atlassian.bitbucket_HostingTickets/gauge-QueuedRequests"

line 0: all points y value undefined!

Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/throughput.gnuplot
Warning: empty y range [0:0], adjusting to [0:1]
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/histogram-thread-001.gnuplot
Traceback (most recent call last):
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 104, in <module>
calculate_histogram(grinder_files, workload_list, sys.stdout)
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 65, in calculate_histogram
tps = float(fields[6])
ValueError: invalid literal for float(): 10,62
line 0: warning: Skipping data file with no valid points
Traceback (most recent call last):
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 104, in <module>
calculate_histogram(grinder_files, workload_list, sys.stdout)
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 65, in calculate_histogram
tps = float(fields[6])
ValueError: invalid literal for float(): 10,62
line 0: warning: Skipping data file with no valid points
Traceback (most recent call last):
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 104, in <module>
calculate_histogram(grinder_files, workload_list, sys.stdout)
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 65, in calculate_histogram
tps = float(fields[6])
ValueError: invalid literal for float(): 10,62
line 0: warning: Skipping data file with no valid points
Traceback (most recent call last):
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 104, in <module>
calculate_histogram(grinder_files, workload_list, sys.stdout)
File "/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py", line 65, in calculate_histogram
tps = float(fields[6])
ValueError: invalid literal for float(): 10,62
line 0: warning: Skipping data file with no valid points
gnuplot> plot "<(/Users/yorlov/Projects/elastic-experiment-executor/e3/analysis/gnuplot/CalculateHistogram.py /Users/yorlov/Projects/elastic-experiment-executor/e3-home/workloads/bitbucket-mixed.json /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/thread-001/worker-node-001/stage-000/e3-worker-node1-0.log)" using 2:xtic(1), for [i=3:5] "" using i
^
line 0: x range is invalid
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/mean-test-time-001.gnuplot
Warning: empty y range [0:0], adjusting to [0:1]
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/mean-test-time-002.gnuplot
Warning: empty y range [0:0], adjusting to [0:1]
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/mean-test-time-003.gnuplot
Warning: empty y range [0:0], adjusting to [0:1]
Plotting /Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/mean-test-time-004.gnuplot
Warning: empty y range [0:0], adjusting to [0:1]
montage: improper image header `/Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000-00006.png' @ error/png.c/ReadPNGImage/4231.
montage: improper image header `/Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000-00004.png' @ error/png.c/ReadPNGImage/4231.
montage: improper image header `/Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000-00001.png' @ error/png.c/ReadPNGImage/4231.
montage: improper image header `/Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000-00002.png' @ error/png.c/ReadPNGImage/4231.
montage: improper image header `/Users/yorlov/Projects/elastic-experiment-executor/e3-home/runs/first-blood/analysis/collectd-thread-001-stage-000-00003.png' @ error/png.c/ReadPNGImage/4231.

Looking at the worker logs (e3-worker-node1-0.log) it's easy to see, that PushChangesSsh provisioning step is failing on all worker threads, so I supposedthat metrics parsing errors, at the end of the experiment, happen because of incomplete provisioning step (in other words, some metrics are just missed, because there is no data in BB).

Tests resulting in error only contribute to the Errors column.
Statistics for individual tests can be found in the data file, including
(possibly incomplete) statistics for erroneous tests. Composite tests
are marked with () and not included in the totals.

Looking at the exception stacktrace, it looks for me that worker just cannot find ssh users for BB instances (arrays with BB users is empty, so we have IndexOutOfBoundsExceptions)

Experiment configuration (first-blood.json) as well as node (e3-app-node-1.json) and worker (e3-worker-node-1.json) descriptions are attached.

Could you suggest what to do in this situation? 

1 answer

Hi Yuri

Were you able to resolve issues? I'm working for Mohami and we are going to use E3 framework for same purpose but we run into similar issues.

I figured out that changing [logger_command] to DEBUG in e3/logging.conf may help with troubleshooting.

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

4,053 views 4 22
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you