Performance testing of Jira, Confluence and Bitbucket with dc_app_performance_toolkit Part7

[#1]

Part 6

results.csv

Summary of the results for the test run. It contains information about response time for all selenium and jmeter tests:

Label,# Samples,Average,Median,90% Line,95% Line,99% Line,Min,Max,Error %
jmeter_login_and_view_dashboard,4,15996,12216,29332,31667,33536,5548,34003,25.0
jmeter_view_page,18,9950,7290,21767,25701,31178,1041,32547,11.0
jmeter_view_page_tree,16,3422,1001,10405,13115,19515,700,21115,100.0
jmeter_recently_viewed,2,5068,5068,8533,8966,9312,738,9399,0.0
jmeter_search_results,2,21082,21082,36756,38716,40283,1488,40675,50.0
jmeter_view_blog,2,21786,21786,30568,31666,32544,10807,32764,50.0
jmeter_view_dashboard,2,6070,6070,6424,6468,6503,5629,6512,0.0
jmeter_create_page_editor,2,6514,6514,7286,7382,7460,5549,7479,100.0
jmeter_create_page,2,12844,12844,17641,18240,18720,6848,18840,0.0
jmeter_open_editor,2,20412,20412,32523,34037,35248,5272,35551,100.0
jmeter_edit_page,2,45840,45840,76536,80373,83443,7470,84210,50.0
jmeter_comment,2,1176,1176,1293,1307,1319,1029,1322,0.0
jmeter_upload_attachment,2,3639,3639,3922,3958,3986,3285,3993,0.0
jmeter_create_blog_editor,1,7213,7213,7213,7213,7213,7213,7213,100.0
jmeter_create_blog,1,15801,15801,15801,15801,15801,15801,15801,0.0
jmeter_view_attachment,1,1000,1000,1000,1000,1000,1000,1000,0.0
jmeter_like_page,1,2020,2020,2020,2020,2020,2020,2020,0.0
selenium_login:open_login_page,17,1355,538,2404,5970,9466,342,10340,0.0
selenium_login:login_and_view_dashboard,17,1977,1660,2917,3929,7006,1027,7775,0.0
selenium_login,17,3456,2317,5472,10064,16635,1520,18278,0.0
selenium_view_page,17,2310,734,3046,8832,19338,599,21964,0.0
selenium_create_page:open_create_page_editor,17,5715,1623,7635,23837,50605,1341,57297,0.0
selenium_create_page:save_created_page,17,5462,3789,10068,17413,19791,3056,20385,0.0
selenium_create_page,17,12186,6244,23697,33888,66310,5331,74415,0.0
selenium_edit_page:open_create_page_editor,16,4595,3460,8714,13476,14733,2,15047,0.0
selenium_edit_page:save_edited_page,15,3365,2567,4053,6707,10897,1975,11945,0.0
selenium_edit_page,16,8029,6406,15966,16456,17122,2,17288,0.0
selenium_create_comment:write_comment,15,2522,2147,4070,4833,4875,1813,4886,0.0
selenium_create_comment:save_comment,15,1251,1146,1652,1669,1685,1120,1689,0.0
selenium_create_comment,16,4187,3982,6027,6661,7047,2,7143,0.0
selenium_view_blog,16,7152,4360,13916,16767,23327,2,24967,0.0
selenium_view_dashboard,16,5155,3646,5286,12210,27374,1,31165,0.0
selenium_log_out,16,5115,862,1096,18352,59433,1,69703,0.0

results_summary.log

This file contains a short log of the test run:

Summary run status                                FAIL
                                             
Artifacts dir                                     2020-07-24_14-32-11                               
OS                                                macOS                                             
DC Apps Performance Toolkit version               3.0.1                                             
Application                                       confluence 7.4.1                                  
Dataset info                                      975425 pages                                      
Application nodes count                           2                                                 
Concurrency                                       2                                                 
Expected test run duration from yml file          2700 sec                                          
Actual test run duration                          793 sec                                           
Finished                                          (False, 'Actual test duration 793 sec < than expected test_duration 2700 sec.')
Compliant                                         (False, 'Test run duration 793 sec < than minimum test duration 2700 sec. Test run concurrency 2 < than minimum test concurrency 200.')
Success                                           (False, 'One or more actions have success rate < 95.0 %.')

Action                                            Success Rate                                      Status                                            
jmeter_comment                                    100.0                                             OK                                                
jmeter_create_blog                                100.0                                             OK                                                
jmeter_create_blog_editor                         0.0                                               Fail                                              
jmeter_create_page                                100.0                                             OK                                                
jmeter_create_page_editor                         0.0                                               Fail                                              
jmeter_edit_page                                  50.0                                              Fail                                              
jmeter_like_page                                  100.0                                             OK                                                
jmeter_login_and_view_dashboard                   75.0                                              Fail                                              
jmeter_open_editor                                0.0                                               Fail                                              
jmeter_recently_viewed                            100.0                                             OK                                                
jmeter_search_results                             50.0                                              Fail                                              
jmeter_upload_attachment                          100.0                                             OK                                                
jmeter_view_attachment                            100.0                                             OK                                                
jmeter_view_blog                                  50.0                                              Fail                                              
jmeter_view_dashboard                             100.0                                             OK                                                
jmeter_view_page                                  88.89                                             Fail                                              
jmeter_view_page_tree                             0.0                                               Fail                                              
selenium_a_login                                  100.0                                             OK                                                
selenium_create_comment                           93.75                                             Fail                                              
selenium_create_page                              94.12                                             Fail                                              
selenium_edit_page                                93.75                                             Fail                                              
selenium_view_blog                                93.75                                             Fail                                              
selenium_view_dashboard                           87.5                                              Fail                                              
selenium_view_page                                100.0                                             OK                                                
selenium_z_log_out                                93.75                                             Fail                                              

selenium.err

This file contains information about all errors with Selenium webdriver:

Action: test_1_selenium_create_page, Error: webdriver = <selenium.webdriver.chrome.webdriver.WebDriver (session="a14471438bdc54e10792c212a59928c0")>
confluence_datasets = {'blog_id': '52215665', 'blogs': [['52377239', 'TES'], ['52215590', 'TES'], ['52215591', 'TES'], ['52540654', 'TES'], ...'], ['46780039', 'TES'], ['46777426', 'TES'], ['50383897', 'TES'], ['47037799', 'TES'], ['47037475', 'TES'], ...], ...}
confluence_screen_shots = None

    def test_1_selenium_create_page(webdriver, confluence_datasets, confluence_screen_shots):
>       modules.create_confluence_page(webdriver, confluence_datasets)

selenium_ui/confluence_ui.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
selenium_ui/confluence/modules.py:100: in create_confluence_page
    measure()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @functools.wraps(func)
    def wrapper():
        if LOGIN_ACTION_NAME in interaction:
            globals.login_failed = False
        if globals.login_failed:
            pytest.skip(f"login is failed")
        start = time.time()
        error_msg = 'Success'
        full_exception = ''
        try:
            func()
            success = True
        except Exception:
            success = False
            # https://docs.python.org/2/library/sys.html#sys.exc_info
            exc_type, full_exception = sys.exc_info()[:2]
            error_msg = f"Failed measure: {interaction} - {exc_type.__name__}"
        end = time.time()
        timing = str(int((end - start) * 1000))
    
        with open(selenium_results_file, "a+") as jtl_file:
            timestamp = round(time.time() * 1000)
            jtl_file.write(f"{timestamp},{timing},{interaction},,{error_msg},,{success},0,0,0,0,,0\n")
    
        print(f"{timestamp},{timing},{interaction},{error_msg},{success}")
    
        if not success:
            if LOGIN_ACTION_NAME in interaction:
                globals.login_failed = True
>           raise Exception(error_msg, full_exception)
E           Exception: ('Failed measure: selenium_create_page - Exception', Exception('Failed measure: selenium_create_page:save_created_page - TimeoutException', TimeoutException("Error in wait_until: Timed out after 20 sec waiting for <selenium.webdriver.support.expected_conditions.visibility_of_element_located object at 0x10eb8f7f0>. \nLocator: ('id', 'title-text')", None, None)))

selenium_ui/conftest.py:118: Exception

selenium.jtl

This file contains raw information about selenium tests in the test run:

timeStamp,elapsed,label,responseCode,responseMessage,threadName,success,bytes,grpThreads,allThreads,Latency,Hostname,Connect
1595590485432,4877,selenium_login:open_login_page,,Success,,True,0,0,0,0,,0
1595590488565,2967,selenium_login:login_and_view_dashboard,,Success,,True,0,0,0,0,,0
1595590488565,8011,selenium_login,,Success,,True,0,0,0,0,,0
1595590494133,5549,selenium_view_page,,Success,,True,0,0,0,0,,0
1595590509607,15472,selenium_create_page:open_create_page_editor,,Success,,True,0,0,0,0,,0
1595590517792,5666,selenium_create_page:save_created_page,,Success,,True,0,0,0,0,,0
1595590517793,23658,selenium_create_page,,Success,,True,0,0,0,0,,0

After the test run you should have a look at the results.csv and results_summary.log files to understand if the test run was successful or not. If you can see errors in those files then you can investigate the cause of the errors by looking at the Jmeter and Selenium error files.

Part 8

[Dom]

For those looking for part 8...
https://community.atlassian.com/t5/Marketplace-Apps-Integrations/Performance-testing-of-Jira-Confluence-and-Bitbucket-with-dc-app/ba-p/1451559