Release 4.0.0

Version 4.0.0

Added

• Jira Service Management support
  • JSM LTS 4.5.10 and 4.13.2
  • User Guide for JSM
  • JSM “large“ and “small“ datasets
• Bump supported versions:
  • Jira LTS from 8.5.8 to 8.5.11
  • New Jira LTS 8.13.3
  • Confluence LTS from 7.4.4 to 7.4.6
  • Bitbucket LTS from 6.10.5 to 6.10.7
  • New Bitbucket LTS 7.6.2
• Util script start_jmeter_ui.py to simplify usage of JMeter UI for debugging and app-specific actions development.
• Average action timings into results_summary.log.
• Warning if basic action timing exceeds 20 seconds threshold.
• Recommended storage amount to the execution environment section of the User Guide.
• AWS vCPU limit note in the User Guide.
• How to stop the database instructions in the User Guide.

Removed

• Support deprecated:
  • EOL Jira 7.13.x and 8.0.x
  • EOL Confluence 6.13.x

Fixed

• Confluence index-snapshot.sh script to properly check if an index snapshot generation is finished without a need to re-run the script.
• Jira get index nodes logic to count only active and alive nodes.
• Confluence re-index instructions in the User Guide.
• Explanation of the acceptable error rate for full-scale runs in the User Guide.
• Bitbucket language check header.
• Bitbucket selenium_create_pull_request action selector to make the action more stable.
• The naming convention for all actions.
• Confluence JMeter create_page action edge case when the page has an incorrect name.

Changed

• Pin supported Python versions to 3.6-3.8.
• Bump bzt version up to 1.15.1.
• Bump locust version to 1.3.1.
• Bump chromedriver version up to 88.0.4324.27 to support Chrome browser version 88.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.2.0

Version 3.2.0

Added

• Bump supported versions:
  • Jira LTS from 8.5.6 to 8.5.8
  • Confluence LTS 7.4.4 (new)
• Custom dataset filtering feature.
• Support of self-signed SSL certificate - secure: True/False toggle in .yml configuration files.
• Check if a run has app-specific actions in results_summary.log.
• Different levels of warnings and errors logging into bzt.log.
• Check if the toolkit executed against Bitbucket Server.
• Force set a language for Selenium scripts to English.

Fixed

• Locust scripts error handling.
• Fail immediately subsequent locust actions if login action failed.
• Bitbucket Selenium scripts case when a pull request has conflicts.
• Confluence JMeter and Locust scripts to run with a postfix.
• Confluence Selenium write_content method to not rely on a <p> tag.
• Removed hipchat requests from Confluence JMeter and Locust scripts.
• Edge case when project key matched JQL reserved words.

Changed

• Redesigned User Guides:
  • updated workflows
  • improved sections order
  • development and execution environment
  • new examples for JMeter, Locust, and Selenium app-specific actions development
• Bump bzt version up to 1.15.0.
• Bump locust version to 1.2.3.
• Bump chromedriver version up to 86.0.4240.22 to support Chrome browser version 86.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.1.0

Version 3.1.0

Added

• Bump supported versions:
  • Jira LTS from 7.13.6 to 7.13.15
  • Jira LTS from 8.5.0 to 8.5.6
  • Confluence LTS from 6.13.8 to 6.13.13
  • Confluence Platform from 7.0.4 to 7.0.5
  • Confluence LTS 7.4.x is not supported yet (work in progress, support will be added in upcoming releases)
  • Bitbucket LTS from 6.10.0 to 6.10.5
  • Bitbucket Platform from 7.0.0 to 7.0.5
• Instructions on how to stop cluster nodes into User Guides.
• Check if the user stated in the .yml configuration files has sufficient permissions for the run.
• Ability run tests with both RTE enabled and RTE disabled for Jira.

Fixed

• Bug in verify response logic for correct error handling.
• Get Jira nodes count logic to filter only active nodes.
  Confluence index-snapshot.sh script to handle multi snapshot case.

Changed

• Recommended AWS Quickstart CloudFormantion templates parameters to reduce AWS costs.
• Prepare dataset by the performance users, but not by the admin user to reduce data permission issues.
• Read WEBDRIVER_VISIBLE from .yml configuration file instead of the corresponding environment variable.
• Bump ChromeDriver version up to 84.0.4147.30 to support Chrome browser version 84.
• Bump downstream libraries' versions.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.0.1

Version 3.0.1

Added

• Get version functionality in case Jira Server or Confluence Server

Fixed

• Language check for Confluence and Bitbucket in case the host uses https protocol
• Selenium selector when no results found by JQL search

Changed

• Documentation about Jira Index was updated to be more clear
• Recommended Jira Database instance class for better Jira Full Re-Index timing consistency
• Recommended Confluence Database instance class to use the latest generation
• Removed hipchat endpoint from jira.jmx

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 3.0.0

Version 3.0.0

Added

• locust executor and Locust scripts for Jira and Confluence (default load executor is still JMeter).
• Official Docker container with the Toolkit.
• Instruction how to set up and run the Toolkit on an EC2 instance in a Docker container.
• Automatic Base URL update after database restore.
• Nodes count information into results_summary.log.
• Dataset information into results_summary.log.
• Check if the script is running not on the NFS server for Bitbucket.
• Check the product language before the start.
• Check Collaborative Editing is enabled for Confluence.
• Check if the script is running on a bastion host.
• Extra logging into bzt.log.

Fixed

• Bug when Jira DC has more than 8000 projects.
• Utility functions were removed from conftest.py.
• Error handling in case the product version was not found.
• Error handling in case the product URL is invalid.
• Bug with an infinite loop on user creation for Confluence.
• Bug with random project selection for Jira.
• Bug when JMeter script fails on Bitbucket with postfix.

Changed

• print_time decorator was refactored for better readability and simplicity.
• Do not override License on database restore.
• bzt version bump up to 1.14.2.
• ChromeDriver version bump up to 83.0.4103.39 to support Chrome browser version 83.
• Downstream library's versions bump.
• Skip the whole test if login failed.

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Locust executor

In release 3.0.0 new locust executor and Locust scripts were added for Jira and Confluence. Locust (https://locust.io) is an open-source performance tool based on Python requests library. The benefit of using locust executor over jmeter executor is simplicity, extensibility and debugging flexibility. Locust scripts are a simple Python code that is straightforward to run, debug and extend.
jmeter executor is set by default in jira.yml and confluence.yml files. But if you want to give Locust a try or think that JMeter UI is too complicated for issues debugging or new app-specific actions creation just change load_executor value to locust in .yml configuration file.
More details about Locust could be found in README.md files.

App-specific actions change required

DCAPT framework uses print_timing decorator to measure timings of all Selenium actions and sub-actions.
In release 3.0.0 print_timing decorator was refactored for better readability and convenience of use. The signature of the decorator was changed, so if you have app-specific actions created for a previous version of the toolkit you need to update it as shown on an example below. Also, now there is no need to pass interaction into every selenium webdriver step.
Old decorator signature and usage example:

def custom_action(webdriver, datasets):
    page = BasePage(webdriver)
    @print_timing
    def measure(webdriver, interaction):
        @print_timing
        def measure(webdriver, interaction):
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/report")
            page.wait_until_visible((By.ID, 'report_app_element_id'), interaction)

        measure(webdriver, 'selenium_app_custom_action:view_report')

        @print_timing
        def measure(webdriver, interaction):
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/dashboard")
            page.wait_until_visible((By.ID, 'dashboard_app_element_id'), interaction)

        measure(webdriver, 'selenium_app_custom_action:view_dashboard')
    measure(webdriver, 'selenium_app_custom_action')

New decorator signature and usage example:

def custom_action(webdriver, datasets):
    page = BasePage(webdriver)

    @print_timing("selenium_app_custom_action")
    def measure():

        @print_timing("selenium_app_custom_action:view_report")
        def sub_measure():
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/report")
            page.wait_until_visible((By.ID, 'report_app_element_id'))
        sub_measure()

        @print_timing("selenium_app_custom_action:view_dashboard")
        def sub_measure():
            page.go_to_url(f"{JIRA_SETTINGS.server_url}/plugin/dashboard")
            page.wait_until_visible((By.ID, 'dashboard_app_element_id'))
        sub_measure()
    measure()

Release 2.0.0

Version 2.0.0

Added

• Support for Bitbucket 7.0.0
• Results_summary.log file with detailed information of the run
• Scenario summary report
• Check if git installed for Bitbucket
• Post-run check for JMeter health
• Post-run artifacts folder cleanup

Fixed

• Documentation typos
• Bitbucket create_pull_request selenium action to be more realistic

Changed

• Refactored all Selenium code for Jira, Confluence, and Bitbucket to Page Object pattern
• ChromeDriver version bump to 80.0.3987.106 to support Chrome version 80
• Downstream libraries versions bump

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 1.3.0 Beta

Version 1.3.0 beta

Added

• Bitbucket 6.10.0 support
• Check whether Remote API (XML-RPC & SOAP) is enabled for Confluence

Fixed

• Escaping of project keys for Jira
• Removed User-Agent header from JMeter scripts
• Removed w3c timings generation

Changed

• ChromeDriver version bump to 80.0.3987.16 to support Chrome version 80
• JMeter version bump to 5.2.1
• bzt version bump to 1.4.1
• downstream libraries versions bump
• Documentation enhancements

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 1.2.1 Beta

Version 1.2.1 beta

Added

• Support of Python 3.8
• Upgrade instructions into README.md

Fixed

• Projects pagination logic when Jira DC has less than 25 projects
• Exessive logging for attachments untar

Changed

• Improved create page actions for Confluence DC
• Enhanced get all projects method for Jira DC - results limiting was introduced
• Dataset scripts logging improved to be automation-friendly

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 1.2.0 Beta

Version 1.2.0 beta

Added

• Confluence 7.0.4 support
• Environment pre-run checker (Python version validation)
• Analytics collector (see README.md for more details)

Fixed

• Default selenium extension test template for Jira and Confluence
• An issue with Confluence custom application port

Changed

• Improved filtering for Confluence blog posts
• ChromeDriver version bumped up to 79.0.3945.36 to support Chrome 79

Upgrade instructions

• git pull from master branch
• activate virtual env for the toolkit (see README.md for details)
• pip install -r requirements.txt

Release 1.1.1 Beta

Version 1.1.1 beta

Fixed

• README.md to reflect the correct status of supported Atlassian apps