Chapter 7. QA Team Responsibilies

Table of Contents

QA Team
Testing Responsibilities
Working with the QA Trackers
Working with Testcases
Communicating with Testers and Users
Release Responsibilities
Post Release Tasks

There are various QA trackers, wiki and Launchpad pages, the links to which can be found on Useful QA Links.

QA Team

The Xubuntu QA team was formed to ensure that the quality of a released Xubuntu conforms to the parameters laid out in the Xubuntu Processes documents. In order to successfully accomplish this, close team working relationships, especially with the development team and the Xubuntu Council are paramount. Xubuntu's success is based on close working amongst all of its various teams.

In addition, the team gives people contributing through testing of Xubuntu the opportunity to become part of the Xubuntu Team. For that to be of practical use, the QA team should keep an eye on testing reports on the trackers and propose users they have seen taking a keen interest in the work of the QA team.

Excluding testing itself, control of the testcases that we use for ISO and Package testing, along with communicating the testing requirements for any particular development cycle (hereafter cycle) to the community, makes up the bulk of the teams work in any given cycle.

Members of the QA team should check the current Xubuntu QA blueprint, and assign themselves to tasks they feel able to undertake.

Testing Responsibilities

At the start of a cycle, the Release Team will discuss which ISO Milestones we will participate in. Then, during a Community Meeting, members of Xubuntu Team will discuss and then ratify Xubuntu's participation during the cycle.

Along with general testing of our OS, dealt with further in Exploratory Testing and Using Development PPAs, further responsibilities lie with ensuring that:

  • Sufficient testing takes place prior to ISO Milestone releases

  • Sufficient package testing takes place following calls to testers

  • Sufficient testing has taken place by Final Release

  • Bugs reported to our trackers are confirmed, or where unable to confirm, further information is requested from the reporter

  • Where appropriate, confirmation of bugs can be asked of members of Xubuntu Team in the team devel IRC channel

Members of the QA team might find it useful subscribing to the Xubuntu Bugs team in order to be aware of bugs being reported by users.

Working with the QA Trackers

The initial decisions on what and when to test, will be taken following discussion with the members of the Xubuntu Release team.

At the start of a cycle, the QA team needs to ensure that:

  • Image testcases we use are still correct.

  • When the intention is that package testing will take place during the cycle, package testcases required during the cycle are still correct.

  • The testsuites on the Package Tracker make sense for what we intend to test during the cycle. Differences between regular and LTS releases are often, but not always, needed.

  • Scheduling of ISO, and when appropriate Package, testing should take place amongst the QA team.

When there are changes to a package we test, following for example a bug fix, a further check of the testcase involved should take place. Further testing calls for that package should be made to check for regression during the cycle.

When a package during test constantly fails, or bug reports indicate a failure in a package for something not tested. The testcase for that package can be disabled temporarily. The QA Lead is responsible for ensuring tests are both disabled and re-enabled when appropriate.

Xubuntu ISOs are built and added to the tracker at approximately 02:00 UTC. This time can be changed by amending our build times on the ubuntu-cdimage crontab and proposing the change.

Working with Testcases

Information on the basic method of working with testcases can be found at the Ubuntu QA Team Manual Testcases page.

We are only concerned with a specific set of tasks: grabbing the branch, making edits and then pushing the changes we need to the main branch. We have people in the Testcase Admin team for the LP Testcases, in addition any member of our Release Team can edit the tracker, this helps ensure that changes are moved through to the trackers quickly.

To edit a testcase:

  • Report the required change as a bug against the testcase project

  • Assign yourself to the bug

  • Create a local branch: bzr branch lp:ubuntu-manual-tests

  • Make changes to the testcase in your local branch

  • Commit the changes: bzr commit -m "Fix LP bug #BUGNO."

  • Push to a personal remote branch: bzr push lp:~username/ubuntu-manual-tests/bug-BUGNO

Once pushed to a personal remote branch, propose the change for merging - Submitting merge proposals.

Respond to any requests for changes when asked by the Testcase Admins in order to get the required change through in a timely manner.

Communicating with Testers and Users

While we have two sets of people in the community that we contact about required testing, Testers will get regular contact from us, but we should only, in general, call on Users at later stages.

The QA Lead will be an administrator on the Testers Launchpad page and can contact those users via LP. Copies of testing calls sent to the devel mailing list should go to this group each time.

Any member of the QA team can:

  • Mail the devel list with a testing call

  • Just prior to an upcoming ISO Milestone testing call, warn on the devel list

  • Make an ISO Milestone testing call

Xubuntu Users will be contacted for ISO testing at later ISO Milestones, at the earliest the Beta 1 milestone, depending on the state of the current ISO and our packages.

Release Responsibilities

Much of the responsibility for the QA team at any release lies with the QA Lead.

However, any member of the QA team can:

  • Work with the testing wiki Release Note

  • Check status of bugs listed on the above draft

  • Check status of work items on the QA blueprint, marking as appropriate

Post Release Tasks

Following release, there are a few tasks that need to be done before the next release cycle begins.

  • QA Lead should set up the blueprints for both the QA team and the Bugs that the whole team uses

  • Check that the draft Release Note is up to date