Junit tests reference scheme

Tests naming scheme

Junit tests detected in the project have to receive unique - and as far as possible readable - names in the runner system. Such names are used in the following contexts :

  • Listing available tests
  • Requiring the runner to run a subset of tests
  • Linking Squash TM test cases to the junit test covering it
  • Displaying tests results in execution reports

In this context, we define the qualified name of the Junit test as follows :

bundle-name:qualified_class_name/Display Name, where:

  • bundle-name : name of the code bundle in which the tests are defined. The runner defines two such bundles :

    • maven.main.bundle : junit code from maven main sources (by default located in the src/main/java and src/main/resources subdirectories)
    • maven.test.bundle : junit code from maven test sources (by default located in the src/test/java and src/test/resources subdirectories)

  • qualified_class_name : the dot-separated qualified name of the class where the test is defined, as specified by java conventions.

  • Display Name : the display name of the test, as defined by junit. By default this is the name of the method that defines the test, but junit allows you to override this using the @DisplayName annotation.

This test name appears in test lists for the TM-TF link, they may also be used to select tests to run from the command line (see below).

There is a small difference in test execution reports : tests are grouped in test groups named ecosystems. These groups are defined like the prefix we add to the Display Name to make qualified test names : according to the code bundle and test class. Ecosystems are named accordingly, but the separator between the bundle and class parts is a dot instead of a colon :

bundle-name.qualified_class_name

In these reports, tests appear under a qualified test result name where the colon and slash separators are also replaced by a dot :

bundle-name.qualified_class_name.Display Name


Tests with the same @displayName

The use of the @displayName junit annotation allows you to give two tests from the same class the same unqualified display name. And as these tests come from the same class, in the same bundle, this will given them the same qualified display name through the runner.

We consider this bad practice and strongly advise against it, even if you don’t use our runner. Giving two tests the same display name will tend to make reports harder to read: if two or more tests cover exactly the same case, then why use time and work to write more than one test, and read more than one result ? On the other hand, if they test different things you should make yourself a favor and name them differently, to avoid confusion.

If, however, you have inherited such tests, or you have your own reason to go against this advice, the runner will work with this tests in the following way :

  • All tests with the same qualified display name will be seen as a single test, and listed and executed as such.
    • All junit tests will be executed and checked as a global unit which result will be the worst result of any test.
  • You will be able to link this global functional test defined by its common qualified display name to Squash TM test cases as any other automated test, execute the test case and see the result from Squash TM

Known limitation : if several tests share the same qualified display name, the runner will not handle metadata declaration (defined through the use of Squash TF’s @Metadata annotations on the methods) properly. We intend to fix this in a later version.