Test Process

This page documents the steps that a CAS developer/contributor should take for testing a CAS server deployment during development. For additional instructions and guidance on the general build process, please see this page.

Test Cases

The following types of test cases in the project are those that CAS developers/contributors need to review,

Unit Tests

Unit tests are composed of small pieces of work or functionality that generally can be tested in isolation. They are created as Java test classes, and individual test scenarios are typically annotated with the @Test annotation and then executed by the test framework.

For example, a src/main/java/Extractor.java type of component would have its test cases inside a src/test/java/ExtractorTests.java test class.

Integration Tests

In some scenarios unit tests also run a series of tests against databases, external systems or APIs to verify functionality and correctness of integration. For example, a MongoDbTicketRegistry type of component would require a MongoDb running instance and special markup and annotations to run tests when that external instance is up and running. Structurally speaking, such tests are almost identical to plain vanilla unit tests and may only contain additional decorations and annotations,depending on the test system.

Functional Tests

Functional tests are categories of tests that verify combinations of scenarios and behaviors from the perspective of an end-user. Such tests typically are composed of an execution script and scenario, and may involve a headless browser to run through a scenario, verify data elements on the screen, etc. This category of tests is usually very effective at reproducing scenarios related to an issue or possible defect, and can be very helpful in troubleshooting and issue diagnosis.

Testing Modules

To test the functionality provided by a given CAS module, execute the following steps:

• For the tomcat, undertow or jetty webapp, add the module reference to the webapp.gradle build script of web application you intend to run:

1
implementation project(":support:cas-server-support-modulename")

Alternatively, pass the required modules automatically:

1
bc ldap,x509

…where bc is an alias for building CAS.

Prepare the embedded container, to run and deploy the web application.

Unit / Integration Testing

To simplify the test execution process, you may take advantage of the testcas.sh script found at the root of the repository as such:

1
2
# chmod +x ./testcas.sh
./testcas.sh --category <category> [--test <test-class>] [--debug] [--with-coverage]

To learn more about the script, use:

1
./testcas.sh --help

All unit and integration tests are executed by the continuous integration system.

Code Coverage & Metrics

Code coverage metrics are collected and reported by the following platforms:

System	Badge
Codacy
SonarCloud
CodeCov

Quality metrics are collected and reported by the following platforms:

System	Badge
Codacy
SonarCloud Quality Gate
SonarCloud Maintainability

Browser & Functional Testing

Automated browser testing is done via the Puppeteer framework. Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol and runs headless by default.

Functional tests start by generating a plain CAS overlay as a baseline that is able to run under HTTPS using a pre-generated keystore. This overlay is supplied the test scenario configuration that explains the required modules, properties, etc to use when CAS is deployed inside an embedded Apache Tomcat container. Once running, the Puppeteer script is executed by Node for the given test scenario to verify specific functionality such as successful logins, generation of tickets, etc.

All functional and browser tests are executed by the continuous integration system.

To install Puppeteer once:

1
npm i -g puppeteer

To help simplify the testing process, you may use the following bash function in your ~/.profile:

1
2
3
4
5
6
function pupcas() {
  cd /path/to/cas
  scenario=$1
  shift 1
  ./ci/tests/puppeteer/run.sh --scenario ./ci/tests/puppeteer/scenarios/"$scenario" $@
}

…which can later be invoked as:

1
pupcas <scenario-name>

To see the list of available test scenarios:

1
./gradlew --build-cache --configure-on-demand --no-daemon -q puppeteerScenarios

Remote debugging may be available on port 5000. To successfully run tests, you need to make sure jq is installed.

The following command-line options are supported for test execution:

For example, the login-success test scenario may be run using:

1
pupcas login-success --hbo

All other build related options and flags (i.e. --info) may be passed directly to the script.

Test Scenario Anatomy

Each test scenario is composed of the following files:

• script.js: The main driver of the test that executes the test, via launching a headless browser via Puppeteer. The basic outline of the test script may be:

1
2
3
4
5
6
7
8
9
10
11
const puppeteer = require('puppeteer');
const cas = require('../../cas.js');

(async () => {
    const browser = await puppeteer.launch(cas.browserOptions());
    const page = await cas.newPage(browser);
    
    // Do stuff and check/assert behavior...
    
    await browser.close();
})();

• script.json: The optional test configuration in JSON that includes necessary CAS modules, properties, and other specifics.

A basic modest outline of the test configuration may be:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
{
  "dependencies": "module1,module2,module3,...",
  "conditions": {
    "docker": "true"
  },
  "properties": [
    "--cas.server.something=something"
  ],
  "jvmArgs": "...",
  "SPRING_APPLICATION_JSON": {},
  "readyScript": "${PWD}/ci/tests/puppeteer/scenarios/${SCENARIO}/ready.sh",
  "initScript": "${PWD}/ci/tests/puppeteer/scenarios/${SCENARIO}/init.sh",
  "exitScript": "${PWD}/ci/tests/puppeteer/scenarios/${SCENARIO}/exit.sh",
  "instances": 2,
  "instance1": {
    "properties": [ ]
  },
  "instance2": {
    "properties": [ ]
  }
}

The only required bit in the test JSON configuration might be the dependencies attribute.

MacOS Firewall Popup

To allow the firewall to accept incoming network connections for Chromium on MacOS, you may apply the following command:

1
2
chromium="/path/to/cas/ci/tests/puppeteer/node_modules/puppeteer/.local-chromium"
sudo codesign --force --deep --sign - "${chromium}/mac-*/chrome-mac/Chromium.app"

Continuous Integration

Unit and integration tests are automatically executed by the CAS CI system, GitHub Actions.