With the production code complete, let’s look at adding some tests.

Unit tests are useful to a point, but ideally you should also test functionality of the complete service. Creek system-tests perform black-box testing of your service or services. It executes test suites against your complete services, and any required third-party services, running in Docker containers.

The Docker images being run are the same images you’ll deploy into your environments. By testing the actual Docker image, you can be confident the service does what is intended, always assuming that your test coverage is sufficient.

Write a system test

System tests are written in YAML files, and executed as part of the Gradle build using the system test plugin. The repo already has a system-tests module, with the plugin applied. Follow these steps to add a test suite to the module:

ProTip: The systemm-test plugin can be applied to individual service modules, allowing for targeted testing of a single service, or more normally applied to a system-tests module, where the functionality of the aggregate as a whole can be tested.

Define test inputs

Start by defining the input to send to the service, i.e. the records to produce to the twitter.tweet.text Kafka topic. These will be produced to the topic after the service has started up.

ProTip: Input data can also be seeded into the test environment before services are started, by placing the input file in the seed directory, rather than the inputs directory.

Create a file at system-tests/src/system-test/example-suite/inputs/twitter.tweet.text.yml with the following content:

---
!creek/kafka-topic@1
topic: twitter.tweet.text
records:
  - key: 1622262145390972929
    value: "@ScottAdamsSays Documentaries are dangerous in this way. That said, MJ clearly had major issues. No documentary needed for that."
  - key: 1622082025166442505
    value: "@PepitoTheCat @BillyM2k @ericnakagawa @PepitoTheCat Responding to feedback, Twitter will enable a light, write-only API for bots providing good content that is free."
  - key: 1622080465418043392
    value: "@teslaownersSV Production will have red light bar"
  - key: 1622000758617939969
    value: "@teslaownersSV V11 has been tougher than expected, as  it is a significant rearchitecture of NNs, plus many more NNs replacing C++. Hoping to ship v11.3 end of week."
  - key: 1621998713806721025
    value: https://t.co/gbkqVT3MXq

ProTip: The name of the file does not need to match the name of the topic, it can be anything. Name the file to make the test cases easy to understand.

The !creek/kafka-topic@1 at the top of the file tells the system test parser how to parse this file. This particular input type is registered by the Creek Kafka test extension. The number after the @ symbol is a version number, allowing the type to evolve without breaking existing tests.

The records property defines the list of records the system tests will produce to Kafka, with each record’s key and value defined.

ProTip: You can define records with null keys and values implicitly by excluding the key and/or value property, or explicitly by setting the key and/or value property to ~.

Define expected outputs

Given the input above, define the expected output, i.e. the records we expect the service to produce to the twitter.handle.usage topic.

Create a file at system-tests/src/system-test/example-suite/expectations/twitter.handle.usage.yml with the following content:

---
!creek/kafka-topic@1
topic: twitter.handle.usage
records:
  - key: "@ScottAdamsSays"
    value: 1
  - key: "@PepitoTheCat"
    value: 2
  - key: "@BillyM2k"
    value: 1
  - key: "@ericnakagawa"
    value: 1
  - key: "@teslaownersSV"
    value: 1
  - key: "@teslaownersSV"
    value: 1

This file follows a similar format to the input file, defining the type the file should be parsed as, and the exact records to expect.

ProTip: You can define expectations of records with null keys and values by setting the key and/or value property to ~. Unlike for inputs and seeds, not defining the key or value property in an expectation file means the property is ignored. Any value is accepted as valid, though it will still need to deserialise correctly.

Add a test suite file

With the test input and expected output defined, it’s time to define the test suite.

Create a file at system-tests/src/system-test/example-suite/suite.yml with the following content:

---
name: basic suite
services:
  - handle-occurrence-service
tests:
  - name: test 1
    inputs:
      - twitter.tweet.text
    expectations:
      - twitter.handle.usage

This defines a very basic test suite, which starts our handle-occurrence-service and executes a single test case. That test case feeds in the records in the twitter.tweet.text YAML file, and expects the output in the twitter.handle.usage YAML file.

Protip: The .yml extension of files listed under a test’s inputs and outputs property is optional.

Protip: A test suite can define options to customise the test suite. For example, the Kafka test extension defines a creek/kafka-options type, that can be used to control required message handling and more.

Running the system tests

The system tests can be executed with the following Gradle command:

./gradlew systemTest

The system tests will start up a Kafka broker, and the handle-occurrence-service, in Docker containers. Once running, it will produce the records defined in the twitter.tweet.text.yml to Kafka and listen for the expected output defined in the twitter.handle.usage.yml file.

Note: All being well the tests will pass! Why not try changing the expected output and re-running to see what happens when tests fail. The system tests output a lot of information about failures.

How do the system tests work?

We believe that being able to test the business functionality of a service, or services, this quickly and easily is both pretty cool and a big part of what drove us to develop Creek, but how does it work?

Very briefly, the system tests work by discovering the handle-occurrence-service’s service descriptor on the class path. The system tests inspect the service descriptor.

As the descriptor defines Kafka based resources, the system tests, with the help of the installed Creek Kafka system-test extension, knows to start a Kafka broker and create any unowned topics, like the twitter.tweet.text topic.

The service descriptor also defines the name of the service’s Docker container, allowing the system tests to start the service.

Finally, the Kafka topic descriptors exposed by the service descriptor provide the information the system tests, and its extensions, need to be able to serialize inputs and deserialize outputs.

More information about the system tests can be found here.

Updated: