Fork me on GitHub

HTTP/1.1, HTTP/2 and WebSockets API stub server for stubbing external systems in Docker and non-containerized environment for integration and contract testing

CircleCI build master branch DockerHub GitHubStars GitHubForks codecov Maven Central Stackoverflow stubby4j

stubb4j

It is a highly flexible and configurable tool for testing interactions of service-oriented (SoA) or/and micro-services architectures (REST, SOAP, WSDL, etc.) over HTTP/1.1, HTTP/2 and WebSockets protocols.

Please refer to Key features for more information

Why the word “stubby”?

It is a stub HTTP server after all, hence the “stubby”. Fun fact: in Australian slang “stubby” means beer bottle

User manual for stubby4j v7.5.3-SNAPSHOT

Table of contents

Advantages of using stubby4j HTTP stub server

There are a number of use cases where you’d want to use WebSockets, HTTP/1.1, HTTP/2 stub server in your development/QA environment. If you are a Software Engineer/Test Engneer/QA, then it should hit close to home with you. As an example, some of these use cases are outlined below (this is by no means an exhaustive list). Use stubby4j when you want to:

Back to top

In the Press

Back to top

Key features

Back to top

Minimal system requirements

Running stubby4j as a standalone JAR

To run stubby4j standalone JARs which are hosted on Maven Central, the following are the minimal JRE requirements. Please note, there is no vendor specific requirement, anyone of the Oracle, OpenJDK, AdoptOpenJDK or Azul Zulu OpenJDK, will do.

stubby4j version Version status Minimal JRE version Notes
11.x.x Work in progress 11 Coming soon…
7.x.x Current & recommended 1.8 v7.x.x on Maven Central
6.x.x Legacy 1.8 v6.x.x on Maven Central
5.x.x Legacy 1.8 v5.x.x on Maven Central
4.x.x Legacy 1.8 v4.x.x on Maven Central

Running stubby4j as a pre-built Docker container

See Running in Docker for more information.

Quick start example

This section explains how to get stubby4j up and running using a very simple example Hello, World, without building stubby4j from locally using Gradle.

Setup

That’s it!

For more information and more complex examples, please dive into the rest of documentation, especially docs/HTTP endpoint configuration HOWTO.html

Back to top

Running in Docker

stubby4j Docker images hosted on https://hub.docker.com/r/azagniotov/stubby4j.

Alternatively you can build your own image locally using one of the project’s Dockerfile under:

Navigate to one of the above desired directory and run the following command to build from the master branch, e.g.:

$ docker build --rm --no-cache -t stubby4j:latest .

or the following command to build from a specific tag, e.g.:

$ docker build --build-arg REVISION=v7.5.2 --rm --no-cache -t stubby4j:7.5.2 .

Please note:, in order to build a JDK 21 Docker image, the REVISION must be latest. The JDK 21 is not supported in prior releases of stubby4j

Refer to https://hub.docker.com/r/azagniotov/stubby4j README How to use this image section regarding how to use the built image

Back to top

Docker Compose

stubby4j Docker images hosted on https://hub.docker.com/r/azagniotov/stubby4j.

Alternatively you can add stubby4j image to your stack using Docker Compose:

# This compose file adds stubby4j https://hub.docker.com/r/azagniotov/stubby4j to your stack
#
# See "Environment variables" section at https://hub.docker.com/r/azagniotov/stubby4j
version: '3.5'
services:
  stubby4j-jre11:
    # 'root' - so that stubby4j can write 'logs' into host machine's directory mapped to container volume
    user: root
    image: azagniotov/stubby4j:latest-jre11
    volumes:
      - "./yaml:/home/stubby4j/data"
    container_name: stubby4j_jre11
    ports:
      - 8884:8884
      - 8891:8891
      - 7445:7445
    environment:
      YAML_CONFIG: smoke-tests-stubs.yaml
      LOCATION: 0.0.0.0
      STUBS_PORT: 8884
      ADMIN_PORT: 8891
      STUBS_TLS_PORT: 7445
      # https://stubby4j.com/#command-line-switches
      WITH_ARGS: "--enable_tls_with_alpn_and_http_2 --debug --watch"

… where the <HOST_MACHINE_DIR_WITH_YAML_CONFIG_TO_MAP_VOLUME_TO> is the host machine directory with the stubby4j YAML config file (see the YAML_CONFIG env var under Environment variables) that you want to map to the container volume /home/stubby4j/data

See smoke test https://github.com/azagniotov/stubby4j/tree/master/docker/smoke-test/docker-compose.yml as a working example.

Back to top

Building

stubby4j is a multi source-set Gradle v7.2.0 project

Run ./gradlew command to:

Run ./gradlew clean jacocoTestReport command to:

Back to top

Logging

The stubby4j app emits STDOUT output stream and file-based logs (i.e.: generated by the log4j2 library).

Running stubby4j as a standalone JAR

Only STDOUT output stream logs are available, unless the --mute command line argument was provided upon startup. See Command-line switches

Running stubby4j as an embedded JAR

When a logging framework implementation on the classpath, in addition to the STDOUT output stream logs, stubby4j also emits file-based logs through the SLF4J logging facade.

Running stubby4j in Docker container

Currently, only in the Docker images tagged as latest-jreXX and 7.3.3-jreXX and higher (i.e.: 7.5.2-jreXX), the stubby4j service emits file-based logs (i.e.: generated by log4j2 library) as well as STDOUT output stream logs. In images tagged as <= 7.3.2-jreXX, only STDOUT output stream logs are available.

Please refer to https://hub.docker.com/r/azagniotov/stubby4j Container application logs section for more information

Back to top

Third-party dependencies

Back to top

Adding stubby4j to your project

The following are the stubby4j artifacts that are hosted on Maven Central:

Gradle

api("io.github.azagniotov:stubby4j:7.5.2")

additionally, by adding a classifier to the JAR name like no-dependencies or no-jetty, i.e.:

api("io.github.azagniotov:stubby4j:7.5.2:no-jetty")

Maven

<dependency>
    <groupId>io.github.azagniotov</groupId>
    <artifactId>stubby4j</artifactId>
    <version>7.5.2</version>
</dependency>

additionally, by adding a classifier to the JAR name like no-dependencies or no-jetty, i.e.:

<dependency>
    <groupId>io.github.azagniotov</groupId>
    <artifactId>stubby4j</artifactId>
    <version>7.5.2</version>
    <classifier>no-dependencies</classifier>
</dependency>

Adding stubby4j SNAPSHOT versions to your project

stubby4j SNAPSHOT version contains the latest changes from the master branch. A snapshot version is built from every commit to the master branch and published to the OSS Sonatype snapshots repository.

In order to pull down a SNAPSHOT version from Maven Central, please make sure that Sonatype’s snapshot repository is configured in your project project build tool, e.g.: Gradle configuration:

repositories {
    maven {
        url 'https://oss.sonatype.org/content/repositories/snapshots/'
    }
}

Now you can include stubby4j SNAPSHOT artifacts in your project:

api("io.github.azagniotov:stubby4j:7.5.3-SNAPSHOT")

additionally, by adding a classifier to the JAR name like no-dependenciess or no-jetty, i.e.:

api("io.github.azagniotov:stubby4j:7.5.3-SNAPSHOT:no-jetty")

Installing stubby4j to local .m2 repository

Run ./gradlew clean build publishToMavenLocal command to:

Now you can include locally installed stubby4j SNAPSHOT artifacts in your project:

api("io.github.azagniotov:stubby4j:7.5.3-SNAPSHOT")

additionally, by adding a classifier to the JAR name like no-dependenciess or no-jetty, i.e.:

api("io.github.azagniotov:stubby4j:7.5.3-SNAPSHOT:no-jetty")

Back to top

Command-line switches

usage:
java -jar stubby4j-x.x.xx.jar [-a <arg>] [-d <arg>] [-da] [-dc] [-ds] [-h]
       [-k <arg>] [-l <arg>] [-m] [-o] [-p <arg>] [-s <arg>] [-t <arg>]
       [-ta] [-v] [-w <arg>]
 -a,--admin <arg>                        Port for admin portal. Defaults
                                         to 8889.
 -d,--data <arg>                         Data file to pre-load endpoints.
                                         Data file to pre-load endpoints.
                                         Optional valid YAML 1.1 is
                                         expected. If YAML is not
                                         provided, you will be expected to
                                         configure stubs via the stubby4j
                                         HTTP POST API.
 -da,--disable_admin_portal              Does not start Admin portal
 -dc,--disable_stub_caching              Since v7.2.0. Disables stubs
                                         in-memory caching when stubs are
                                         successfully matched to the
                                         incoming HTTP requests
 -ds,--disable_ssl                       Disables TLS support (enabled by
                                         default) and disables the
                                         '--enable_tls_with_alpn_and_http_
                                         2' flag, if the latter was
                                         provided
 -h,--help                               This help text.
 -k,--keystore <arg>                     Keystore file for custom TLS. By
                                         default TLS is enabled using
                                         internal self-signed certificate.
 -l,--location <arg>                     Hostname at which to bind stubby.
 -m,--mute                               Mute console output.
 -o,--debug                              Dumps raw HTTP request to the
                                         console (if console is not
                                         muted!).
 -p,--password <arg>                     Password for the provided
                                         keystore file.
 -s,--stubs <arg>                        Port for stub portal. Defaults to
                                         8882.
 -t,--tls <arg>                          Port for TLS connection. Defaults
                                         to 7443.
 -ta,--enable_tls_with_alpn_and_http_2   Since v7.4.0. Enables HTTP/2 over
                                         TCP (h2c) and HTTP/2 over TLS
                                         (h2) on TLS v1.2 or newer using
                                         ALPN extension
 -v,--version                            Prints out to console stubby
                                         version.
 -w,--watch <arg>                        Since v2.0.11. Periodically scans
                                         for changes in last modification
                                         date of the main YAML and
                                         referenced external files (if
                                         any). The flag can accept an
                                         optional arg value which is the
                                         watch scan time in milliseconds.
                                         If milliseconds is not provided,
                                         the watch scans every 100ms. If
                                         last modification date changed
                                         since the last scan period, the
                                         stub configuration is reloaded

Back to top

Making requests over TLS

When this section was written, as of November 2021, there are still enough legacy applications out there that have not (or not able to) upgraded to the more secure, recommended and industry-standard TLS protocol versions v1.2 and/or its successor v1.3. Therefore, in order to acommodate a range of integration testing needs, stubby4j continues to support the legacy versions of TLS protocol.

Furthermore, the reader should be aware of that as part of continuous improvement of Java security, the industry continues discourage use of aforementioned protocols and in March 2021, the RFC8996 deprecating TLS 1.0 (introduced in 1999) and TLS 1.1 (introduced in 2006) was approved.

Supported protocol versions

stubby4j can accept requests over most available versions of the SSL (Secure Sockets Layer) and its successor TLS (Transport Layer Security) protocols. Supported versions are the legacy SSLv3, TLSv1.0 and TLSv1.1, as well as the current TLSv1.2 and TLSv1.3 (the TLS 1.3 standard was released in August 2018 and is a successor to TLS 1.2).

TLS v1.3 support

When running stubby4j as a standalone JAR, if the underlying JDK version supports TLSv1.3, then this protocol version will also be supported and enabled in stubby4j. When stubby4j is run from one of the pre-built Docker images, the TLSv1.3 is supported by default.

Please note, if you are running on JDK 1.8, it does not mean that your JDK build version & vendor necessarily support TLSv1.3. For example:

Server-side TLS configuration

During TLS configuration in stubby4j, the following happens:

  1. The property jdk.tls.disabledAlgorithms (located in java.security configuration file) is modified at runtime where the following values SSLv3, TLSv1 and TLSv1.1 are removed, in order to workaround the JDK-8254713: Disable TLS 1.0 and 1.1

  2. The TLS in stubby4j is enabled by default using an internal, multi-hostname/IP self-signed certificate in PKCS12 format imported into the server’s key-store. See OpenSSL config file used for the certificate generation, i.e.,: stubby4j is behaving as its own certificate authority.

    The default self-signed certificate can be overridden by supplying your own keystore/certificate (e.g.: generated from your own certificate signed by a certificate authority) when configuring stubby4j command-line arguments. In other words, this allows you to load top-level certificates from a root certificate authority. When providing a keystore file to stubby4j, the keystore should have .PKCS12 or .JKS file extension. See command-line switches for more information.

Client-side TLS configuration

Since stubby4j’s TLS layer configured (by default) using a self-signed certificate, it is not going to be possible for web clients to validate stubby4j’s default self-signed certificate against clients’ own trust-store containing a list of trusted Certificate Authority (CA) certificates.

When TLS/SSL handshake happens, clients and servers exchange SSL certificates, cipher suite requirements, and randomly generated data for creating session keys. As part of its “hello” reply to the client’s “hello” message, the server sends a message containing the server’s SSL certificate (among other things like cipher suite and random string of bytes).

In other words, somehow, a web client making a request to stubby4j server over TLS has to ensure that it can trust stubby4j’s self-signed certificate. There are a number of options available for web clients to achieve the trust between the two parties during TLS/SSL handshake:

  1. Configuring web client’s X.509 trust strategy/manager to trust all certificates

    This is analogous to supplying the -k (or --insecure) option to cURL, which turns off cURL’s verification of the server’s certificate, e.g.:

    shell script $ curl -X GET --tls-max 1.0 https://localhost:7443/hello -v -k

    When a web client configures its own SSLSocketFactory (or SSLContext), the client can also configure its own X.509 certificate trust strategy/manager. This trust strategy/manager must be a trust all (or a strategy/manager that trusts self-signed certificates). In this case, even when server responds with a self-signed certificate, the server’s identity will be verified as valid. Please note, trusting any certificate is very insecure and should not be used in production environments.

  2. Providing stubby4j self-signed certificate to the web client before making requests over TLS

    This is analogous to supplying the --cacert option to cURL, which tells cURL to use the specified certificate file to verify the peer, e.g.:

    shell script $ curl -X GET --tls-max 1.0 https://localhost:7443/hello -v \ --cacert src/main/resources/ssl/openssl.downloaded.stubby4j.self.signed.v3.pem

    If you do not want to configure a trust all X.509 manager/strategy for your web client, as an alternative it is possible to ensure that your web client already has stubby4j’s default self-signed certificate before making requests. In order to make web client to be aware of the self-signed certificate, you need to download and save the certificate from the running stubby4j server and then load it to the trust-store of your client when building SSLSocketFactory (or SSLContext).

    Please see the following code of the HttpClientUtils in functional tests for the openssl, keytool commands & Java code examples.

    If you use a non-Java web client, you can use an already downloaded (via the openssl s_client command) stubby4j self-signed certificate in PEM format to load into your web client trust store.

    If your web client is a Java-based app, then you can load the aforementioned PEM certificate which was already converted in to two JKS and PKCS12 formats. You can use JKS or PKCS12 certificate to load into your Java web client trust store.

    Alternatively, you can check all the certificates on the GitHub https://github.com/azagniotov/stubby4j/tree/master/src/main/resources/ssl

    Server hostname verification by the client

    During an SSL handshake, hostname verification establishes that the hostname in the URL matches the hostname in the server’s identification; this verification is necessary to prevent man-in-the-middle attacks.

    If (1) you imported stubby4j self-signed certificate into your web client trust store as per above and (2) stubby4j app is running on one of the following IPs or a localhost (alternatively, you can check the SSL conf on the GitHub https://github.com/azagniotov/stubby4j/tree/master/src/main/resources/ssl), then your web client will be able to successfully verify URL hostname of the request against the imported stubby4j certificate’s SAN (subject alternative names) list.

    If you are running the stubby4j app on some other hostname/IP, then the hostname verification by your client will fail because the imported stubby4j self-signed certificate does not contain that hostname/IP. There are a number of options available for web clients to workaround the the hostname verification:

    1. Skip the hostname verification check or relax it (please note, skipping the hostname verification check is very insecure and should not be used in production environments)
    2. Make a pull request or raise an issue with a request asking me to add the hostname/IP into the SAN list of the stubby4j self-signed certificate ;)

If you have any questions about the TLS configuration in stubby4j, please feel free to raise an issue.

Back to top

Support for HTTP/2 on HTTPS URIs over TLS

Support for HTTP/2 on HTTPS URIs can be enabled by providing --enable_tls_with_alpn_and_http_2 flag to the stubby4j JAR. See command-line switches for more information. The HTTP/2 support will be enabled only for the the TLS layer in stubby4j. In other words, web clients will not be able to make HTTP/2 requests to http://.... URLs.

When the aforementioned flag is provided, the TLS in stubby4j is enabled with the ALPN (RFC 7301) extension. ALPN is the TLS extension that HTTP/2 is expected to use and the ALPN helps to negotiate the HTTP/2 between client & server without losing valuable time or network packet round-trips.

Please note the following restrictions when enabling HTTP/2 via the aforementioned flag:

  1. As per HTTP/2 RFC, the HTTP/2 over TLS in stubby4j will be enabled only for TLSv1.2 or higher.
  2. Web clients making HTTP/2 requests over TLS to stubby4j should be using ALPN TLS extension in their configuration to negotiate HTTP/2.
  3. In stubby4j, the HTTP/2 over TLS will be enabled for JDK 1.8 (versions from 1.8.0_252 included and later) and higher.

Back to top

WebSockets configuration HOWTO

See docs/websockets_configuration_howto.html for details

HTTP endpoint configuration HOWTO

See docs/http_endpoint_configuration_howto.html for details

Performance optimization index

stubby4j uses a number of techniques to optimize evaluation of stubs

Regex pattern pre-compilation

During parsing of stubs config, the request.url, request.query, request.headers & request.post (or request.file) values are checked for presence of regex. If one of the aforementioned properties is a stubbed regex, then a regex pattern will be compiled & cached in memory. This way, the pattern(s) are compiled during config parsing, not stub evaluation.

Back to top

The admin portal

See docs/admin_portal.html for details

Back to top

The stubs portal

Requests sent to any url at localhost:8882 (or wherever you told stubby to run) will search through the available endpoints and, if a match is found, respond with that endpoint’s response data

How endpoints are matched

For a given endpoint, stubby only cares about matching the properties of the request that have been defined in the YAML. The exception to this rule is method; if it is omitted it is defaulted to GET.

For instance, the following will match any POST request to the root url:

-  request:
      url: /
      method: POST
   response: {}

The request could have any headers and any post body it wants. It will match the above.

Pseudocode (StubRepository#matchStub):

    if (<incoming request>.hashCode found in <local cache>) {
        get <cached stubbed endpoint> from <local cache> by <incoming request>.hashCode
        return <cached stubbed endpoint>
    } else {
        for each <stubbed endpoint> of stored endpoints {
            for each <property> of <stubbed endpoint> {
                if (<stubbed endpoint>.<property> != <incoming request>.<property>) {
                    next stubbed endpoint
                }
            }
            store in <local cache> the found <stubbed endpoint> by hashCode

            return <stubbed endpoint>
        }
    }

Back to top

Client programmatic API

When running stubby4j as an embedded JAR, you can start-up and manage stubby4j server programmatically with the help of StubbyClient.

In addition, it is possible to manage embedded stubby4j using the StubbyClient without pointing it to an actual local physical YAML config file. Instead, consumers can provide stubs YAML configuration payload of type String, when calling the StubbyClient#startJettyYamless(..). The following StubbyClient functional test demonstrates the API usage.

StubbyClient also enables consumers to manage stubby4j stubs via the Admin portal. Check the https://stubby4j.com/docs/admin_portal.html#client-programmatic-api for more information.

Back to top

Change log

See https://github.com/azagniotov/stubby4j/blob/master/CHANGELOG.md

Authors

See docs/authors.html

Contributors

See docs/contributors.html

See also

See docs/copyright.html for details

License

MIT. See https://github.com/azagniotov/stubby4j/blob/master/LICENSE for details.

Back to top