Sponsor the Team View on GitHub

jqwik Latest Release: 1.9.2

Property-Based Testing in Java

The jqwik User Guide 1.3.7

Table of Contents

How to Use

jqwik is an alternative test engine for the JUnit 5 platform. That means that you can use it either stand-alone or combine it with any other JUnit 5 engine, e.g. Jupiter (the standard engine) or Vintage (aka JUnit 4). All you have to do is add all needed engines to your testCompile dependencies as shown in the gradle file below.

The latest release of jqwik is deployed to Maven Central.

Snapshot releases can be fetched from https://oss.sonatype.org/content/repositories/snapshots.

Required Version of JUnit Platform

The minimum required version of the JUnit platform is 1.7.0.

Gradle

Since version 4.6, Gradle has built-in support for the JUnit platform. Set up is rather simple; here are the relevant parts of a project’s build.gradle file:

repositories {
    ...
    mavenCentral()

    # For snapshot releases only:
    maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }

}

ext.junitPlatformVersion = '1.7.0'
ext.junitJupiterVersion = '5.7.0'

ext.jqwikVersion = '1.3.7'

compileTestJava {
    // To enable argument names in reporting and debugging
	options.compilerArgs += '-parameters'
}

test {
	useJUnitPlatform {
		includeEngines 'jqwik'
        
        // Or include several Junit engines if you use them
        // includeEngines 'jqwik', 'junit-jupiter', 'junit-vintage'

		// includeTags 'fast', 'medium'
		// excludeTags 'slow'
	}

	include '**/*Properties.class'
	include '**/*Test.class'
	include '**/*Tests.class'
}

dependencies {
    ...

    // aggregate jqwik dependency
    testImplementation "net.jqwik:jqwik:${jqwikVersion}"

    // Add if you also want to use the Jupiter engine or Assertions from it
    testImplementation "org.junit.jupiter:junit-jupiter:5.7.0"

    // Add any other test library you need...
    testImplementation "org.assertj:assertj-core:3.12.2"

}

With version 1.0.0 net.jqwik:jqwik has become an aggregating module to simplify jqwik integration for standard users. If you want to be more explicit about the real dependencies you can replace this dependency with

    testImplementation "net.jqwik:jqwik-api:${jqwikVersion}"
    testRuntime "net.jqwik:jqwik-engine:${jqwikVersion}"

In jqwik’s samples repository you can find a rather minimal starter example for jqwik with Gradle.

See the Gradle section in JUnit 5’s user guide for more details on how to configure Gradle for the JUnit 5 platform. There is also a comprehensive list of options for Gradle’s test task.

Seeing jqwik Reporting in Gradle Output

Since Gradle does not yet support JUnit platform reporting (see this Github issue) jqwik has switched to do its own reporting by default. This behaviour can be configured through parameter useJunitPlatformReporter (default: false).

If you want to see jqwik’s reports in the output use Gradle’s command line option --info:

> gradle clean test --info
...
mypackage.MyClassProperties > myPropertyMethod STANDARD_OUT
    timestamp = 2019-02-28T18:01:14.302, MyClassProperties:myPropertyMethod = 
                                  |-----------------------jqwik-----------------------
    tries = 1000                  | # of calls to property
    checks = 1000                 | # of not rejected calls
    generation = RANDOMIZED       | parameters are randomly generated
    after-failure = PREVIOUS_SEED | use the previous seed
    edge-cases#mode = MIXIN       | edge cases are generated first
    edge-cases#total = 0          | # of all combined edge cases
    edge-cases#tried = 0          | # of edge cases tried in current run
    seed = 1685744359484719817    | random seed to reproduce generated values

Maven

Starting with version 2.22.0, Maven Surefire and Maven Failsafe provide native support for executing tests on the JUnit Platform and thus for running jqwik properties. The configuration of Maven Surefire is described in the Maven section of JUnit 5’s user guide.

Additionally you have to add the following dependency to your pom.xml file:

<dependencies>
    ...
    <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik</artifactId>
        <version>1.3.7</version>
        <scope>test</scope>
    </dependency>
</dependencies>

In jqwik’s samples repository you can find a rather minimal starter example for jqwik with Maven.

Snapshot Releases

Snapshot releases are available through Sonatype’s snapshot repositories.

Adding

https://oss.sonatype.org/content/repositories/snapshots

as a maven repository will allow you to use jqwik’s snapshot release which contains all the latest features.

Project without Build Tool

I’ve never tried it but using jqwik without gradle or some other tool to manage dependencies should also work. You will have to add at least the following jars to your classpath:

Creating a Property

Properties are the core concept of property-based testing.

You create a Property by annotating a public, protected or package-scoped method with @Property. In contrast to examples a property method is supposed to have one or more parameters, all of which must be annotated with @ForAll.

At test runtime the exact parameter values of the property method will be filled in by jqwik.

Just like an example test a property method has to

If not specified differently, jqwik will run 1000 tries, i.e. a 1000 different sets of parameter values and execute the property method with each of those parameter sets. The first failed execution will stop value generation and be reported as failure - usually followed by an attempt to shrink the falsified parameter set.

Here are two properties whose failures might surprise you:

import net.jqwik.api.*;
import org.assertj.core.api.*;

class PropertyBasedTests {

	@Property
	boolean absoluteValueOfAllNumbersIsPositive(@ForAll int anInteger) {
		return Math.abs(anInteger) >= 0;
	}

	@Property
	void lengthOfConcatenatedStringIsGreaterThanLengthOfEach(
		@ForAll String string1, @ForAll String string2
	) {
		String conc = string1 + string2;
		Assertions.assertThat(conc.length()).isGreaterThan(string1.length());
		Assertions.assertThat(conc.length()).isGreaterThan(string2.length());
	}
}

Mind that only parameters that are annotated with ‘@ForAll’ are considered for value generation. Other kinds of parameters can be injected through the resolve parameter hook.

Failure Reporting

If a property fails then jqwik’s reporting is more thorough:

In the case of lengthOfConcatenatedStringIsGreaterThanLengthOfEach from above the report looks like that:

PropertyBasedTests:lengthOfConcatenatedStringIsGreaterThanLengthOfEach = 
  java.lang.AssertionError: 
    Expecting:
     <0>
    to be greater than:
     <0> 
                              |-----------------------jqwik-----------------------
tries = 16                    | # of calls to property
checks = 16                   | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = SAMPLE_FIRST  | try previously failed sample, then previous seed
edge-cases#mode = MIXIN       | edge cases are mixed in
edge-cases#total = 4          | # of all combined edge cases
edge-cases#tried = 0          | # of edge cases tried in current run
seed = -2370223836245802816   | random seed to reproduce generated values

Shrunk Sample (<n> steps)
-------------------------
  string1: ""
  string2: ""

Original Sample
---------------
  string1: "乮��깼뷼檹瀶�������የ뷯����ঘ꼝���焗봢牠"
  string2: ""

  Original Error
  --------------
  java.lang.AssertionError: 
    Expecting:
     <29>
    to be greater than:
     <29>

The source code names of property method parameters can only be reported when compiler argument -parameters is used. jqwik goes for structured reporting with collections, arrays and maps. If you want to provide nice reporting for your own domain classes you can either

Additional Reporting Options

You can switch on additional reporting aspects by adding a @Report(Reporting[]) annotation to a property method.

The following reporting aspects are available:

Unlike sample reporting these reports will show the freshly generated parameters, i.e. potential changes to mutable objects during property execution cannot be seen here.

Optional @Property Attributes

The @Property annotation has a few optional values:

The effective values for tries, seed, after-failure mode, generation mode edge-cases mode and edge cases numbers are reported after each run property:

tries = 10 
checks = 10 
generation = EXHAUSTIVE
after-failure = PREVIOUS_SEED
edge-cases#mode = MIXIN 
edge-cases#total = 2 
edge-cases#tried = 2 
seed = 42859154278924201

Setting Defaults for @Property Attributes

If you want to set the defaults for all property methods in a container class (and all the groups in it) you can use annotation @PropertyDefaults.

In the following example both properties are tried 10 times. Shrinking mode is set for all but is overridden in the second property.

@PropertyDefaults(tries = 10, shrinking = ShrinkingMode.FULL)
class PropertyDefaultsExamples {

	@Property
	void aLongRunningProperty(@ForAll String aString) {}

	@Property(shrinking = ShrinkingMode.OFF)
	void anotherLongRunningProperty(@ForAll String aString) {}
}

Thus, the order in which a property method’s attributes are determined is:

  1. Use jqwik’s built-in defaults
  2. which can be overridden in the configuration file
  3. which can be changed in a container class’ @PropertyDefaults annotation
  4. which can be overridden by a method’s @Property annotation attributes.

Creating an Example-based Test

jqwik also supports example-based testing. In order to write an example test annotate a public, protected or package-scoped method with @Example. Example-based tests work just like plain JUnit-style test cases.

A test case method must

Here is a test class with two example-based tests:

import static org.assertj.core.api.Assertions.*;

import net.jqwik.api.*;
import org.assertj.core.data.*;

class ExampleBasedTests {
	
	@Example
	void squareRootOf16is4() { 
		assertThat(Math.sqrt(16)).isCloseTo(4.0, Offset.offset(0.01));
	}

	@Example
	boolean add1plu3is4() {
		return (1 + 3) == 4;
	}
}

Internally jqwik treats examples as properties with the number of tries hardcoded to 1. Thus, everything that works for property methods also works for example methods – including random generation of parameters annotated with @ForAll.

Assertions

jqwik does not come with any assertions, so you have to use one of the third-party assertion libraries, e.g. Hamcrest or AssertJ.

If you have Jupiter in your test dependencies anyway, you can also use the static methods in org.junit.jupiter.api.Assertions.

Lifecycle

To understand the lifecycle it is important to know that the tree of test elements consists of two main types of elements:

So a typical tree might look like:

Jqwik Engine
    class MyFooTests
        @Property fooProperty1()
        @Property fooProperty2()
        @Example fooExample()
    class MyBarTests
        @Property barProperty()
        @Group class Group1 
            @Property group1Property()
        @Group class Group2 
            @Example group2Example()

Mind that packages do not show up as in-between containers!

When running your whole test suite there are additional things happening:

jqwik gives you more than one way to hook into the lifecycle of containers, properties and tries.

Simple Property Lifecycle

If you need nothing but some initialization and cleanup of the container instance per property or example:

import net.jqwik.api.*;

class SimpleLifecycleTests implements AutoCloseable {

	SimpleLifecycleTests() {
		System.out.println("Before each");
	}

	@Example void anExample() {
		System.out.println("anExample");
	}

	@Property(tries = 5)
	void aProperty(@ForAll String aString) {
		System.out.println("aProperty: " + aString);
	}

	@Override
	public void close() throws Exception {
		System.out.println("After each");
	}
}

In this example both the constructor and close() will be called twice times: Once for anExample() and once for aProperty(...). However, all five calls to aProperty(..) will share the same instance of SimpleLifecycleTests.

Annotated Lifecycle Methods

The other way to influence all elements of a test run is through annotated lifecycle methods, which you might already know from JUnit 4 and 5. jqwik currently has eight annotations:

Given the following container class:

class FullLifecycleExamples {

	@BeforeContainer
	static void beforeContainer() {
		System.out.println("before container");
	}

	@AfterContainer
	static void afterContainer() {
		System.out.println("after container");
	}

	@BeforeProperty
	void beforeProperty() {
		System.out.println("before property");
	}

	@AfterProperty
	void afterProperty() {
		System.out.println("after property");
	}

	@BeforeTry
	void beforeTry() {
		System.out.println("before try");
	}

	@AfterTry
	void afterTry() {
		System.out.println("after try");
	}

	@Property(tries = 3)
	void property(@ForAll @IntRange(min = -5, max = 5) int anInt) {
		System.out.println("property: " + anInt);
	}
}

Running this test container should produce something like the following output (maybe with your test report in-between):

before container

before property
before try
property: 3
after try
before try
property: 1
after try
before try
property: 4
after try
after property

after container

All those lifecycle methods are being run through jqwik’s mechanism for writing lifecycle hooks under the hood.

Single Property Lifecycle

All lifecycle methods described in the previous section apply to all property methods of a container class. In rare cases, however, you may feel the need to hook into the lifecycle of a single property, for example when you expect a property to fail.

Here is one example that checks that a property will fail with an AssertionError and succeed in that case:

@Property
@PerProperty(SucceedIfThrowsAssertionError.class)
void expectToFail(@ForAll int aNumber) {
    Assertions.assertThat(aNumber).isNotEqualTo(1);
}

private class SucceedIfThrowsAssertionError implements PerProperty.Lifecycle {
    @Override
    public PropertyExecutionResult onFailure(PropertyExecutionResult propertyExecutionResult) {
        if (propertyExecutionResult.throwable().isPresent() &&
                propertyExecutionResult.throwable().get() instanceof AssertionError) {
            return propertyExecutionResult.mapToSuccessful();
        }
        return propertyExecutionResult;
    }
}

Have a look at PerProperty.Lifecycle to find out which aspects of a property’s lifecycle you can control.

Grouping Tests

Within a containing test class you can group other containers by embedding another non-static and non-private inner class and annotating it with @Group. Grouping examples and properties is a means to improve the organization and maintainability of your tests.

Groups can be nested and there lifecycle is also nested, that means that the lifecycle of a test class is also applied to inner groups of that container. Have a look at this example:

import net.jqwik.api.*;

class TestsWithGroups {

	@Property
	void outer(@ForAll String aString) {
	}

	@Group
	class Group1 {
		@Property
		void group1Property(@ForAll String aString) {
		}

		@Group
		class Subgroup {
			@Property
			void subgroupProperty(@ForAll String aString) {
			}
		}
	}

	@Group
	class Group2 {
		@Property
		void group2Property(@ForAll String aString) {
		}
	}
}

Naming and Labeling Tests

Using Java-style camel case naming for your test container classes and property methods will sometimes lead to hard to read display names in your test reports and your IDE. Therefore, jqwik provides a simple way to insert spaces into the displayed name of your test container or property: just add underscores (_), which are valid Java identifier characters. Each underscore will be replaced by a space for display purposes.

If you want to tweak display names even more, test container classes, groups, example methods and property methods can be labeled using the annotation @Label("a label"). This label will be used to display the element in test reports or within the IDE. In the following example, every test relevant element has been labeled:

@Label("Naming")
class NamingExamples {

	@Property
	@Label("a property")
	void aPropertyWithALabel() { }

	@Group
	@Label("A Group")
	class GroupWithLabel {
		@Example
		@Label("an example with äöüÄÖÜ")
		void anExampleWithALabel() { }
	}

    @Group
    class Group_with_spaces {
        @Example
        void example_with_spaces() { }
    }

}

Labels can consist of any characters and don’t have to be unique - but you probably want them to be unique within their container.

Tagging Tests

Test container classes, groups, example methods and property methods can be tagged using the annotation @Tag("a-tag"). You can have many tags on the same element.

Those tag can be used to filter the set of tests run by the IDE or the build tool. Tags are handed down from container (class or group) to its children (test methods or groups).

Have a look at the following example. Including the tag integration-test will include all tests of the class.

@Tag("integration-test")
class TaggingExamples {

	@Property
	@Tag("fast")
	void aFastProperty() { }

	@Example
	@Tag("slow") @Tag("involved")
	void aSlowTest() { }
}

Tags must follow certain rules as described here

Disabling Tests

From time to time you might want to disable a test or all tests in a container temporarily. You can do that by adding the @Disabled annotation to a property method or a container class.

import net.jqwik.api.Disabled;

@Disabled("for whatever reason")
class DisablingExamples {

	@Property
	@Disabled
	void aDisabledProperty() { }

}

Disabled properties will be reported by IDEs and build tools as “skipped” together with the reason - if one has been provided.

Be careful not to use the Jupiter annotation with the same name. Jqwik will refuse to execute methods that have Jupiter annotations.

Default Parameter Generation

jqwik tries to generate values for those property method parameters that are annotated with @ForAll. If the annotation does not have a value parameter, jqwik will use default generation for the following types:

If you use @ForAll with a value, e.g. @ForAll("aMethodName"), the method referenced by "aMethodName" will be called to provide an Arbitrary of the required type (see Parameter Provider Methods).

Constraining Default Generation

Default parameter generation can be influenced and constrained by additional annotations, depending on the requested parameter type.

Allow Null Values

Unique Values

String Length

Character Sets

When generating chars any unicode character might be generated.

When generating Strings, however, Unicode “noncharacters” and “private use characters” will not be generated unless you explicitly include them using @Chars or @CharRange (see below).

You can use the following annotations to restrict the set of allowed characters and even combine several of them:

They work for generated Strings and Characters.

List, Set, Stream, Map and Array Size

Integer Constraints

Decimal Constraints

Constraining parameterized types

When you want to constrain the generation of contained parameter types you can annotate the parameter type directly, e.g.:

@Property
void aProperty(@ForAll @Size(min= 1) List<@StringLength(max=10) String> listOfStrings) {
}

will generate lists with a minimum size of 1 filled with Strings that have 10 characters max.

Providing variable types

While checking properties of generically typed classes or functions, you often don’t care about the exact type of variables and therefore want to express them with type variables. jqwik can also handle type variables and wildcard types. The handling of upper and lower bounds works mostly as you would expect it.

Consider the following examples:

class VariableTypedPropertyExamples {

	@Property
	<T> boolean unboundedGenericTypesAreResolved(@ForAll List<T> items, @ForAll T newItem) {
		items.add(newItem);
		return items.contains(newItem);
	}

	@Property
	<T extends Serializable & Comparable> void someBoundedGenericTypesCanBeResolved(@ForAll List<T> items, @ForAll T newItem) {
	}

	@Property
	void someWildcardTypesWithUpperBoundsCanBeResolved(@ForAll List<? extends Serializable> items) {
	}

}

In the case of unbounded type variables or an unbounded wildcard type, jqwik will create instanced of a special class (WildcardObject) under the hood.

In the case of bounded type variables and bounded wildcard types, jqwik will check if any registered arbitrary provider can provide suitable arbitraries and choose randomly between those.

There is, however, a potentially unexpected behaviour, when the same type variable is used in more than one place and can be resolved by more than one arbitrary. In this case it can happen that the variable does not represent the same type in all places. You can see this above in property method someBoundedGenericTypesCanBeResolved() where items might be a list of Strings but newItem of some number type - and all that in the same call to the method!

Self-Made Annotations

You can make your own annotations instead of using jqwik’s built-in ones. BTW, ‘@Example’ is nothing but a plain annotation using @Property as “meta”-annotation.

The following example provides an annotation to constrain String or Character generation to German letters only:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
@NumericChars
@AlphaChars
@Chars({'ä', 'ö', 'ü', 'Ä', 'Ö', 'Ü', 'ß'})
@Chars({' ', '.', ',', ';', '?', '!'})
@StringLength(min = 10, max = 100)
public @interface GermanText { }

@Property(tries = 10) @Reporting(Reporting.GENERATED)
void aGermanText(@ForAll @GermanText String aText) {}

The drawback of self-made annotations is that they do not forward their parameters to meta-annotations, which constrains their applicability to simple cases.

Customized Parameter Generation

Sometimes the possibilities of adjusting default parameter generation through annotations is not enough. You want to control the creation of values programmatically. The means to do that are provider methods.

Parameter Provider Methods

Look at the following example:

@Property
boolean concatenatingStringWithInt(
    @ForAll("shortStrings") String aShortString,
    @ForAll("10 to 99") int aNumber
) {
    String concatenated = aShortString + aNumber;
    return concatenated.length() > 2 && concatenated.length() < 11;
}

@Provide
Arbitrary<String> shortStrings() {
    return Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(8);
}

@Provide("10 to 99")
Arbitrary<Integer> numbers() {
    return Arbitraries.integers().between(10, 99);
}

The String value of the @ForAll annotation serves as a reference to a method within the same class (or one of its superclasses or owning classes). This reference refers to either the method’s name or the String value of the method’s @Provide annotation.

The providing method has to return an object of type @Arbitrary<T> where T is the static type of the parameter to be provided. Optionally the provider method can take tow optional parameters:

These two objects can be used to get detailed information about the parameter, like annotations and embedded type parameters, and to resolve other types, usually from type parameters embedded in the original parameter. Use with care!

Parameter provision usually starts with a static method call to Arbitraries, maybe followed by one or more filtering, mapping or combining actions.

Providing Arbitraries for Embedded Types

There is an alternative syntax to @ForAll("methodRef") using a From annotation:

@Property
boolean concatenatingStringWithInt(
    @ForAll @From("shortStrings") String aShortString,
    @ForAll @From("10 to 99") int aNumber
) { ... }

Why this redundancy? Well, @From becomes a necessity when you want to provide the arbitrary of an embedded type parameter. Consider this example:

@Property
boolean joiningListOfStrings(@ForAll List<@From("shortStrings") String> listOfStrings) {
    String concatenated = String.join("", listOfStrings);
    return concatenated.length() <= 8 * listOfStrings.size();
}

Here, the list is created using the default list arbitrary, but the String elements are generated using the arbitrary from the method shortStrings.

Static Arbitraries methods

The starting point for generation usually is a static method call on class Arbitraries.

Generate values yourself

Select or generate values randomly

Select randomly with Weights

If you have a set of values to choose from with weighted probabilities, use ` Arbitraries.frequency(...):

@Property
void abcdWithFrequencies(@ForAll("abcdWeighted") String aString) {
    Statistics.collect(aString);
}

@Provide
Arbitrary<String> abcdWeighted() {
    return Arbitraries.frequency(
        Tuple.of(1, "a"),
        Tuple.of(5, "b"),
        Tuple.of(10, "c"),
        Tuple.of(20, "d")
    );
}

The first value of the tuple specifies the frequency of a particular value in relation to the sum of all frequencies. In the given example the sum is 36, thus "a" will be generated with a probability of 1/36 whereas "d" has a generation probability of 20/36 (= 5/9).

Shrinking moves towards the start of the frequency list.

Characters and Strings

java.util.Random

Shuffling Permutations

Default Types

Numeric Arbitrary Types

Creating an arbitrary for numeric values also starts by calling a static method on class Arbitraries. There are two fundamental types of numbers: integral numbers and decimal numbers. jqwik supports all of Java’s built-in number types.

Each type has its own fluent interface but all numeric arbitrary types share some things:

Integrals

Decimals

Decimal arbitrary types come with a few additional capabilities:

Random Numeric Distribution

With release 1.3.0 jqwik provides you with a means to influence the probability distribution of randomly generated numbers. The way to do that is by calling withDistribution(distribution). Currently three different distributions are supported:

The specified distribution does not influence the generation of edge cases.

The following example generates numbers between 0 and 20 using a gaussian probability distribution with its mean at 10 and a standard deviation of about 3.3:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = Histogram.class)
void gaussianDistributedIntegers(@ForAll("gaussians") int aNumber) {
    Statistics.collect(aNumber);
}

@Provide
Arbitrary<Integer> gaussians() {
    return Arbitraries
               .integers()
               .between(0, 20)
               .shrinkTowards(10)
               .withDistribution(RandomDistribution.gaussian());
}

Look at the statistics to see if it fits your expectation:

[RandomDistributionExamples:gaussianDistributedIntegers] (1000) statistics = 
       # | label | count | 
    -----|-------|-------|---------------------------------------------------------------------------------
       0 |     0 |    15 | ■■■■■
       1 |     1 |     8 | ■■
       2 |     2 |    12 | ■■■■
       3 |     3 |     9 | ■■■
       4 |     4 |    14 | ■■■■
       5 |     5 |    28 | ■■■■■■■■■
       6 |     6 |    38 | ■■■■■■■■■■■■■
       7 |     7 |    67 | ■■■■■■■■■■■■■■■■■■■■■■■
       8 |     8 |    77 | ■■■■■■■■■■■■■■■■■■■■■■■■■■
       9 |     9 |   116 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |    10 |   231 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    11 |   101 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      12 |    12 |    91 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      13 |    13 |    60 | ■■■■■■■■■■■■■■■■■■■■
      14 |    14 |    45 | ■■■■■■■■■■■■■■■
      15 |    15 |    36 | ■■■■■■■■■■■■
      16 |    16 |    19 | ■■■■■■
      17 |    17 |    10 | ■■■
      18 |    18 |     7 | ■■
      19 |    19 |     1 | 
      20 |    20 |    15 | ■■■■■

You can notice that values 0 and 20 should have the lowest probability but they do not. This is because they will be generated a few times as edge cases.

Collections, Streams, Iterators and Arrays

Arbitraries for multi value types require to start with an Arbitrary instance for the element type. You can then create the corresponding multi value arbitrary from there:

Collecting Values in a List

If you do not want any random combination of values in your list - as can be done with Arbitrary.list() - you have the possibility to collect random values in a list until a certain condition is fulfilled. Arbitrary.collect(Predicate condition) is what you need in those cases.

Imagine you need a list of integers the sum of which should be at least 1000. Here’s how you could do that:

Arbitrary<Integer> integers = Arbitraries.integers().between(1, 100);
Arbitrary<List<Integer>> collected = integers.collect(list -> sum(list) >= 1000);

Optional

Using Arbitrary.optional() allows to generate an optional of any type. Optional.empty() values are injected with a probability of 0.05, i.e. 1 in 20.

Tuples of same base type

If you want to generate tuples of the same base types that also use the same generator, that’s how you can do it:

Arbitrary<Tuple.Tuple2> integerPair = Arbitrary.integers().between(1, 25).tuple2();

There’s a method for tuples of length 1 to 4:

Maps

Generating instances of type Map is a bit different since two arbitraries are needed, one for the key and one for the value. Therefore you have to use Arbitraries.maps(...) like this:

@Property
void mapsFromNumberToString(@ForAll("numberMaps")  Map<Integer, String> map) {
    Assertions.assertThat(map.keySet()).allMatch(key -> key >= 0 && key <= 1000);
    Assertions.assertThat(map.values()).allMatch(value -> value.length() == 5);
}

@Provide
Arbitrary<Map<Integer, String>> numberMaps() {
    Arbitrary<Integer> keys = Arbitraries.integers().between(1, 100);
    Arbitrary<String> values = Arbitraries.strings().alpha().ofLength(5);
    return Arbitraries.maps(keys, values);
}

For generating individual Map.Entry instances there is Arbitraries.entries(...).

Functional Types

Interfaces that have a single (non default) method are considered to be Functional types; they are sometimes called SAM types for “single abstract method”. If a functional type is used as a @ForAll-parameter jqwik will automatically generate instances of those functions. The generated functions have the following characteristics:

Let’s look at an example:

@Property
void fromIntToString(@ForAll Function<Integer, @StringLength(5) String> function) {
    assertThat(function.apply(42)).hasSize(5);
    assertThat(function.apply(1)).isEqualTo(function.apply(1));
}

This works for any interface-based functional types, even your own. If you register a default provider for a functional type with a priority of 0 or above, it will take precedence.

If the functions need some specialized arbitrary for return values or if you want to fix the function’s behaviour for some range of values, you can define the arbitrary manually:

@Property
void emptyStringsTestFalse(@ForAll("predicates") Predicate<String> predicate) {
    assertThat(predicate.test("")).isFalse();
}

@Provide
Arbitrary<Predicate<String>> predicates() {
    return Functions
        .function(Predicate.class)
        .returns(Arbitraries.of(true, false))
        .when(parameters -> parameters.get(0).equals(""), parameters -> false);
}

In this example the generated predicate will always return false when given an empty String and randomly choose between true and false in all other cases.

Fluent Configuration Interfaces

Most specialized arbitrary interfaces provide special methods to configure things like size, length, boundaries etc. Have a look at the Java doc for the following types, which are organized in a flat hierarchy:

Here are a two examples to give you a hint of what you can do:

@Provide
Arbitrary<String> alphaNumericStringsWithMinLength5() {
    return Arbitraries.strings().ofMinLength(5).alpha().numeric();
}

@Provide
Arbitrary<List<Integer>> fixedSizedListOfPositiveIntegers() {
    return Arbitraries.integers().greaterOrEqual(0).list().ofSize(17);
}

Generate null values

Predefined generators will never create null values. If you want to allow that, call Arbitrary.injectNull(double probability). The following provider method creates an arbitrary that will return a null String in about 1 of 100 generated values.

@Provide 
Arbitrary<String> stringsWithNull() {
  return Arbitraries.strings(0, 10).injectNull(0.01);
}

Inject duplicate values

Sometimes it is important that your generator will create a previous value again in order to trigger certain scenarios or branches in your code. Imagine you want to check if your carefully hand-crafted String comparator really is as symmetric as it’s supposed to be:

Comparator<String> comparator = (s1, s2) -> {
    if (s1.length() + s2.length() == 0) return 0;
    if (s1.compareTo(s2) > 0) {
        return 1;
    } else {
        return -1;
    }
};

@Property
boolean comparing_strings_is_symmetric(@ForAll String first, @ForAll String second) {
    int comparison = comparator.compare(first, second);
    return comparator.compare(second, first) == -comparison;
}

The property (most probably) succeeds and will give you confidence in your code. Or does it? Natural scepticism makes you check some statistics:

@Property
boolean comparing_strings_is_symmetric(@ForAll String first, @ForAll String second) {
    int comparison = comparator.compare(first, second);
    String comparisonRange = comparison < 0 ? "<0" : comparison > 0 ? ">0" : "=0";
    String empty = first.isEmpty() || second.isEmpty() ? "empty" : "not empty";
    Statistics.collect(comparisonRange, empty);
    return comparator.compare(second, first) == -comparison;
}

The following output

[comparing strings is symmetric] (1000) statistics = 
    <0 not empty (471) : 47,10 %
    >0 not empty (456) : 45,60 %
    <0 empty     ( 37) :  3,70 %
    >0 empty     ( 35) :  3,50 %
    =0 empty     (  1) :  0,10 %

reveals that our generated test data is missing one combination: Comparison value of 0 for non-empty strings. In theory a generic String arbitrary could generate the same non-empty string but it’s highly unlikely. This is where we have to think about raising the probability of the same value being generated more often:

@Property
boolean comparing_strings_is_symmetric(@ForAll("pair") Tuple2<String, String> pair) {
    String first = pair.get1();
    String second = pair.get2();
    int comparison = comparator.compare(first, second);
    return comparator.compare(second, first) == -comparison;
}

@Provide
Arbitrary<Tuple2<String, String>> pair() {
    return Arbitraries.strings().injectDuplicates(0.1).tuple2();
}

This will cover the missing case and will reveal a bug in the comparator. Mind that you have to make sure that the same generator instance is being used for the two String values - using tuple2() does that.

Filtering

If you want to include only part of all the values generated by an arbitrary, use Arbitrary.filter(Predicate<T> filterPredicate). The following arbitrary will filter out all even numbers from the stream of generated integers:

@Provide 
Arbitrary<Integer> oddNumbers() {
  return Arbitraries.integers().filter(aNumber -> aNumber % 2 != 0);
}

Keep in mind that your filter condition should not be too restrictive. If the generator fails to find a suitable value after 10000 trials, the current property will be abandoned by throwing an exception.

Creating unique values

If you want to make sure that all the values generated by an arbitrary are unique, use Arbitrary.unique(). The following arbitrary will generate integers between 1 and 1000 but never the same integer twice:

@Provide
Arbitrary<Integer> oddNumbers() {
  return Arbitraries.integers().between(1, 1000).unique();
}

This means that a maximum of 1000 values can be generated. If the generator fails to find a yet unseen value after 10000 trials, the current property will be abandoned by throwing an exception.

Mapping

Sometimes it’s easier to start with an existing arbitrary and use its generated values to build other objects from them. In that case, use Arbitrary.map(Function<T, U> mapper). The following example uses generated integers to create numerical Strings:

@Provide 
Arbitrary<String> fiveDigitStrings() {
  return Arbitraries.integers(10000, 99999).map(aNumber -> String.valueOf(aNumber));
}

You could generate the same kind of values by constraining and filtering a generated String. However, the shrinking target would probably be different. In the example above, shrinking will move towards the lowest allowed number, that is 10000.

Flat Mapping

Similar as in the case of Arbitrary.map(..) there are situations in which you want to use a generated value in order to create another Arbitrary from it. Sounds complicated? Have a look at the following example:

@Property
boolean fixedSizedStrings(@ForAll("listsOfEqualSizedStrings")List<String> lists) {
    return lists.stream().distinct().count() == 1;
}

@Provide
Arbitrary<List<String>> listsOfEqualSizedStrings() {
    Arbitrary<Integer> integers2to5 = Arbitraries.integers().between(2, 5);
    return integers2to5.flatMap(stringSize -> {
        Arbitrary<String> strings = Arbitraries.strings() //
                .withCharRange('a', 'z') //
                .ofMinLength(stringSize).ofMaxLength(stringSize);
        return strings.list();
    });
}

The provider method will create random lists of strings, but in each list the size of the contained strings will always be the same - between 2 and 5.

Flat Mapping with Tuple Types

In the example above you used a generated value in order to create another arbitrary. In those situations you often want to also provide the original values to your property test.

Imagine, for instance, that you’d like to test properties of String.substring(begin, end). To randomize the method call, you not only need a string but also the begin and end indices. However, both have dependencies:

@Property
void substringLength(@ForAll("stringWithBeginEnd") Tuple3<String, Integer, Integer> stringBeginEnd) {
    String aString = stringBeginEnd.get1();
    int begin = stringBeginEnd.get2();
    int end = stringBeginEnd.get3();
    Assertions.assertThat(aString.substring(begin, end).length())
        .isEqualTo(end - begin);
}

@Provide
Arbitrary<Tuple3<String, Integer, Integer>> stringWithBeginEnd() {
    Arbitrary<String> stringArbitrary = Arbitraries.strings() //
            .withCharRange('a', 'z') //
            .ofMinLength(2).ofMaxLength(20);
    return stringArbitrary //
            .flatMap(aString -> Arbitraries.integers().between(0, aString.length()) //
                    .flatMap(end -> Arbitraries.integers().between(0, end) //
                            .map(begin -> Tuple.of(aString, begin, end))));
}

Mind the nested flat mapping, which is an aesthetic nuisance but nevertheless very useful.

Randomly Choosing among Arbitraries

If you have several arbitraries of the same type, you can create a new arbitrary of the same type which will choose randomly one of those arbitraries before generating a value:

@Property
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    String classifier = anInt < -1000 ? "below" : anInt > 1000 ? "above" : "one";
    Statistics.collect(classifier);
    
    return anInt < -1000 //
            || Math.abs(anInt) == 1 //
            || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-2000, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 2000);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);
    
    return Arbitraries.oneOf(below1000, above1000, oneOrMinusOne);
}

In this example the statistics should also give you an equal distribution between the three types of integers.

If you don’t want to choose with equal probability - but with differing frequency - you can do that in a similar way:

@Property(tries = 100)
@Report(Reporting.GENERATED)
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    return anInt < -1000 //
               || Math.abs(anInt) == 1 //
               || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-1050, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 1050);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);

    return Arbitraries.frequencyOf(
        Tuple.of(1, below1000),
        Tuple.of(3, above1000),
        Tuple.of(6, oneOrMinusOne)
    );
}

Combining Arbitraries

Sometimes just mapping a single stream of generated values is not enough to generate a more complicated domain object. In those cases you can combine several arbitraries to a single result arbitrary using Combinators.combine() with up to eight arbitraries. Create an issue on github if you need more than eight.

The following example generates Person instances from three arbitraries as inputs.

@Property
void validPeopleHaveIDs(@ForAll("validPeople") Person aPerson) {
    Assertions.assertThat(aPerson.getID()).contains("-");
    Assertions.assertThat(aPerson.getID().length()).isBetween(5, 24);
}

@Provide
Arbitrary<Person> validPeople() {
    Arbitrary<String> names = Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(3).ofMaxLength(21);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    return Combinators.combine(names, ages)
        .as((name, age) -> new Person(name, age));
}

class Person {
    private final String name;
    private final int age;

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String getID() {
        return name + "-" + age;
    }

    @Override
    public String toString() {
        return String.format("%s:%s", name, age);
    }
}

The property should fail, thereby shrinking the falsified Person instance to

Shrunk Sample (<n> steps)
-------------------------
  aPerson: aaaaaaaaaaaaaaaaaaaaa:100

The Combinators.combine method accepts up to 8 parameters of type Arbitrary. If you need more you have a few options:

Combining Arbitraries with Builder

There’s an alternative way to combine arbitraries to create an aggregated object by using a builder for the aggregated object. Consider the example from above and throw a PersonBuilder into the mix:

static class PersonBuilder {

    private String name = "A name";
    private int age = 42;

    public PersonBuilder withName(String name) {
        this.name = name;
        return this;
    }

    public PersonBuilder withAge(int age) {
        this.age = age;
        return this;
    }

    public Person build() {
        return new Person(name, age);
    }
}

Then you can go about generating people in the following way:

@Provide
Arbitrary<Person> validPeopleWithBuilder() {
    Arbitrary<String> names = 
        Arbitraries.strings().withCharRange('a', 'z').ofMinLength(2).ofMaxLength(20);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    
    return Combinators.withBuilder(() -> new PersonBuilder())
        .use(names).in((builder, name) -> builder.withName(name))
        .use(ages).in((builder, age)-> builder.withAge(age))
        .build( builder -> builder.build());
}

Have a look at Combinators.withBuilder(Supplier) and Combinators.withBuilder(Arbitrary) to check the API.

Flat Combination

If generating domain values requires to use several generated values to be used in generating another one, there’s the combination of flat mapping and combining:

@Property
boolean fullNameHasTwoParts(@ForAll("fullName") String aName) {
    return aName.split(" ").length == 2;
}

@Provide
Arbitrary<String> fullName() {
    IntegerArbitrary firstNameLength = Arbitraries.integers().between(2, 10);
    IntegerArbitrary lastNameLength = Arbitraries.integers().between(2, 10);
    return Combinators.combine(firstNameLength, lastNameLength).flatAs( (fLength, lLength) -> {
        Arbitrary<String> firstName = Arbitraries.strings().alpha().ofLength(fLength);
        Arbitrary<String> lastName = Arbitraries.strings().alpha().ofLength(fLength);
        return Combinators.combine(firstName, lastName).as((f,l) -> f + " " + l);
    });
}

Often, however, there’s an easier way to achieve the same goal which does not require the flat combination of arbitraries:

@Provide
Arbitrary<String> fullName2() {
    Arbitrary<String> firstName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    Arbitrary<String> lastName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    return Combinators.combine(firstName, lastName).as((f, l) -> f + " " + l);
}

This is not only easier to understand but it usually improves shrinking.

Ignoring Exceptions During Generation

Once in a while, usually when combining generated values, it’s difficult to figure out in advance all the constraints that make the generation of objects valid. In a good object-oriented model, however, the objects themselves – i.e. their constructors or factory methods – take care that only valid objects can be created. The attempt to create an invalid value will be rejected with an exception.

As a good example have a look at JDK’s LocalDate class, which allows to instantiate dates using LocalDate.of(int year, int month, int dayOfMonth). In general dayOfMonth can be between 1 and 31 but trying to generate a “February 31” will throw a DateTimeException. Therefore, when you want to randomly generated dates between “January 1 1900” and “December 31 2099” you have two choices:

@Provide
Arbitrary<LocalDate> datesBetween1900and2099() {
  Arbitrary<Integer> years = Arbitraries.integers().between(1900, 2099);
  Arbitrary<Integer> months = Arbitraries.integers().between(1, 12);
  Arbitrary<Integer> days = Arbitraries.integers().between(1, 31);
  
  return Combinators.combine(years, months, days)
  	  .as(LocalDate::of)
  	  .ignoreException(DateTimeException.class);
}

Fix an Arbitrary’s genSize

Some generators (e.g. most number generators) are sensitive to the genSize value that is used when creating them. The default value for genSize is the number of tries configured for the property they are used in. If there is a need to influence the behaviour of generators you can do so by using Arbitrary.fixGenSize(int)..

Recursive Arbitraries

Sometimes it seems like a good idea to compose arbitraries and thereby recursively calling an arbitrary creation method. Generating recursive data types is one application field but you can also use it for other stuff.

Probabilistic Recursion

Look at the following example which generates sentences by recursively adding words to a sentence:

@Property
@Report(Reporting.GENERATED)
boolean sentencesEndWithAPoint(@ForAll("sentences") String aSentence) {
	return aSentence.endsWith(".");
	// return !aSentence.contains("x"); // using this condition instead 
	                                    // should shrink to "AAAAx."
}

@Provide
Arbitrary<String> sentences() {
	return Arbitraries.lazyOf(
		() -> word().map(w -> w + "."),
		this::sentence,
		this::sentence,
		this::sentence
	);
}

private Arbitrary<String> sentence() {
	return Combinators.combine(sentences(), word())
					  .as((s, w) -> w + " " + s);
}

private StringArbitrary word() {
    return Arbitraries.strings().alpha().ofLength(5);
}

There are two things to which you must pay attention:

There is also a caveat of which you should be aware: Never use this construct if suppliers make use of variable state like method parameters or changing instance members. In those cases use lazy() as explained below.

Using lazy() instead of lazyOf()

There is an almost equivalent variant to the example above:

@Property
boolean sentencesEndWithAPoint(@ForAll("sentences") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> sentences() {
    Arbitrary<String> sentence = Combinators.combine(
        Arbitraries.lazy(this::sentences),
        word()
    ).as((s, w) -> w + " " + s);

    return Arbitraries.oneOf(
        word().map(w -> w + "."),
        sentence,
        sentence,
        sentence
    );
}

private StringArbitrary word() {
    return Arbitraries.strings().alpha().ofLength(5);
}

The disadvantage of lazy() combined with oneOf() or frequencyOf() is its worse shrinking behaviour compared to lazyOf(). Therefore, choose lazyOf() whenever you can.

Deterministic Recursion

An alternative to probabilistic recursion shown above, is to use deterministic recursion with a counter to determine the base case. If you then use an arbitrary value for the counter, the generated sentences will be very similar, and you can often forgo using Arbitraries.lazyOf() or Arbitraries.lazy():

@Property(tries = 10)
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
    Arbitrary<Integer> length = Arbitraries.integers().between(0, 10);
    Arbitrary<String> lastWord = word().map(w -> w + ".");
    return length.flatMap(l -> deterministic(l, lastWord));
}

@Provide
Arbitrary<String> deterministic(int length, Arbitrary<String> sentence) {
    if (length == 0) {
        return sentence;
    }
    Arbitrary<String> more = Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
    return deterministic(length - 1, more);
}

Deterministic Recursion with recursive()

To further simplify this jqwik provides a helper function: Arbitraries.recursive(...). Using that further simplifies the example:

@Property(tries = 10)
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
	Arbitrary<Integer> length = Arbitraries.integers().between(0, 10);
	Arbitrary<String> lastWord = word().map(w -> w + ".");

	return length.flatMap(depth -> Arbitraries.recursive(
		() -> lastWord,
		this::prependWord,
		depth
	));
}

private Arbitrary<String> prependWord(Arbitrary<String> sentence) {
    return Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
}

Using Arbitraries Directly

Most of the time arbitraries are used indirectly, i.e. jqwik uses them under the hood to inject generated values as parameters. There are situations, though, in which you might want to generate values directly.

Generating a Single Value

Getting a single random value out of an arbitrary is easy and can be done with Arbitrary.sample():

Arbitrary<String> strings = Arbitraries.of("string1", "string2", "string3");
String aString = strings.sample();
assertThat(aString).isIn("string1", "string2", "string3");

Among other things, this allows you to use jqwik’s generation functionality with other test engines like Jupiter. Mind that jqwik uses a default genSize of 1000 under the hood and that the Random object will be either taken from the current property’s context or freshly instantiated if used outside a property.

Generating a Stream of Values

Getting a stream of generated values is just as easy with Arbitrary.sampleStream():

List<String> values = Arrays.asList("string1", "string2", "string3");
Arbitrary<String> strings = Arbitraries.of(values);
Stream<String> streamOfStrings = strings.sampleStream();

assertThat(streamOfStrings).allMatch(values::contains);

Generating all possible values

There are a few cases when you don’t want to generate individual values from an arbitrary but use all possible values to construct another arbitrary. This can be achieved through Arbitrary.allValues().

Return type is Optional<Stream<T>> because jqwik can only perform this task if exhaustive generation is doable.

Iterating through all possible values

You can also use an arbitrary to iterate through all values it specifies. Use Arbitrary.forEachValue(Consumer action). for that purpose. This only works when exhaustive generation is possible. In other cases the attempt to iterate will result in an exception.

This is typically useful when your test requires to assert some fact for all values of a given (sub)set of objects. Here’s a contrived example:

@Property
void canPressAnyKeyOnKeyboard(@ForAll Keyboard keyboard, @ForAll Key key) {
    keyboard.press(key);
    assertThat(keyboard.isPressed(key));

    Arbitrary<Key> unpressedKeys = Arbitraries.of(keyboard.allKeys()).filter(k -> !k.equals(key));
    unpressedKeys.forEachValue(k -> assertThat(keyboard.isPressed(k)).isFalse());
}

In this example a simple for loop over allKeys() would also work. In more complicated scenarios jqwik will do all the combinations and filtering for you.

Contract Tests

When you combine type variables with properties defined in superclasses or interfaces you can do some kind of contract testing. That means that you specify the properties in a generically typed interface and specify the concrete class to instantiate in a test container implementing the interface.

The following example was influence by a similar feature in junit-quickcheck. Here’s the contract:

interface ComparatorContract<T> {
	Comparator<T> subject();

	@Property
	default void symmetry(@ForAll("anyT") T x, @ForAll("anyT") T y) {
		Comparator<T> subject = subject();

		Assertions.assertThat(signum(subject.compare(x, y))).isEqualTo(-signum(subject.compare(y, x)));
	}

	@Provide
	Arbitrary<T> anyT();
}

And here’s the concrete test container that can be run to execute the property with generated Strings:

class StringCaseInsensitiveProperties implements ComparatorContract<String> {

	@Override public Comparator<String> subject() {
		return String::compareToIgnoreCase;
	}

	@Override
	@Provide
	public Arbitrary<String> anyT() {
		return Arbitraries.strings().alpha().ofMaxLength(20);
	}
}

What we can see here is that jqwik is able to figure out the concrete type of type variables when they are used in subtypes that fill in the variables.

Stateful Testing

Despite its bad reputation state is an important concept in object-oriented languages like Java. We often have to deal with stateful objects or components whose state can be changed through methods.

Thinking in a more formal way we can look at those objects as state machines and the methods as actions that move the object from one state to another. Some actions have preconditions to constrain when they can be invoked and some objects have invariants that should never be violated regardless of the sequence of performed actions.

To make this abstract concept concrete, let’s look at a simple stack implementation:

public class MyStringStack {
	public void push(String element) { ... }
	public String pop() { ... }
	public void clear() { ... }
	public boolean isEmpty() { ... }
	public int size() { ... }
	public String top() { ... }
}

Specify Actions

We can see at least three actions with their preconditions and expected state changes:

Check Postconditions

The fundamental property that jqwik should try to falsify is:

For any valid sequence of actions all required state changes
(aka postconditions) should be fulfilled.

We can formulate that quite easily as a jqwik property:

class MyStringStackProperties {

	@Property
	void checkMyStack(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
		actions.run(new MyStringStack());
	}

	@Provide
	Arbitrary<ActionSequence<MyStringStack>> sequences() {
		return Arbitraries.sequences(Arbitraries.oneOf(push(), pop(), clear()));
	}

	private Arbitrary<Action<MyStringStack>> push() {
		return Arbitraries.strings().alpha().ofLength(5).map(PushAction::new);
	}

	private Arbitrary<Action<MyStringStack>> clear() {
		return Arbitraries.just(new ClearAction());
	}

	private Arbitrary<Action<MyStringStack>> pop() {
		return Arbitraries.just(new PopAction());
	}
}

The interesting API elements are

To give jqwik something to falsify, we broke the implementation of clear() so that it won’t clear everything if there are more than two elements on the stack:

public void clear() {
    // Wrong implementation to provoke falsification for stacks with more than 2 elements
    if (elements.size() > 2) {
        elements.remove(0);
    } else {
        elements.clear();
    }
}

Running the property should now produce a result similar to:

org.opentest4j.AssertionFailedError: 
  Run failed after following actions:
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      clear
    final state: ["AAAAA", "AAAAA"]

Number of actions

jqwik will vary the number of generated actions according to the number of tries of your property. For the default of 1000 tries a sequence will have up to 32 actions. If need be you can specify the number of actions to generate using either the fluent interface or the @Size annotation:

@Property
// check stack with sequences of 7 actions:
void checkMyStack(@ForAll("sequences") @Size(max = 7) ActionSequence<MyStringStack> actions) {
    actions.run(new MyStringStack());
}

The minimum number of generated actions in a sequence is 1 since checking an empty sequence does not make sense.

Check Invariants

We can also add invariants to our sequence checking property:

@Property
void checkMyStackWithInvariant(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
    actions
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isGreaterThanOrEqualTo(0))
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isLessThan(5))
        .run(new MyStringStack());
}

If we first fix the bug in MyStringStack.clear() our property should eventually fail with the following result:

org.opentest4j.AssertionFailedError: 
  Run failed after following actions:
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
    final state: ["AAAAA", "AAAAA", "AAAAA", "AAAAA", "AAAAA"]

Assumptions

If you want to constrain the set of generated values in a way that embraces more than one parameter, filtering does not work. What you can do instead is putting one or more assumptions at the beginning of your property.

The following property works only on strings that are not equal:

@Property
boolean comparingUnequalStrings(
        @ForAll @StringLength(min = 1, max = 10) String string1,
        @ForAll @StringLength(min = 1, max = 10) String string2
) {
    Assume.that(!string1.equals(string2));

    return string1.compareTo(string2) != 0;
}

This is a reasonable use of Assume.that(boolean condition) because most generated value sets will pass through.

Have a look at a seemingly similar example:

@Property
boolean findingContainedStrings(
        @ForAll @StringLength(min = 1, max = 10) String container,
        @ForAll @StringLength(min = 1, max = 5) String contained
) {
    Assume.that(container.contains(contained));

    return container.indexOf(contained) >= 0;
}

Despite the fact that the property condition itself is correct, the property will most likely fail with the following message:

org.opentest4j.AssertionFailedError: 
    Property [findingContainedStrings] exhausted after [1000] tries and [980] rejections

tries = 1000 
checks = 20 
generation = RANDOMIZED
after-failure = PREVIOUS_SEED
edge-cases#mode = MIXIN 
seed = 1066117555581106850

The problem is that - given a random generation of two strings - only in very few cases one string will be contained in the other. jqwik will report a property as exhausted if the ratio between generated and accepted parameters is higher than 5. You can change the maximum discard ratio by specifying a parameter maxDiscardRatio in the @Property annotation. That’s why changing to @Property(maxDiscardRatio = 100) in the previous example will probably result in a successful property run, even though only a handful cases - of 1000 generated - will actually be checked.

In many cases turning up the accepted discard ration is a bad idea. With some creativity we can often avoid the problem by generating out test data a bit differently. Look at this variant of the above property, which also uses Assume.that() but with a much lower discard ratio:

@Property
boolean findingContainedStrings_variant(
        @ForAll @StringLength(min = 5, max = 10) String container,
        @ForAll @IntRange(min = 1, max = 5) int length,
        @ForAll @IntRange(min = 0, max = 9) int startIndex
) {
    Assume.that((length + startIndex) <= container.length());

    String contained = container.substring(startIndex, startIndex + length);
    return container.indexOf(contained) >= 0;
}

Result Shrinking

If a property could be falsified with a generated set of values, jqwik will try to “shrink” this sample in order to find a “smaller” sample that also falsifies the property.

Try this property:

@Property
boolean stringShouldBeShrunkToAA(@ForAll @AlphaChars String aString) {
    return aString.length() > 5 || aString.length() < 2;
}

The test run result should look something like:

AssertionFailedError: Property [stringShouldBeShrunkToAA] falsified with sample {0="aa"}

tries = 38 
checks = 38 
...
Shrunk Sample (5 steps)
-------------------------
  aString: "AA"

Original Sample
---------------
  aString: "RzZ"

In this case the original sample could be any string between 2 and 5 chars, whereas the final sample should be exactly AA since this is the shortest failing string and A has the lowest numeric value of all allowed characters.

Integrated Shrinking

jqwik’s shrinking approach is called integrated shrinking, as opposed to type-based shrinking which most property-based testing tools use. The general idea and its advantages are explained here.

Consider a somewhat more complicated example:

@Property
boolean shrinkingCanTakeAWhile(@ForAll("first") String first, @ForAll("second") String second) {
    String aString = first + second;
    return aString.length() > 5 || aString.length() < 2;
}

@Provide
Arbitrary<String> first() {
    return Arbitraries.strings()
        .withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(10)
        .filter(string -> string.endsWith("h"));
}

@Provide
Arbitrary<String> second() {
    return Arbitraries.strings()
        .withCharRange('0', '9')
        .ofMinLength(0).ofMaxLength(10)
        .filter(string -> string.length() >= 1);
}

Shrinking still works, although there’s quite a bit of filtering and string concatenation happening:

AssertionFailedError: Property [shrinkingCanTakeLong] falsified with sample {0="a", 1="000"}}

checks = 20 
tries = 20 
...
Shrunk Sample (3 steps)
-----------------------
  first: "a"
  second: "000"

Original Sample
---------------
  first: "h"
  second: "901"

This example also shows that sometimes there is no single “smallest example”. Depending on the starting random seed, this property will shrink to either {0="a", 1="000"}, {0="ah", 1="00"} or {0="aah", 1="0"}, all of which are considered to be the smallest possible for jqwik’s current way of measuring a sample’s size.

Switch Shrinking Off

Sometimes shrinking takes a really long time or won’t finish at all (usually a jqwik bug!). In those cases you can switch shrinking off for an individual property:

@Property(shrinking = ShrinkingMode.OFF)
void aPropertyWithLongShrinkingTimes(
	@ForAll List<Set<String>> list1, 
	@ForAll List<Set<String>> list2
) {	... }

Switch Shrinking to Full Mode

Sometimes you can find a message like

shrinking bound reached = after 1000 steps.

in your testrun’s output. This happens in rare cases when jqwik has not found the end of its search for simpler falsifiable values after 1000 iterations. In those cases you can try

@Property(shrinking = ShrinkingMode.FULL)

to tell jqwik to go all the way, even if it takes a million steps, even if it never ends…

Change the Shrinking Target

By default shrinking of numbers will move towards zero (0). If zero is outside the bounds of generation the closest number to zero - either the min or max value - is used as a target for shrinking. There are cases, however, when you’d like jqwik to choose a different shrinking target, usually when the default value of a number is not 0.

Consider generating signals with a standard frequency of 50 hz that can vary by plus/minus 5 hz. If possible, shrinking of falsified scenarios should move towards the standard frequency. Here’s how the provider method might look:

@Provide
Arbitrary<List<Signal>> signals() {
	Arbitrary<Long> frequencies = 
	    Arbitraries
            .longs()
            .between(45, 55)
            .shrinkTowards(50);

	return frequencies.map(f -> Signal.withFrequency(f)).list().ofMaxSize(1000);
}

Currently shrinking targets are supported for all number types.

Platform Reporting with Reporter Object

If you want to provide additional information during a test or a property using System.out.println() is a common choice. The JUnit platform, however, provides a better mechanism to publish additional information in the form of key-value pairs. Those pairs will not only printed to stdout but are also available to downstream tools like test report generators in continue integration.

You can hook into this reporting mechanism through jqwik’s Reporter object. This object is available in lifecycle hooks but you can also have it injected as a parameter into your test method:

@Example
void reportInCode(Reporter reporter, @ForAll List<@AlphaChars String> aList) {
	reporter.publishReport("listOfStrings", aList);
	reporter.publishValue("birthday", LocalDate.of(1969, 1, 20).toString());
}

net.jqwik.api.Reporter has different publishing methods. Those with report in their name use jqwik’s reporting mechanism and formats described above.

Collecting and Reporting Statistics

In many situations you’d like to know if jqwik will really generate the kind of values you expect and if the frequency and distribution of certain value classes meets your testing needs. Statistics.collect() is made for this exact purpose.

In the most simple case you’d like to know how often a certain value is being generated:

@Property
void simpleStats(@ForAll RoundingMode mode) {
    Statistics.collect(mode);
}

will create an output similar to that:

[MyTest:simpleStats] (1000) statistics = 
    FLOOR       (158) : 16 %
    HALF_EVEN   (135) : 14 %
    DOWN        (126) : 13 %
    UP          (120) : 12 %
    HALF_UP     (118) : 12 %
    CEILING     (117) : 12 %
    UNNECESSARY (117) : 12 %
    HALF_DOWN   (109) : 11 %

More typical is the case in which you’ll classify generated values into two or more groups:

@Property
void integerStats(@ForAll int anInt) {
    Statistics.collect(anInt > 0 ? "positive" : "negative");
}

[MyTest:integerStats] (1000) statistics = 
    negative (506) : 51 %
    positive (494) : 49 %

You can also collect the distribution in more than one category and combine those categories:

@Property
void combinedIntegerStats(@ForAll int anInt) {
    String posOrNeg = anInt > 0 ? "positive" : "negative";
    String evenOrOdd = anInt % 2 == 0 ? "even" : "odd";
    String bigOrSmall = Math.abs(anInt) > 50 ? "big" : "small";
    Statistics.collect(posOrNeg, evenOrOdd, bigOrSmall);
}

[MyTest:combinedIntegerStats] (1000) statistics = 
    negative even big   (222) : 22 %
    positive even big   (201) : 20 %
    positive odd big    (200) : 20 %
    negative odd big    (194) : 19 %
    negative even small ( 70) :  7 %
    positive odd small  ( 42) :  4 %
    negative odd small  ( 38) :  4 %
    positive even small ( 33) :  3 %

And, of course, you can combine different generated parameters into one statistical group:

@Property
void twoParameterStats(
    @ForAll @Size(min = 1, max = 10) List<Integer> aList,
    @ForAll @IntRange(min = 0, max = 10) int index
) {
    Statistics.collect(aList.size() > index ? "index within size" : null);
}

[MyTest:twoParameterStats] (1000) statistics = 
    index within size (507) : 51 %

As you can see, collected null values are not being reported.

Here are a couple of examples to try out.

Labeled Statistics

If you want more than one statistic in a single property, you must give them labels for differentiation:

@Property
void severalStatistics(@ForAll @IntRange(min = 1, max = 10) Integer anInt) {
    String range = anInt < 3 ? "small" : "large";
    Statistics.label("range").collect(range);
    Statistics.label("value").collect(anInt);
}

produces the following reports:

[MyTest:labeledStatistics] (1000) range = 
    large (783) : 78 %
    small (217) : 22 %

[MyTest:labeledStatistics] (1000) value = 
    1  (115) : 12 %
    5  (109) : 11 %
    10 (105) : 11 %
    4  (103) : 10 %
    2  (102) : 10 %
    3  ( 99) : 10 %
    6  ( 97) : 10 %
    8  ( 92) :  9 %
    7  ( 91) :  9 %
    9  ( 87) :  9 %

Statistics Report Formatting

There is a @StatisticsReport annotation that allows to change statistics report formats or to even switch it off. The annotation can be used on property methods or on container classes.

The value attribute is of type StatisticsReportMode.OFF and can have one of:

Switch Statistics Reporting Off

You can switch off statistics report as simple as that:

@Property
@StatisticsReport(StatisticsReport.StatisticsReportMode.OFF)
void queryStatistics(@ForAll int anInt) {
	Statistics.collect(anInt);
}

Histograms

jqwik comes with two report formats to display collected data as histograms: Histogram and NumberRangeHistogram.

Histogram displays the collected raw data as a histogram:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = Histogram.class)
void integers(@ForAll("gaussians") int aNumber) {
    Statistics.collect(aNumber);
}

@Provide
Arbitrary<Integer> gaussians() {
    return Arbitraries
               .integers()
               .between(0, 20)
               .shrinkTowards(10)
               .withDistribution(RandomDistribution.gaussian());
}

[HistogramExamples:integers] (1000) statistics = 
       # | label | count | 
    -----|-------|-------|---------------------------------------------------------------------------------
       0 |     0 |    13 | ■■■■
       1 |     1 |    13 | ■■■■
       2 |     2 |    15 | ■■■■■
       3 |     3 |     6 | ■■
       4 |     4 |    10 | ■■■
       5 |     5 |    22 | ■■■■■■■
       6 |     6 |    49 | ■■■■■■■■■■■■■■■■
       7 |     7 |    60 | ■■■■■■■■■■■■■■■■■■■■
       8 |     8 |   102 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
       9 |     9 |   100 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |    10 |   233 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    11 |   114 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      12 |    12 |    74 | ■■■■■■■■■■■■■■■■■■■■■■■■■
      13 |    13 |    64 | ■■■■■■■■■■■■■■■■■■■■■
      14 |    14 |    43 | ■■■■■■■■■■■■■■
      15 |    15 |    32 | ■■■■■■■■■■
      16 |    16 |    16 | ■■■■■
      17 |    17 |     8 | ■■
      18 |    18 |     7 | ■■
      19 |    20 |    19 | ■■■■■■

NumberRangeHistogram clusters the collected raw data into ranges:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = NumberRangeHistogram.class)
void integersInRanges(@ForAll @IntRange(min = -1000, max = 1000) int aNumber) {
    Statistics.collect(aNumber);
}

[HistogramExamples:integersInRanges] (1000) statistics = 
       # |         label | count | 
    -----|---------------|-------|---------------------------------------------------------------------------------
       0 | [-1000..-900[ |    20 | ■■■■■
       1 |  [-900..-800[ |    17 | ■■■■
       2 |  [-800..-700[ |    16 | ■■■■
       3 |  [-700..-600[ |     8 | ■■
       4 |  [-600..-500[ |    12 | ■■■
       5 |  [-500..-400[ |    14 | ■■■
       6 |  [-400..-300[ |    17 | ■■■■
       7 |  [-300..-200[ |    46 | ■■■■■■■■■■■
       8 |  [-200..-100[ |    59 | ■■■■■■■■■■■■■■
       9 |     [-100..0[ |   315 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |      [0..100[ |   276 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    [100..200[ |    47 | ■■■■■■■■■■■
      12 |    [200..300[ |    49 | ■■■■■■■■■■■■
      13 |    [300..400[ |    25 | ■■■■■■
      14 |    [400..500[ |    14 | ■■■
      15 |    [500..600[ |    13 | ■■■
      16 |    [600..700[ |    15 | ■■■
      17 |    [700..800[ |    14 | ■■■
      18 |    [800..900[ |    11 | ■■
      19 |   [900..1000] |    12 | ■■■

Both types can be subclassed to override behaviour like the number of buckets, the maximum drawing range of the bar, the order of elements and the label of a bucket.

Make Your Own Statistics Report Format

In order to format statistics to your own liking you have to create an implementation of type StatisticsReportFormat and

@Property
@StatisticsReport(format = MyStatisticsFormat.class)
void statisticsWithHandMadeFormat(@ForAll Integer anInt) {
	String range = anInt < 0 ? "negative" : anInt > 0 ? "positive" : "zero";
	Statistics.collect(range);
}

class MyStatisticsFormat implements StatisticsReportFormat {
	@Override
	public List<String> formatReport(List<StatisticsEntry> entries) {
		return entries.stream()
					  .map(e -> String.format("%s: %d", e.name(), e.count()))
					  .collect(Collectors.toList());
	}
}

Running this property should produce a report similar to that:

[StatisticsExamples:statisticsWithHandMadeFormat] (1000) statistics = 
    negative: 520
    positive: 450
    zero: 30

Checking Coverage of Collected Statistics

Just looking at the statistics of generated values might not be sufficient. Sometimes you want to make sure that certain scenarios are being covered by your generators and fail a property otherwise. In jqwik you do that by first collecting statistics and then specifying coverage conditions for those statistics.

Check Percentages and Counts

The following example does that for generated values of enum RoundingMode:

@Property(generation = GenerationMode.RANDOMIZED)
void simpleStats(@ForAll RoundingMode mode) {
	Statistics.collect(mode);

	Statistics.coverage(coverage -> {
		coverage.check(RoundingMode.CEILING).percentage(p -> p > 5.0);
		coverage.check(RoundingMode.FLOOR).count(c -> c > 2);
	});
}

The same thing is possible for values collected with a specific label and in a fluent API style.

@Property(generation = GenerationMode.RANDOMIZED)
void labeledStatistics(@ForAll @IntRange(min = 1, max = 10) Integer anInt) {
	String range = anInt < 3 ? "small" : "large";

	Statistics.label("range")
			  .collect(range)
			  .coverage(coverage -> coverage.check("small").percentage(p -> p > 20.0));

	Statistics.label("value")
			  .collect(anInt)
			  .coverage(coverage -> coverage.check(0).count(c -> c > 0));
}

Start by looking at Statistics.coverage() to see all the options you have for checking percentages and counts.

Check Ad-hoc Query Coverage

Instead of classifying values at collection time you have the possibility to collect the raw data and use a query when doing coverage checking:

@Property
@StatisticsReport(StatisticsReport.StatisticsReportMode.OFF)
void queryStatistics(@ForAll int anInt) {
	Statistics.collect(anInt);

	Statistics.coverage(coverage -> {
		Predicate<List<Integer>> isZero = params -> params.get(0) == 0;
		coverage.checkQuery(isZero).percentage(p -> p > 5.0);
	});
}

In those cases you probably want to switch off reporting, otherwise the reports might get very long - and without informative value.

Providing Default Arbitraries

Sometimes you want to use a certain, self-made Arbitrary for one of your own domain classes, in all of your properties, and without having to add @Provide method to all test classes. jqwik enables this feature by using Java’s java.util.ServiceLoader mechanism. All you have to do is:

jqwik will then add an instance of your arbitrary provider into the list of its default providers. Those default providers are considered for every test parameter annotated with @ForAll that has no explicit value. By using this mechanism you can also replace the default providers packaged into jqwik.

Simple Arbitrary Providers

A simple provider is one that delivers arbitraries for types without type variables. Consider the class Money:

public class Money {
	public BigDecimal getAmount() {
		return amount;
	}

	public String getCurrency() {
		return currency;
	}

	public Money(BigDecimal amount, String currency) {
		this.amount = amount;
		this.currency = currency;
	}

	public Money times(int factor) {
		return new Money(amount.multiply(new BigDecimal(factor)), currency);
	}
}

If you register the following class MoneyArbitraryProvider:

package my.own.provider;

public class MoneyArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Money.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		Arbitrary<BigDecimal> amount = Arbitraries.bigDecimals()
				  .between(BigDecimal.ZERO, new BigDecimal(1_000_000_000))
				  .ofScale(2);
		Arbitrary<String> currency = Arbitraries.of("EUR", "USD", "CHF");
		return Collections.singleton(Combinators.combine(amount, currency).as(Money::new));
	}
}

in file META-INF/services/net.jqwik.api.providers.ArbitraryProvider with such an entry:

my.own.provider.MoneyArbitraryProvider

The following property will run without further ado - regardless the class you put it in:

@Property
void moneyCanBeMultiplied(@ForAll Money money) {
    Money times2 = money.times(2);
    Assertions.assertThat(times2.getCurrency()).isEqualTo(money.getCurrency());
    Assertions.assertThat(times2.getAmount())
        .isEqualTo(money.getAmount().multiply(new BigDecimal(2)));
}

Arbitrary Providers for Parameterized Types

Providing arbitraries for generic types requires a little bit more effort since you have to create arbitraries for the “inner” types as well. Let’s have a look at the default provider for java.util.Optional<T>:

public class OptionalArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Optional.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		TypeUsage innerType = targetType.getTypeArguments().get(0);
		return subtypeProvider.apply(innerType).stream()
			.map(Arbitrary::optional)
			.collect(Collectors.toSet());
	}
}

Mind that provideFor returns a set of potential arbitraries. That’s necessary because the subtypeProvider might also deliver a choice of subtype arbitraries. Not too difficult, is it?

Arbitrary Provider Priority

When more than one provider is suitable for a given type, jqwik will randomly choose between all available options. That’s why you’ll have to take additional measures if you want to replace an already registered provider. The trick is to override a provider’s priority() method that returns 0 by default:

public class AlternativeStringArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isAssignableFrom(String.class);
	}

	@Override
	public int priority() {
		return 1;
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		return Collections.singleton(Arbitraries.just("A String"));
	}
}

If you register this class as arbitrary provider any @ForAll String will be resolved to "A String".

Create your own Annotations for Arbitrary Configuration

All you can do to constrain default parameter generation is adding another annotation to a parameter or its parameter types. What if the existing parameters do not suffice your needs? Is there a way to enhance the set of constraint annotations? Yes, there is!

The mechanism you can plug into is similar to what you do when providing your own default arbitrary providers. That means:

  1. Create an implementation of an interface, in this case ArbitraryConfigurator.
  2. Register the implementation using using Java’s java.util.ServiceLoader mechanism.

Arbitrary Configuration Example: @Odd

To demonstrate the idea let’s create an annotation @Odd which will constrain any integer generation to only generate odd numbers. First things first, so here’s the @Odd annotation together with the configurator implementation:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER, ElementType.TYPE_USE })
@Retention(RetentionPolicy.RUNTIME)
public @interface Odd {
}

public class OddConfigurator extends ArbitraryConfiguratorBase {
	public Arbitrary<Integer> configure(Arbitrary<Integer> arbitrary, Odd odd) {
		return arbitrary.filter(number -> Math.abs(number % 2) == 1);
	}
}

Mind that the implementation uses an abstract base class - instead of the interface itself - which simplifies implementation if you’re only interested in a single annotation.

If you now register the implementation, the following example will work:

@Property
boolean oddIntegersOnly(@ForAll @Odd int aNumber) {
    return Math.abs(aNumber % 2) == 1;
}

There are a few catches, though:

Domain and Domain Context

Until now you have seen two ways to specify which arbitraries will be created for a given parameter:

In many cases both approaches can be tedious to set up or require constant repetition of the same annotation value. There’s another way that allows you to collect a number of arbitrary providers (and also arbitrary configurators) in a single place, called a DomainContext and tell a property method or container to only use providers and configurators from those domain contexts that are explicitly stated in a @Domain(Class<? extends DomainContext>) annotation.

As for ways to implement domain context classes have a look at DomainContext and AbstractDomainContextBase.

Domain example: American Addresses

Let’s say that US postal addresses play a crucial role in the software that we’re developing. That’s why there are a couple of classes that represent important domain concepts: Street, State, City and Address. Since we have to generate instances of those classes for our properties, we collect all arbitrary provision code in AmericanAddresses. Now look at this example:

class AddressProperties {

	@Property
	@Domain(AmericanAddresses.class)
	void anAddressWithAStreetNumber(@ForAll Address anAddress, @ForAll int streetNumber) {
	}

	@Property
	@Domain(AmericanAddresses.class)
	void globalDomainNotPresent(@ForAll Address anAddress, @ForAll String anyString) {
	}

	@Property
	@Domain(DomainContext.Global.class)
	@Domain(AmericanAddresses.class)
	void globalDomainCanBeAdded(@ForAll Address anAddress, @ForAll String anyString) {
	}
}

The first two properties above will resolve their arbitraries solely through providers specified in AmericanAddresses, whereas the last one also uses the default (global) context. Since AmericanAddresses does not configure any arbitrary provider for String parameters, property method globalDomainNotPresent will fail with a CannotFindArbitraryException.

Generation from a Type’s Interface

Some domain classes are mostly data holders. They come with constructors or factory methods to create them and you might want to create different instances by “just” filling the constructors’ parameters with values that are themselves generated. Using the building blocks you’ve seen until now requires the use of Arbitrary.map() or even Combinators.combine(...).as(...) to invoke the relevant constructor(s) and/or factories yourself. There’s a simpler way, though…

Consider a simple Person class:

public class Person {

	private final String name;
	private final int age;

	public Person(String name, int age) {
		if (name == null || name.trim().isEmpty())
			throw new IllegalArgumentException();
		if (age < 0 || age > 130)
			throw new IllegalArgumentException();

		this.name = name;
		this.age = age;
	}

	@Override
	public String toString() {
		return String.format("%s (%d)", name, age);
	}
}

A first step to use arbitrarily generated Person objects without having to write a lot of jqwik-specific boiler plat code could look like that:

@Property
void aPersonsIsAlwaysValid(@ForAll @UseType Person aPerson) {
    Assertions.assertThat(aPerson.name).isNotBlank();
    Assertions.assertThat(aPerson.age).isBetween(0, 130);
}

Notice the annotation @UseType which tells jqwik to use the type information of Person to generate it. By default the framework will use all public constructors and all public, static factory methods in the class in order to generate instances. Whenever there’s an exception during generation they will be ignored; that way you’ll only get valid instances.

There are quite a few ways usage and configuration options. Have a look at the complete example and check the following api entry points:

Generation of Edge Cases

It’s well-known that many programming bugs and specification gaps happen at the border of allowed value ranges. For example, in the domain of integer numbers the minimum (Integer.MIN_VALUE) and maximum (Integer.MAX_VALUE) belong in the set of those edge cases. Many people use the term a bit more loosely and also include other special values that tend to have a higher chance of revealing implementation problems, like 0 for numbers or an empty string.

jqwik has special treatment for edge cases. Most base type arbitraries come with their set of edge cases. You can find out about edge cases by asking an arbitrary about it. Run the following example

@Example
void printEdgeCases() {
    System.out.println(Arbitraries.integers().edgeCases());
    System.out.println(Arbitraries.strings().withCharRange('a', 'z').edgeCases());
    System.out.println(Arbitraries.floats().list().edgeCases());
}

and you will see this output:

EdgeCases[-2, -1, 0, 2, 1, -2147483648, 2147483647]
EdgeCases["a", "z", ""]
EdgeCases[[], [0.0], [1.0], [-1.0], [0.01], [-0.01], [-3.4028235E38], [3.4028235E38]]

You may notice that edge cases are not just hard-coded values but also make use of underlying arbitraries’ edge cases to arrive at new ones. That’s why a list of floats arbitrary has single element lists of floats as edge cases. Edge cases are also being combined and permuted when Combinators are used. Also, most methods from Arbitrary - like map(), filter() and flatMap() - provide sensible edge cases behaviour. Thus, your self-made domain-specific arbitraries get edge cases automatically.

jqwik makes use of edge cases in two ways:

  1. Whenever an arbitrary is asked to produce a value it will mix-in edge cases from time to time.
  2. By default jqwik will mix the combination of permutations of edge cases of a property’s parameters with purely randomized generation of parameters. You can even try all edge case combinations first as the next property shows.
@Property(edgeCases = EdgeCasesMode.FIRST)
void combinedEdgeCasesOfTwoParameters(
    @ForAll List<Integer> intList,
    @ForAll @IntRange(min = -100, max = 0) int anInt
) {
    String parameters = String.format("%s, %s", intList, anInt);
    System.out.println(parameters);
}

Run it and have a look at the output.

Configuring Edge Case Injection

How jqwik handles edge cases generation can be controlled with an annotation property and a configuration parameter.

To switch it off for a single property, use:

@Property(edgeCases = EdgeCasesMode.NONE)
void combinedEdgeCasesOfTwoParameters(
    @ForAll List<Integer> intList,
    @ForAll @IntRange(min = -100, max = 0) int anInt
) {
    // whatever you do   
}

Exhaustive Generation

Sometimes it is possible to run a property method with all possible value combinations. Consider the following example:

@Property
boolean allSquaresOnChessBoardExist(
    @ForAll @CharRange(from = 'a', to = 'h') char column,
    @ForAll @CharRange(from = '1', to = '8') char row
) {
    return new ChessBoard().square(column, row).isOnBoard();
}

The property is supposed to check that all valid squares in chess are present on a new chess board. If jqwik generated the values for column and row randomly, 1000 tries might or might not produce all 64 different combinations. Why not change strategies in cases like that and just iterate through all possible values?

This is exactly what jqwik will do:

Exhaustive generation is considered for:

Data-Driven Properties

In addition to the usual randomized generation of property parameters you have also the possibility to feed a property with preconceived or deterministically generated parameter sets. Why would you want to do that? One reason might be that you are aware of some problematic test cases but they are rare enough that jqwik’s randomization strategies don’t generate them (often enough). Another reason could be that you’d like to feed some properties with prerecorded data - maybe even from production. And last but not least there’s a chance that you want to check for a concrete result given a set of input parameters.

Feeding data into a property is quite simple:

@Data
Iterable<Tuple2<Integer, String>> fizzBuzzExamples() {
    return Table.of(
        Tuple.of(1, "1"),
        Tuple.of(3, "Fizz"),
        Tuple.of(5, "Buzz"),
        Tuple.of(15, "FizzBuzz")
    );
}

@Property
@FromData("fizzBuzzExamples")
void fizzBuzzWorks(@ForAll int index, @ForAll String result) {
    Assertions.assertThat(fizzBuzz(index)).isEqualTo(result);
}

All you have to do is annotate the property method with @FromData("dataProviderReference"). The method you reference must be annotated with @Data and return an object of type Iterable<? extends Tuple>. The Table class is just a convenient way to create such an object, but you can return any collection or create an implementation of your own.

Keep in mind that the Tuple subtype you choose must conform to the number of @ForAll parameters in your property method, e.g. Tuple.Tuple3 for a method with three parameters. Otherwise jqwik will fail the property and tell you that the provided data is inconsistent with the method’s parameters.

Data points are fed to the property in their provided order. The tries parameter of @Property will constrain the maximum data points being tried. Unlike parameterized tests in JUnit4 or Jupiter, jqwik will report only the first falsified data point. Thus, fixing the first failure might lead to another falsified data point later on. There is also no shrinking being done for data-driven properties since jqwik has no information about the constraints under which the external data was conceived or generated.

Rerunning Falsified Properties

When you rerun properties after they failed, they will - by default - use the previous random seed so that the next run will generate the exact same parameter data and thereby expose the same failing behaviour. This simplifies debugging and regression testing since it makes a property’s falsification stick until the problem has been fixed.

If you want to, you can change this behaviour for a given property like this:

@Property(afterFailure = AfterFailureMode.RANDOM_SEED)
void myProperty() { ... }

The afterFailure property can have one of four values:

You can also determine the default behaviour of all properties by setting the defaultAfterFailure property in the configuration file to one of those enum values.

jqwik Configuration

jqwik will look for a file jqwik.properties in your classpath in which you can configure a few basic parameters:

database = .jqwik-database          # The database file in which to store data of previous runs.
                                    # Set to empty to fully disable test run recording.
defaultTries = 1000                 # The default number of tries for each property
defaultMaxDiscardRatio = 5          # The default ratio before assumption misses make a property fail
useJunitPlatformReporter = false    # Set to true if you want to use platform reporting
defaultAfterFailure = PREVIOUS_SEED # Set default behaviour for falsified properties:
                                    # PREVIOUS_SEED, SAMPLE_ONLY or SAMPLE_FIRST
reportOnlyFailures = false          # Set to true if only falsified properties should be reported
defaultGeneration = AUTO            # Set default behaviour for generation:
                                    # AUTO, RANDOMIZED, or EXHAUSTIVE
defaultEdgeCases = MIXIN            # Set default behaviour for edge cases generation:
                                    # FIRST, MIXIN, or NONE
defaultShrinking = BOUNDED          # Set default shrinking behaviour:
                                    # BOUNDED, FULL, or OFF
boundedShrinkingSeconds = 10        # The maximum number of seconds to shrink if
                                    # shrinking behaviour is set to BOUNDED

Advanced Topics

Implement your own Arbitraries and Generators

Looking at jqwik’s most prominent interfaces – Arbitrary and RandomGenerator – you might think that rolling your own implementations is a reasonable thing to do. I’d like to tell you that it never is, but I’ve learned that “never” is a word you should never use. There’s just too many things to consider when implementing a new type of Arbitrary to make it work smoothly with the rest of the framework.

Therefore, use the innumerable features to combine existing arbitraries into your special one. If you cannot figure out how to create an arbitrary with the desired behaviour either ask on stack overflow or open a Github issue.

Lifecycle Hooks

Similar to Jupiter’s Extension Model jqwik provides a means to extend and change the way how properties and containers are being configured, run and reported on. The API – interfaces, classes and annotations – for accessing those lifecycle hooks lives in the package net.jqwik.api.lifecycle and is – as of this release – still in the API evolution status EXPERIMENTAL: Some parts of it will probably change without notice in later versions.

Principles of Lifecycle Hooks

There are a few fundamental principles that determine and constrain the lifecycle hook API:

  1. There are several types of lifecycle hooks, each of which is an interface that extends net.jqwik.api.lifecycle.LifecycleHook.
  2. A concrete lifecycle hook is an implementation of one or more lifecycle hook interfaces.
  3. You can add a concrete lifecycle hook to a container class or a property method with the annotation @AddLifecycleHook. By default, a lifecycle hook is only added to the annotated element, not to its children. However, you can override this behaviour by either:
    • Override LifecycleHook.propagateTo()
    • Use the annotation attribute @AddLifecycleHook.propagateTo()
  4. To add a global lifecycle use Java’s java.util.ServiceLoader mechanism and add the concrete lifecylcle hook class to file META-INF/services/net.jqwik.api.lifecycle.LifecycleHook. Do not forget to override LifecycleHook.propagateTo() if the global hook should be applied to all test elements.
  5. In a single test run there will only be a single instance of each concrete lifecycle hook implementation. That’s why you have to use jqwik’s lifecycle storage mechanism if shared state across several calls to lifecycle methods is necessary.
  6. Since all instances of lifecycle hooks are created before the whole test run is started, you cannot use non-static inner classes of test containers to implement lifecycle interfaces.
  7. If relevant, the order in which hook methods are being applied is determined by dedicated methods in the hook interface, e.g. BeforeContainerHook.beforeContainerProximity().

Mind that much of what you can do with hooks can also be done using the simpler mechanisms of annotated lifecycle methods or a property lifecycle class. You usually start to consider using lifecycle hooks when you want to reuse generic behaviour in many places or even across projects.

Lifecycle Hook Types

All lifecycle hook interfaces extend net.jqwik.api.lifecycle.LifecycleHook which has two methods that may be overridden:

jqwik currently supports eight types of lifecycle hooks:

Lifecycle Execution Hooks

With these hooks you can determine if a test element will be run at all, and what potential actions should be done before or after running it.

SkipExecutionHook

Implement SkipExecutionHook to filter out a test container or property method depending on some runtime condition.

Given this hook implementation:

public class OnMacOnly implements SkipExecutionHook {
    @Override
    public SkipResult shouldBeSkipped(final LifecycleContext context) {
        if (System.getProperty("os.name").equals("Mac OS X")) {
            return SkipResult.doNotSkip();
        }
        return SkipResult.skip("Only on Mac");
    }
}

The following property will only run on a Mac:

@Property
@AddLifecycleHook(OnMacOnly.class)
void macSpecificProperty(@ForAll int anInt) {
}
BeforeContainerHook

Implement BeforeContainerHook for a hook that’s supposed to do some work exactly once before any of its property methods and child containers will be run. This is typically used to set up a resource to share among all properties within this container.

AfterContainerHook

Implement AfterContainerHook for a hook that’s supposed to do some work exactly once after all of its property methods and child containers have been run. This is typically used to tear down a resource that has been shared among all properties within this container.

AroundContainerHook

AroundContainerHook is a convenience interface to implement both BeforeContainerHook and AfterContainerHook in one go. This is typically used to set up and tear down a resource that is intended to be shared across all the container’s children.

Here’s an example that shows how to start and stop an external server once for all properties of a test container:

@AddLifecycleHook(ExternalServerResource.class)
class AroundContainerHookExamples {
    @Example
    void example1() {
        System.out.println("Running example 1");
    }
    @Example
    void example2() {
        System.out.println("Running example 2");
    }
}

class ExternalServerResource implements AroundContainerHook {
    @Override
    public void beforeContainer(final ContainerLifecycleContext context) {
        System.out.println("Starting server...");
    }
  
    @Override
    public void afterContainer(final ContainerLifecycleContext context) {
        System.out.println("Stopping server...");
    }
}

Running this example should output

Starting server...

Running example 1

Running example 2

Stopping server...

If you wanted to do something before and/or after the whole jqwik test run, using a container hook and registering it globally is probably the easiest way.

AroundPropertyHook

AroundPropertyHook comes in handy if you need to define behaviour that should “wrap” the execution of a property, i.e., do something directly before or after running a property - or both. Since you have access to an object that describes the final result of a property you can also change the result, e.g. make a failed property successful or vice versa.

Here is a hook implementation that will measure the time spent on running a property and publish the result using a Reporter:

@Property(tries = 100)
@AddLifecycleHook(MeasureTime.class)
void measureTimeSpent(@ForAll Random random) throws InterruptedException {
    Thread.sleep(random.nextInt(50));
}

class MeasureTime implements AroundPropertyHook {
    @Override
    public PropertyExecutionResult aroundProperty(PropertyLifecycleContext context, PropertyExecutor property) {
        long before = System.currentTimeMillis();
        PropertyExecutionResult executionResult = property.execute();
        long after = System.currentTimeMillis();
        context.reporter().publish("time", String.format("%d ms", after - before));
        return executionResult;
    }
}

The additional output from reporting is concise:

timestamp = ..., time = 2804 ms
AroundTryHook

Wrapping the execution of a single try can be achieved by implementing AroundTryHook. This hook can be used for a lot of things. An incomplete list:

The following example shows how to fail if a single try will take longer than 100 ms:

@Property(tries = 10)
@AddLifecycleHook(FailIfTooSlow.class)
void sleepingProperty(@ForAll Random random) throws InterruptedException {
    Thread.sleep(random.nextInt(101));
}

class FailIfTooSlow implements AroundTryHook {
    @Override
    public TryExecutionResult aroundTry(
        final TryLifecycleContext context,
        final TryExecutor aTry,
        final List<Object> parameters
    ) {
        long before = System.currentTimeMillis();
        TryExecutionResult result = aTry.execute(parameters);
        long after = System.currentTimeMillis();
        long time = after - before;
        if (time >= 100) {
            String message = String.format("%s was too slow: %s ms", context.label(), time);
            return TryExecutionResult.falsified(new AssertionFailedError(message));
        }
        return result;
    }
}

Since the sleep time is chosen randomly the property will fail from time to time with the following error:

org.opentest4j.AssertionFailedError: sleepingProperty was too slow: 100 ms

Other Hooks

ResolveParameterHook

Besides the well-known @ForAll-parameters, property methods and annotated lifecycle methods can take other parameters as well. These can be injected by concrete implementations of ResolveParameterHook.

Consider this stateful Calculator:

public class Calculator {
    private int result = 0;
  
    public int result() {
        return result;
    }
  
    public void plus(int addend) {
        result += addend;
    }
}

When going to check its behaviour with properties you’ll need a fresh calculator instance in each try. This can be achieved by adding a resolver hook that creates a freshly instantiated calculator per try.

@AddLifecycleHook(CalculatorResolver.class)
class CalculatorProperties {
    @Property
    void addingANumberTwice(@ForAll int aNumber, Calculator calculator) {
        calculator.plus(aNumber);
        calculator.plus(aNumber);
        Assertions.assertThat(calculator.result()).isEqualTo(aNumber * 2);
    }
}

class CalculatorResolver implements ResolveParameterHook {
    @Override
    public Optional<ParameterSupplier> resolve(
        final ParameterResolutionContext parameterContext,
        final LifecycleContext lifecycleContext
    ) {
        return Optional.of(optionalTry -> new Calculator());
    }
    @Override
    public PropagationMode propagateTo() {
        // Allow annotation on container level
        return PropagationMode.ALL_DESCENDANTS;
    }
}

There are a few constraints regarding parameter resolution of which you should be aware:

RegistrarHook

Use RegistrarHook if you need to apply several hook implementations that implement the desired behaviour together but cannot be implemented in a single class. For example, more than one implementation of the same hook type is needed, but those implementations have a different proximity or require a different propagation mode.

This is really advanced stuff, the mechanism of which will probably evolve or change in the future. If you really really want to see an example, look at JqwikSpringExtension

Lifecycle Storage

As described above one of the fundamental principles is that there will be only a single instance of any lifecycle hook implementation during runtime. Since – depending on configuration and previous rung – containers and properties are not run in a strict sequential order this guarantee comes with a drawback: You cannot use a hook instance’s member variables to hold state that should be shared across all tries of a property or across all properties of a container or across different lifecycle phases of a single try. That’s when lifecycle storage management enters the stage in the form of type net.jqwik.api.lifecycle.Store.

A Store object…

You create a store like this:

Store<MyObject> myObjectStore = Store.create("myObjectStore", Lifespan.PROPERTY, () -> new MyObject());

And you retrieve a store similarly:

Store<MyObject> myObjectStore = Store.get("myObjectStore");

A store with the same identifier can only be created once, that’s why there is also a convenience method for creating or retrieving it:

Store<MyObject> myObjectStore = Store.getOrCreate("myObjectStore", Lifespan.PROPERTY, () -> new MyObject());

You now have the choice to use or update the shared state:

Store<MyObject> myObjectStore = ...;

myObjectStore.get().doSomethingWithMyObject();
myObjectStore.update(old -> {
    old.changeState();
    return old;
});

Let’s look at an example…

TemporaryFileHook

The following hook implementation gives you the capability to access one (and only one) temporary file per try using parameter resolution:

class TemporaryFileHook implements ResolveParameterHook {

    public static final Tuple.Tuple2 STORE_IDENTIFIER = Tuple.of(TemporaryFileHook.class, "temporary files");
  
    @Override
    public Optional<ParameterSupplier> resolve(ParameterResolutionContext parameterContext, LifecycleContext lifecycleContext) {
        if (parameterContext.typeUsage().isOfType(File.class)) {
            return Optional.of(ignoreTry -> getTemporaryFileForTry());
        }
        return Optional.empty();
    }
  
    private File getTemporaryFileForTry() {
        Store<File> tempFileStore = Store.getOrCreate(STORE_IDENTIFIER, Lifespan.TRY, this::createTempFile);
        tempFileStore.onClose(file -> file.delete());
        return tempFileStore.get();
    }
  
    private File createTempFile() {
        try {
            return File.createTempFile("temp", ".txt");
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}

There are a few interesting things going on:

With this information you can probably figure out how the following test container works – especially why the assertion in @AfterTry-method assertFileNotEmpty() succeeds.

@AddLifecycleHook(value = TemporaryFileHook.class, propagateTo = PropagationMode.ALL_DESCENDANTS)
class TemporaryFilesExample {
    @Property(tries = 10)
    void canWriteToFile(File anyFile, @ForAll @AlphaChars @StringLength(min = 1) String fileContents) throws Exception {
        assertThat(anyFile).isEmpty();
        writeToFile(anyFile, fileContents);
        assertThat(anyFile).isNotEmpty();
    }
  
    @AfterTry
    void assertFileNotEmpty(File anyFile) {
        assertThat(anyFile).isNotEmpty();
    }
  
    private void writeToFile(File anyFile, String contents) throws IOException {
        BufferedWriter writer = new BufferedWriter(new FileWriter(anyFile));
        writer.write(contents);
        writer.close();
    }
}

API Evolution

In agreement with the JUnit 5 platform jqwik uses the @API Guardian project to communicate version and status of all parts of its API. The different types of status are:

-STABLE: Intended for features that will not be changed in a backwards-incompatible way in the current major version (1.*).

-MAINTAINED: Intended for features that will not be changed in a backwards-incompatible way for at least the current minor release of the current major version. If scheduled for removal, it will be demoted to DEPRECATED first.

-EXPERIMENTAL: Intended for new, experimental features where we are looking for feedback. Use this element with caution; it might be promoted to MAINTAINED or STABLE in the future, but might also be removed without prior notice, even in a patch.

-DEPRECATED: Should no longer be used; might disappear in the next minor release.

-INTERNAL: Must not be used by any code other than jqwik itself. Might be removed without prior notice.

Since annotation @API has runtime retention you find the actual API status in an element’s source code, its Javadoc but also through reflection. If a certain element, e.g. a method, is not annotated itself, then it carries the status of its containing class.

Release Notes

Read this version’s release notes.