Sponsor the Team View on GitHub

jqwik Latest Release: 1.9.2

Property-Based Testing in Java

The jqwik User Guide 1.9.1

Table of Contents Detailed Table of Contents

Detailed Table of Contents

How to Use

jqwik is an alternative test engine for the JUnit 5 platform. That means that you can use it either stand-alone or combine it with any other JUnit 5 engine, e.g. Jupiter (the standard engine) or Vintage (aka JUnit 4). All you have to do is add all needed engines to your testImplementation dependencies as shown in the gradle file below.

The latest release of jqwik is deployed to Maven Central. Snapshot releases are created on a regular basis and can be fetched from jqwik’s snapshot repository.

Required Version of JUnit Platform

The minimum required version of the JUnit platform is 1.11.1.

Gradle

Since version 4.6, Gradle has built-in support for the JUnit platform. Set up is rather simple; here are the relevant parts of a project’s build.gradle file:

repositories {
    ...
    mavenCentral()

    # For snapshot releases only:
    maven { url 'https://s01.oss.sonatype.org/content/repositories/snapshots' }

}

ext.junitJupiterVersion = '5.11.1'
ext.jqwikVersion = '1.9.1'

compileTestJava {
    // To enable argument names in reporting and debugging
	options.compilerArgs += '-parameters'
}

test {
	useJUnitPlatform {
		includeEngines 'jqwik'
        
        // Or include several Junit engines if you use them
        // includeEngines 'jqwik', 'junit-jupiter', 'junit-vintage'

		// includeTags 'fast', 'medium'
		// excludeTags 'slow'
	}

	include '**/*Properties.class'
	include '**/*Test.class'
	include '**/*Tests.class'
}

dependencies {
    ...

    // aggregate jqwik dependency
    testImplementation "net.jqwik:jqwik:${jqwikVersion}"

    // Add if you also want to use the Jupiter engine or Assertions from it
    testImplementation "org.junit.jupiter:junit-jupiter:${junitJupiterVersion}"

    // Add any other test library you need...
    testImplementation "org.assertj:assertj-core:3.23.1"

    // Optional but recommended to get annotation related API warnings
	compileOnly("org.jetbrains:annotations:23.0.0")

}

With version 1.0.0 net.jqwik:jqwik has become an aggregating module to simplify jqwik integration for standard users. If you want to be more explicit about the real dependencies you can replace this dependency with

    testImplementation "net.jqwik:jqwik-api:${jqwikVersion}"
    testImplementation "net.jqwik:jqwik-web:${jqwikVersion}"
    testImplementation "net.jqwik:jqwik-time:${jqwikVersion}"
    testRuntime "net.jqwik:jqwik-engine:${jqwikVersion}"

In jqwik’s samples repository you can find a rather minimal starter example for jqwik with Gradle.

See the Gradle section in JUnit 5’s user guide for more details on how to configure Gradle for the JUnit 5 platform. There is also a comprehensive list of options for Gradle’s test task.

Seeing jqwik Reporting in Gradle Output

Since Gradle does not yet support JUnit platform reporting – see this Github issue – jqwik has switched to do its own reporting by default. This behaviour can be configured through parameter jqwik.reporting.usejunitplatform (default: false).

If you want to see jqwik’s reports in the output use Gradle’s command line option --info:

> gradle clean test --info
...
mypackage.MyClassProperties > myPropertyMethod STANDARD_OUT
    timestamp = 2019-02-28T18:01:14.302, MyClassProperties:myPropertyMethod = 
                                  |-----------------------jqwik-----------------------
    tries = 1000                  | # of calls to property
    checks = 1000                 | # of not rejected calls
    generation = RANDOMIZED       | parameters are randomly generated
    after-failure = PREVIOUS_SEED | use the previous seed
    when-fixed-seed = ALLOW       | fixing the random seed is allowed
    edge-cases#mode = MIXIN       | edge cases are generated first
    edge-cases#total = 0          | # of all combined edge cases
    edge-cases#tried = 0          | # of edge cases tried in current run
    seed = 1685744359484719817    | random seed to reproduce generated values

Maven

Starting with version 2.22.0, Maven Surefire and Maven Failsafe provide native support for executing tests on the JUnit Platform and thus for running jqwik properties. The configuration of Maven Surefire is described in the Maven section of JUnit 5’s user guide.

Additionally you have to add the following dependency to your pom.xml file:

<dependencies>
    ...
    <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik</artifactId>
        <version>1.9.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

In jqwik’s samples repository you can find a rather minimal starter example for jqwik with Maven.

Snapshot Releases

Snapshot releases are available through jqwik’s snapshot repositories.

Adding

https://s01.oss.sonatype.org/content/repositories/snapshots

as a maven repository will allow you to use jqwik’s snapshot release which contains all the latest features.

Project without Build Tool

I’ve never tried it but using jqwik without gradle or some other tool to manage dependencies should also work. You will have to add at least the following jars to your classpath:

Optional jars are:

Writing Properties

Creating a Property

Properties are the core concept of property-based testing.

You create a Property by annotating a public, protected or package-scoped method with @Property. In contrast to examples a property method is supposed to have one or more parameters, all of which must be annotated with @ForAll.

At test runtime the exact parameter values of the property method will be filled in by jqwik.

Just like an example test a property method has to

If not specified differently, jqwik will run 1000 tries, i.e. a 1000 different sets of parameter values and execute the property method with each of those parameter sets. The first failed execution will stop value generation and be reported as failure - usually followed by an attempt to shrink the falsified parameter set.

Here are two properties whose failures might surprise you:

import net.jqwik.api.*;
import org.assertj.core.api.*;

class PropertyBasedTests {

	@Property
	boolean absoluteValueOfAllNumbersIsPositive(@ForAll int anInteger) {
		return Math.abs(anInteger) >= 0;
	}

	@Property
	void lengthOfConcatenatedStringIsGreaterThanLengthOfEach(
		@ForAll String string1, @ForAll String string2
	) {
		String conc = string1 + string2;
		Assertions.assertThat(conc.length()).isGreaterThan(string1.length());
		Assertions.assertThat(conc.length()).isGreaterThan(string2.length());
	}
}

Mind that only parameters that are annotated with ‘@ForAll’ are considered for value generation. Other kinds of parameters can be injected through the resolve parameter hook.

Failure Reporting

If a property fails then jqwik’s reporting is more thorough:

In the case of lengthOfConcatenatedStringIsGreaterThanLengthOfEach from above the report looks like that:

PropertyBasedTests:lengthOfConcatenatedStringIsGreaterThanLengthOfEach = 
  java.lang.AssertionError: 
    Expecting:
     <0>
    to be greater than:
     <0> 
                              |-----------------------jqwik-----------------------
tries = 16                    | # of calls to property
checks = 16                   | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = SAMPLE_FIRST  | try previously failed sample, then previous seed
when-fixed-seed = ALLOW       | fixing the random seed is allowed
edge-cases#mode = MIXIN       | edge cases are mixed in
edge-cases#total = 4          | # of all combined edge cases
edge-cases#tried = 0          | # of edge cases tried in current run
seed = -2370223836245802816   | random seed to reproduce generated values

Shrunk Sample (<n> steps)
-------------------------
  string1: ""
  string2: ""

Original Sample
---------------
  string1: "乮��깼뷼檹瀶�������የ뷯����ঘ꼝���焗봢牠"
  string2: ""

  Original Error
  --------------
  java.lang.AssertionError: 
    Expecting:
     <29>
    to be greater than:
     <29>

The source code names of property method parameters can only be reported when compiler argument -parameters is used. jqwik goes for structured reporting with collections, arrays and maps. If you want to provide nice reporting for your own domain classes you can either

Additional Reporting Options

You can switch on additional reporting aspects by adding a @Report(Reporting[]) annotation to a property method.

The following reporting aspects are available:

Unlike sample reporting these reports will show the freshly generated parameters, i.e. potential changes to mutable objects during property execution cannot be seen here.

Platform Reporting with Reporter Object

If you want to provide additional information during a test or a property using System.out.println() is a common choice. The JUnit platform, however, provides a better mechanism to publish additional information in the form of key-value pairs. Those pairs will not only printed to stdout but are also available to downstream tools like test report generators in continue integration.

You can hook into this reporting mechanism through jqwik’s Reporter object. This object is available in lifecycle hooks but you can also have it injected as a parameter into your test method:

@Example
void reportInCode(Reporter reporter, @ForAll List<@AlphaChars String> aList) {
	reporter.publishReport("listOfStrings", aList);
	reporter.publishValue("birthday", LocalDate.of(1969, 1, 20).toString());
}

net.jqwik.api.Reporter has different publishing methods. Those with report in their name use jqwik’s reporting mechanism and formats described above.

Adding Footnotes to Failure Reports

By using the platform reporting mechanism you can publish additional key-value pairs for each and every run of a property method. In many cases, however, you only want some clarifying information for failing property tries. Footnotes provide this capability:

@EnableFootnotes
public class FootnotesExamples {
	@Property
	void differenceShouldBeBelow42(@ForAll int number1, @ForAll int number2, Footnotes footnotes) {
		int difference = Math.abs(number1 - number2);
		footnotes.addFootnote(Integer.toString(difference));
		Assertions.assertThat(difference).isLessThan(42);
	}
}

Unlike standard reporting, the footnotes feature must be explicitly enabled through the annotation @EnableFootnotes, which can be added to container classes or individual property methods. Now you can add a parameter of type net.jqwik.api.footnotes.Footnotes to a property method or a lifecycle method annotated with either @BeforeTry or @AfterTry. The footnote string will then be part of the sample reporting:

Shrunk Sample (5 steps)
-----------------------
  number1: 0
  number2: 42
  footnotes: net.jqwik.api.footnotes.Footnotes[differenceShouldBeBelow42]

  #1 42

Original Sample
---------------
  number1: 399308
  number2: -14
  footnotes: net.jqwik.api.footnotes.Footnotes[differenceShouldBeBelow42]

  #1 399322

For footnotes that require significant computation you can also use Footnotes.addAfterFailure(Supplier<String> footnoteSupplier). Those suppliers will only be evaluated if the property fails, and then as early as possible. Mind that this evaluation can still happen quite often during shrinking.

Optional @Property Attributes

The @Property annotation has a few optional values:

The effective values for tries, seed, after-failure mode, generation mode edge-cases mode and edge cases numbers are reported after each run property:

tries = 10 
checks = 10 
generation = EXHAUSTIVE
after-failure = SAMPLE_FIRST
when-fixed-seed = ALLOW
edge-cases#mode = MIXIN 
edge-cases#total = 2 
edge-cases#tried = 2 
seed = 42859154278924201

Setting Defaults for @Property Attributes

If you want to set the defaults for all property methods in a container class (and all the groups in it) you can use annotation @PropertyDefaults.

In the following example both properties are tried 10 times. Shrinking mode is set for all but is overridden in the second property.

@PropertyDefaults(tries = 10, shrinking = ShrinkingMode.FULL)
class PropertyDefaultsExamples {

	@Property
	void aLongRunningProperty(@ForAll String aString) {}

	@Property(shrinking = ShrinkingMode.OFF)
	void anotherLongRunningProperty(@ForAll String aString) {}
}

Thus, the order in which a property method’s attributes are determined is:

  1. Use jqwik’s built-in defaults,
  2. which can be overridden in the configuration file,
  3. which can be changed in a container class’ @PropertyDefaults annotation,
  4. which override @PropertyDefaults attributes in a container’s superclass or implemented interfaces,
  5. which can be overridden by a method’s @Property annotation attributes.

Creating an Example-based Test

jqwik also supports example-based testing. In order to write an example test annotate a public, protected or package-scoped method with @Example. Example-based tests work just like plain JUnit-style test cases.

A test case method must

Here is a test class with two example-based tests:

import static org.assertj.core.api.Assertions.*;

import net.jqwik.api.*;
import org.assertj.core.data.*;

class ExampleBasedTests {
	
	@Example
	void squareRootOf16is4() { 
		assertThat(Math.sqrt(16)).isCloseTo(4.0, Offset.offset(0.01));
	}

	@Example
	boolean add1plu3is4() {
		return (1 + 3) == 4;
	}
}

Internally jqwik treats examples as properties with the number of tries hardcoded to 1. Thus, everything that works for property methods also works for example methods – including random generation of parameters annotated with @ForAll.

Assertions

jqwik does not come with any assertions, so you have to use one of the third-party assertion libraries, e.g. Hamcrest or AssertJ.

If you have Jupiter in your test dependencies anyway, you can also use the static methods in org.junit.jupiter.api.Assertions.

Lifecycle

To understand the lifecycle it is important to know that the tree of test elements consists of two main types of elements:

So a typical tree might look like:

Jqwik Engine
    class MyFooTests
        @Property fooProperty1()
        @Property fooProperty2()
        @Example fooExample()
    class MyBarTests
        @Property barProperty()
        @Group class Group1 
            @Property group1Property()
        @Group class Group2 
            @Example group2Example()

Mind that packages do not show up as in-between containers!

When running your whole test suite there are additional things happening:

jqwik gives you more than one way to hook into the lifecycle of containers, properties and tries.

Simple Property Lifecycle

If you need nothing but some initialization and cleanup of the container instance per property or example:

import net.jqwik.api.*;

class SimpleLifecycleTests implements AutoCloseable {

	SimpleLifecycleTests() {
		System.out.println("Before each");
	}

	@Example void anExample() {
		System.out.println("anExample");
	}

	@Property(tries = 5)
	void aProperty(@ForAll String aString) {
		System.out.println("aProperty: " + aString);
	}

	@Override
	public void close() throws Exception {
		System.out.println("After each");
	}
}

In this example both the constructor and close() will be called twice times: Once for anExample() and once for aProperty(...). However, all five calls to aProperty(..) will share the same instance of SimpleLifecycleTests.

Annotated Lifecycle Methods

The other way to influence all elements of a test run is through annotated lifecycle methods, which you might already know from JUnit 4 and 5. jqwik currently has eight annotations:

Given the following container class:

class FullLifecycleExamples {

	@BeforeContainer
	static void beforeContainer() {
		System.out.println("before container");
	}

	@AfterContainer
	static void afterContainer() {
		System.out.println("after container");
	}

	@BeforeProperty
	void beforeProperty() {
		System.out.println("before property");
	}

	@AfterProperty
	void afterProperty() {
		System.out.println("after property");
	}

	@BeforeTry
	void beforeTry() {
		System.out.println("before try");
	}

	@AfterTry
	void afterTry() {
		System.out.println("after try");
	}

	@Property(tries = 3)
	void property(@ForAll @IntRange(min = -5, max = 5) int anInt) {
		System.out.println("property: " + anInt);
	}
}

Running this test container should produce something like the following output (maybe with your test report in-between):

before container

before property
before try
property: 3
after try
before try
property: 1
after try
before try
property: 4
after try
after property

after container

All those lifecycle methods are being run through jqwik’s mechanism for writing lifecycle hooks under the hood.

Annotated Lifecycle Variables

One of the lifecycle annotations from above has an additional meaning: @BeforeTry: It can also be used on a test container class’ member variable to make sure that it will be reset to its initial value - the one it had before the first try - for each try:

class BeforeTryMemberExample {

	@BeforeTry
	int theAnswer = 42;

	@Property
	void theAnswerIsAlways42(@ForAll int addend) {
		Assertions.assertThat(theAnswer).isEqualTo(42);
		theAnswer += addend;
	}
}

Single Property Lifecycle

All lifecycle methods described in the previous section apply to all property methods of a container class. In rare cases, however, you may feel the need to hook into the lifecycle of a single property, for example when you expect a property to fail.

Here is one example that checks that a property will fail with an AssertionError and succeed in that case:

@Property
@PerProperty(SucceedIfThrowsAssertionError.class)
void expectToFail(@ForAll int aNumber) {
    Assertions.assertThat(aNumber).isNotEqualTo(1);
}

private class SucceedIfThrowsAssertionError implements PerProperty.Lifecycle {
    @Override
    public PropertyExecutionResult onFailure(PropertyExecutionResult propertyExecutionResult) {
        if (propertyExecutionResult.throwable().isPresent() &&
                propertyExecutionResult.throwable().get() instanceof AssertionError) {
            return propertyExecutionResult.mapToSuccessful();
        }
        return propertyExecutionResult;
    }
}

Have a look at PerProperty.Lifecycle to find out which aspects of a property’s lifecycle you can control.

Grouping Tests

Within a containing test class you can group other containers by embedding another non-static and non-private inner class and annotating it with @Group. Grouping examples and properties is a means to improve the organization and maintainability of your tests.

Groups can be nested, which makes their lifecycles also nested. That means that the lifecycle of a test class is also applied to inner groups of that container. Have a look at this example:

import net.jqwik.api.*;

class TestsWithGroups {

	@Property
	void outer(@ForAll String aString) {
	}

	@Group
	class Group1 {
		@Property
		void group1Property(@ForAll String aString) {
		}

		@Group
		class Subgroup {
			@Property
			void subgroupProperty(@ForAll String aString) {
			}
		}
	}

	@Group
	class Group2 {
		@Property
		void group2Property(@ForAll String aString) {
		}
	}
}

Naming and Labeling Tests

Using Java-style camel case naming for your test container classes and property methods will sometimes lead to hard to read display names in your test reports and your IDE. Therefore, jqwik provides a simple way to insert spaces into the displayed name of your test container or property: just add underscores (_), which are valid Java identifier characters. Each underscore will be replaced by a space for display purposes.

If you want to tweak display names even more, test container classes, groups, example methods and property methods can be labeled using the annotation @Label("a label"). This label will be used to display the element in test reports or within the IDE. In the following example, every test relevant element has been labeled:

@Label("Naming")
class NamingExamples {

	@Property
	@Label("a property")
	void aPropertyWithALabel() { }

	@Group
	@Label("A Group")
	class GroupWithLabel {
		@Example
		@Label("an example with äöüÄÖÜ")
		void anExampleWithALabel() { }
	}

    @Group
    class Group_with_spaces {
        @Example
        void example_with_spaces() { }
    }

}

Labels can consist of any characters and don’t have to be unique - but you probably want them to be unique within their container.

Tagging Tests

Test container classes, groups, example methods and property methods can be tagged using the annotation @Tag("a-tag"). You can have many tags on the same element.

Those tag can be used to filter the set of tests run by the IDE or the build tool. Tags are handed down from container (class or group) to its children (test methods or groups).

Have a look at the following example. Including the tag integration-test will include all tests of the class.

@Tag("integration-test")
class TaggingExamples {

	@Property
	@Tag("fast")
	void aFastProperty() { }

	@Example
	@Tag("slow") @Tag("involved")
	void aSlowTest() { }
}

Tags must follow certain rules as described here. Note that the @Tag annotation you’ll have to use with jqwik is net.jqwik.api.Tag rather than org.junit.jupiter.api.Tag.

Disabling Tests

From time to time you might want to disable a test or all tests in a container temporarily. You can do that by adding the @Disabled annotation to a property method or a container class.

import net.jqwik.api.Disabled;

@Disabled("for whatever reason")
class DisablingExamples {

	@Property
	@Disabled
	void aDisabledProperty() { }

}

Disabled properties will be reported by IDEs and build tools as “skipped” together with the reason - if one has been provided.

Be careful not to use the Jupiter annotation with the same name. Jqwik will refuse to execute methods that have Jupiter annotations.

Default Parameter Generation

jqwik tries to generate values for those property method parameters that are annotated with @ForAll. If the annotation does not have a value parameter, jqwik will use default generation for the following types:

If you use @ForAll with a value, e.g. @ForAll("aMethodName"), the method referenced by "aMethodName" will be called to provide an Arbitrary of the required type (see Arbitrary Provider Methods). Also, when you use it with a supplier attribute, e.g. @ForAll(supplier=MySupplier.class), an arbitrary supplier implementation is invoked.

Constraining Default Generation

Default parameter generation can be influenced and constrained by additional annotations, depending on the requested parameter type.

Allow Null Values

String Length

String not Blank

Character Sets

When generating chars any unicode character might be generated.

When generating Strings, however, Unicode “noncharacters” and “private use characters” will not be generated unless you explicitly include them using @Chars or @CharRange (see below).

You can use the following annotations to restrict the set of allowed characters and even combine several of them:

They work for generated Strings and Characters.

List, Set, Stream, Iterator, Map and Array Size

Unique Elements

Integer Constraints

Decimal Constraints

Constraining parameterized types

When you want to constrain the generation of contained parameter types you can annotate the parameter type directly, e.g.:

@Property
void aProperty(@ForAll @Size(min= 1) List<@StringLength(max=10) String> listOfStrings) {
}

will generate lists with a minimum size of 1 filled with Strings that have 10 characters max.

Constraining array types

Before version 1.6.2 annotations on array types - and also in vararg types - were handed down to the array’s component type. That means that @ForAll @WithNull String[] aStringArray used to inject null values for aStringArray as well as for elements in the array.

This behaviour has changed with version 1.6.2 in an incompatible way: Annotations are only applied to the array itself. The reason is that there was no way to specify if an annotation should be applied to the array type, the component type or both. Therefore, the example above must be re-written as:

@Property
void myProperty(@ForAll("stringArrays") String[] aStringArray) {...}

@Provide
Arbitrary<String[]> stringArrays() {
  return Arbitraries.strings().injectNull(0.05).array(String[].class).injectNull(0.05);
}

This is arguably more involved, but allows the finer control that is necessary in some cases.

Providing variable types

While checking properties of generically typed classes or functions, you often don’t care about the exact type of variables and therefore want to express them with type variables. jqwik can also handle type variables and wildcard types. The handling of upper and lower bounds works mostly as you would expect it.

Consider the following examples:

class VariableTypedPropertyExamples {

	@Property
	<T> boolean unboundedGenericTypesAreResolved(@ForAll List<T> items, @ForAll T newItem) {
		items.add(newItem);
		return items.contains(newItem);
	}

	@Property
	<T extends Serializable & Comparable> void someBoundedGenericTypesCanBeResolved(@ForAll List<T> items, @ForAll T newItem) {
	}

	@Property
	void someWildcardTypesWithUpperBoundsCanBeResolved(@ForAll List<? extends Serializable> items) {
	}

}

In the case of bounded type variables and bounded wildcard types, jqwik will check if any registered arbitrary provider can provide suitable arbitraries and choose randomly between those.

There is, however, a potentially unexpected behaviour, when the same type variable is used in more than one place and can be resolved by more than one arbitrary. In this case it can happen that the variable does not represent the same type in all places. You can see this above in property method someBoundedGenericTypesCanBeResolved() where items might be a list of Strings but newItem of some number type - and all that in the same call to the method!

Self-Made Annotations

You can make your own annotations instead of using jqwik’s built-in ones. BTW, ‘@Example’ is nothing but a plain annotation using @Property as “meta”-annotation.

The following example provides an annotation to constrain String or Character generation to German letters only:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
@NumericChars
@AlphaChars
@Chars({'ä', 'ö', 'ü', 'Ä', 'Ö', 'Ü', 'ß'})
@Chars({' ', '.', ',', ';', '?', '!'})
@StringLength(min = 10, max = 100)
public @interface GermanText { }

@Property(tries = 10) @Reporting(Reporting.GENERATED)
void aGermanText(@ForAll @GermanText String aText) {}

The drawback of self-made annotations is that they do not forward their parameters to meta-annotations, which constrains their applicability to simple cases.

Customized Parameter Generation

Sometimes the possibilities of adjusting default parameter generation through annotations is not enough. You want to control the creation of values programmatically. The means to do that are provider methods.

Arbitrary Provider Methods

Look at the following example:

@Property
boolean concatenatingStringWithInt(
    @ForAll("shortStrings") String aShortString,
    @ForAll("10 to 99") int aNumber
) {
    String concatenated = aShortString + aNumber;
    return concatenated.length() > 2 && concatenated.length() < 11;
}

@Provide
Arbitrary<String> shortStrings() {
    return Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(8);
}

@Provide("10 to 99")
Arbitrary<Integer> numbers() {
    return Arbitraries.integers().between(10, 99);
}

The String value of the @ForAll annotation serves as a reference to a method within the same class (or one of its superclasses or owning classes). This reference refers to either the method’s name or the String value of the method’s @Provide annotation.

The providing method has to return an arbitrary instance that matches type @Arbitrary<? extends TParam> where TParam is the static type of the parameter to be provided. If the return type cannot be matched, jqwik will throw a CannotFindArbitraryException.

Caveat:

Up to version 1.7.4, it was possible that the provider method’s return type was broader than the property method’s parameter type. Since version 1.8.0 this is no longer the case. That’s why the following code will now fail at runtime with a CannotFindArbitraryException:

@Property
void favouritePrimes(@ForAll("favouritePrimes") int aFavourite) {
}

@Provide
Arbitrary<?> favouritePrimes() {
    return Arbitraries.of(3, 5, 7, 13, 17, 23, 41, 101);
}

How to write a provider method

Arbitrary provision often starts with a static method call to Arbitraries, maybe followed by one or more filtering, mapping, flat mapping or combining actions.

In general, all methods that return and modify arbitrary instances can be used.

Provider Methods with Parameters

The examples of provider methods you’ve seen so far had no parameters. In more complicated scenarios, however, you may want to tune an arbitrary depending on the concrete parameter to be generated.

The provider method can have a few optional parameters:

The following example uses a TypeUsage parameter to unify two provider methods.
Imagine you want to randomly choose one of your favourite primes; that’s easy:

@Property
void favouritePrimes(@ForAll("favouritePrimes") int aFavourite) {
}

@Provide
Arbitrary<Integer> favouritePrimes() {
	return Arbitraries.of(3, 5, 7, 13, 17, 23, 41, 101);
}

From time to time, though, you may want to use details from the type usage to change how the arbitrary is created. In the following example, an annotation is used to decide whether to generate large primes:

@Property
void largePrimes(@ForAll("favouritePrimes") @LargePrimes int aPrime) {
	System.out.println("prime = " + aPrime);
}

@Property
void smallPrimes(@ForAll("favouritePrimes") int aPrime) {
	System.out.println("prime = " + aPrime);
}

@Provide
Arbitrary<Integer> favouritePrimes(TypeUsage targetType) {
	if (targetType.findAnnotation(LargePrimes.class).isPresent()) {
		return Arbitraries.integers().greaterOrEqual(2).filter(this::isPrime);
	}
	return Arbitraries.of(3, 5, 7, 13, 17, 23, 41);
}

Arbitrary Suppliers

Similar to provider methods you can specify an ArbitrarySupplier implementation in the @ForAll annotation:

@Property
boolean concatenatingStringWithInt(
	@ForAll(supplier = ShortStrings.class) String aShortString,
	@ForAll(supplier = TenTo99.class) int aNumber
) {
	String concatenated = aShortString + aNumber;
	return concatenated.length() > 2 && concatenated.length() < 11;
}

class ShortStrings implements ArbitrarySupplier<String> {
	@Override
	public Arbitrary<String> get() {
		return Arbitraries.strings().withCharRange('a', 'z')
						  .ofMinLength(1).ofMaxLength(8);
	}
}

class TenTo99 implements ArbitrarySupplier<Integer> {
	@Override
	public Arbitrary<Integer> get() {
		return Arbitraries.integers().between(10, 99);
	}
}

Although this is a bit more verbose than using a provider method, it has two advantages:

The ArbitrarySupplier interface requires to override exactly one of two methods:

Providing Arbitraries for Embedded Types

There is an alternative syntax to @ForAll("methodRef") using a From annotation:

@Property
boolean concatenatingStringWithInt(
    @ForAll @From("shortStrings") String aShortString,
    @ForAll @From("10 to 99") int aNumber
) { ... }

Why this redundancy? Well, @From becomes a necessity when you want to provide the arbitrary of an embedded type parameter. Consider this example:

@Property
boolean joiningListOfStrings(@ForAll List<@From("shortStrings") String> listOfStrings) {
    String concatenated = String.join("", listOfStrings);
    return concatenated.length() <= 8 * listOfStrings.size();
}

Here, the list is created using the default list arbitrary, but the String elements are generated using the arbitrary from the method shortStrings.

Alternatively, you can also use a supplier attribute in @From:

@Property
boolean joiningListOfStrings(@ForAll List<@From(supplier=ShortStrings.class) String> listOfStrings) {
    String concatenated = String.join("", listOfStrings);
    return concatenated.length() <= 8 * listOfStrings.size();
}

Static Arbitraries methods

The starting point for generation usually is a static method call on class Arbitraries.

Generate values yourself

Select or generate values randomly

Select randomly with Weights

If you have a set of values to choose from with weighted probabilities, use Arbitraries.frequency(...):

@Property
void abcdWithFrequencies(@ForAll("abcdWeighted") String aString) {
    Statistics.collect(aString);
}

@Provide
Arbitrary<String> abcdWeighted() {
    return Arbitraries.frequency(
        Tuple.of(1, "a"),
        Tuple.of(5, "b"),
        Tuple.of(10, "c"),
        Tuple.of(20, "d")
    );
}

The first value of the tuple specifies the frequency of a particular value in relation to the sum of all frequencies. In the given example the sum is 36, thus "a" will be generated with a probability of 1/36 whereas "d" has a generation probability of 20/36 (= 5/9).

Shrinking moves towards the start of the frequency list.

Characters and Strings

You can browse the API for generating strings and chars here:

When it comes to defining the base set of possible chars to choose from Character and String arbitraries work very similarly, e.g.

CharacterArbitrary chars = Arbitraries.chars()
                              .numeric()
                              .alpha()
                              .with('.', ',', ';', '!', '?');

creates a generator for alphanumeric chars plus the most common punctuation (but no spaces!). For strings combined of this letters, the code is:

StringArbitrary strings = Arbitraries.strings()
                            .numeric()
                            .alpha()
                            .withChars('.', ',', ';', '!', '?');

String Size

Without any additional configuration, the size of generated strings is between 0 and 255. To change this StringArbitrary comes with additional capabilities to set the minimal and maximal length of a string:

You can also influence the random distribution of the length. If you want, for example, a uniform distribution of string length between 5 and 25 characters, this is how you do it:

Arbitraries.strings().ofMinLength(5).ofMaxLength(25)
		   .withLengthDistribution(RandomDistribution.uniform());

String with Unique Characters

In case you need a string with unique characters you can use StringArbitrary.uniqueChars(). Here’s an example:

Arbitraries.strings().alpha().ofMaxLength(25)
		   .uniqueChars();

Alternatively you can use the annotation @UniqueChars on a @ForAll String parameter:

@Property
boolean noDuplicateCharsInStrings(@ForAll @UniqueChars String aString) {
    return aString.distinct().count() == aString.length();
}

java.util.Random

Shuffling Permutations

Default Types

Numeric Arbitrary Types

Creating an arbitrary for numeric values also starts by calling a static method on class Arbitraries. There are two fundamental types of numbers: integral numbers and decimal numbers. jqwik supports all of Java’s built-in number types.

Each type has its own fluent interface but all numeric arbitrary types share some things:

Integrals

Decimals

Decimal arbitrary types come with a few additional capabilities:

Special Decimal Values

Since the generation of decimal values is constrained by the significant decimal places, some special values, like MIN_NORMAL and MIN_VALUE, will never be generated, although they are attractors of bugs in some cases. That’s why DecimalArbitrary and FloatArbitrary provide you with the capability to add special values into the possible generation scope:

Special values are also considered to be edge cases and they are used in exhaustive generation. Standard special values are: MIN_VALUE, MIN_NORMAL, NaN, POSITIV_INFINITY and NEGATIVE_INFINITY.

Random Numeric Distribution

With release 1.3.0 jqwik provides you with a means to influence the probability distribution of randomly generated numbers. The way to do that is by calling withDistribution(distribution). Currently three different distributions are supported:

The specified distribution does not influence the generation of edge cases.

The following example generates numbers between 0 and 20 using a gaussian probability distribution with its mean at 10 and a standard deviation of about 3.3:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = Histogram.class)
void gaussianDistributedIntegers(@ForAll("gaussians") int aNumber) {
    Statistics.collect(aNumber);
}

@Provide
Arbitrary<Integer> gaussians() {
    return Arbitraries
               .integers()
               .between(0, 20)
               .shrinkTowards(10)
               .withDistribution(RandomDistribution.gaussian());
}

Look at the statistics to see if it fits your expectation:

[RandomDistributionExamples:gaussianDistributedIntegers] (1000) statistics = 
       # | label | count | 
    -----|-------|-------|---------------------------------------------------------------------------------
       0 |     0 |    15 | ■■■■■
       1 |     1 |     8 | ■■
       2 |     2 |    12 | ■■■■
       3 |     3 |     9 | ■■■
       4 |     4 |    14 | ■■■■
       5 |     5 |    28 | ■■■■■■■■■
       6 |     6 |    38 | ■■■■■■■■■■■■■
       7 |     7 |    67 | ■■■■■■■■■■■■■■■■■■■■■■■
       8 |     8 |    77 | ■■■■■■■■■■■■■■■■■■■■■■■■■■
       9 |     9 |   116 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |    10 |   231 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    11 |   101 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      12 |    12 |    91 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      13 |    13 |    60 | ■■■■■■■■■■■■■■■■■■■■
      14 |    14 |    45 | ■■■■■■■■■■■■■■■
      15 |    15 |    36 | ■■■■■■■■■■■■
      16 |    16 |    19 | ■■■■■■
      17 |    17 |    10 | ■■■
      18 |    18 |     7 | ■■
      19 |    19 |     1 | 
      20 |    20 |    15 | ■■■■■

You can notice that values 0 and 20 should have the lowest probability but they do not. This is because they will be generated a few times as edge cases.

Collections, Streams, Iterators and Arrays

Arbitraries for multi value types require to start with an Arbitrary instance for the element type. You can then create the corresponding multi value arbitrary from there:

Size of Multi-value Containers

Without any additional configuration, the size of generated containers (lists, sets, arrays etc.) is between 0 and 255. To change this all the arbitraries from above support

Usually the distribution of generated container size is heavily distorted towards the allowed minimum. If you want to influence the random distribution you can use withSizeDistribution(RandomDistribution). For example:

Arbitraries.integers().list().ofMaxSize(100)
    .withSizeDistribution(RandomDistribution.uniform());

See the section on random numeric distribution to check out the available distribution implementations.

Collecting Values in a List

If you do not want any random combination of values in your list - as can be done with Arbitrary.list() - you have the possibility to collect random values in a list until a certain condition is fulfilled. Arbitrary.collect(Predicate condition) is what you need in those cases.

Imagine you need a list of integers the sum of which should be at least 1000. Here’s how you could do that:

Arbitrary<Integer> integers = Arbitraries.integers().between(1, 100);
Arbitrary<List<Integer>> collected = integers.collect(list -> sum(list) >= 1000);

Optional

Using Arbitrary.optional(double presenceProbability) allows to generate an optional of any type. Optional.empty() values are injected with a probability of 1 - presenceProbability.

Just using Arbitrary.optional() uses a presenceProbability of 0.95, i.e. 1 in 20 generates is empty.

Tuples of same base type

If you want to generate tuples of the same base types that also use the same generator, that’s how you can do it:

Arbitrary<Tuple.Tuple2> integerPair = Arbitrary.integers().between(1, 25).tuple2();

There’s a method for tuples of length 1 to 5:

Maps

Generating instances of type Map is a bit different since two arbitraries are needed, one for the key and one for the value. Therefore you have to use Arbitraries.maps(...) like this:

@Property
void mapsFromNumberToString(@ForAll("numberMaps")  Map<Integer, String> map) {
    Assertions.assertThat(map.keySet()).allMatch(key -> key >= 0 && key <= 1000);
    Assertions.assertThat(map.values()).allMatch(value -> value.length() == 5);
}

@Provide
Arbitrary<Map<Integer, String>> numberMaps() {
    Arbitrary<Integer> keys = Arbitraries.integers().between(1, 100);
    Arbitrary<String> values = Arbitraries.strings().alpha().ofLength(5);
    return Arbitraries.maps(keys, values);
}

Map Size

Influencing the size of a generated map works exactly like in other multi-value containers.

Map Entries

For generating individual Map.Entry instances there is Arbitraries.entries(...).

Functional Types

Interfaces that have a single (non default) method are considered to be Functional types; they are sometimes called SAM types for “single abstract method”. If a functional type is used as a @ForAll-parameter jqwik will automatically generate instances of those functions. The generated functions have the following characteristics:

Let’s look at an example:

@Property
void fromIntToString(@ForAll Function<Integer, @StringLength(5) String> function) {
    assertThat(function.apply(42)).hasSize(5);
    assertThat(function.apply(1)).isEqualTo(function.apply(1));
}

This works for any interface-based functional types, even your own. If you register a default provider for a functional type with a priority of 0 or above, it will take precedence.

If the functions need some specialized arbitrary for return values or if you want to fix the function’s behaviour for some range of values, you can define the arbitrary manually:

@Property
void emptyStringsTestFalse(@ForAll("predicates") Predicate<String> predicate) {
    assertThat(predicate.test("")).isFalse();
}

@Provide
Arbitrary<Predicate<String>> predicates() {
    return Functions
        .function(Predicate.class)
        .returns(Arbitraries.of(true, false))
        .when(parameters -> parameters.get(0).equals(""), parameters -> false);
}

In this example the generated predicate will always return false when given an empty String and randomly choose between true and false in all other cases.

Fluent Configuration Interfaces

Most specialized arbitrary interfaces provide special methods to configure things like size, length, boundaries etc. Have a look at the Java doc for the following types, which are organized in a flat hierarchy:

Here are a two examples to give you a hint of what you can do:

@Provide
Arbitrary<String> alphaNumericStringsWithMinLength5() {
    return Arbitraries.strings().ofMinLength(5).alpha().numeric();
}

@Provide
Arbitrary<List<Integer>> fixedSizedListOfPositiveIntegers() {
    return Arbitraries.integers().greaterOrEqual(0).list().ofSize(17);
}

Generate null values

Predefined generators will never create null values. If you want to allow that, call Arbitrary.injectNull(double probability). The following provider method creates an arbitrary that will return a null String in about 1 of 100 generated values.

@Provide 
Arbitrary<String> stringsWithNull() {
  return Arbitraries.strings(0, 10).injectNull(0.01);
}

Inject duplicate values

Sometimes it is important that your generator will create a previous value again in order to trigger certain scenarios or branches in your code. Imagine you want to check if your carefully hand-crafted String comparator really is as symmetric as it’s supposed to be:

Comparator<String> comparator = (s1, s2) -> {
    if (s1.length() + s2.length() == 0) return 0;
    if (s1.compareTo(s2) > 0) {
        return 1;
    } else {
        return -1;
    }
};

@Property
boolean comparing_strings_is_symmetric(@ForAll String first, @ForAll String second) {
    int comparison = comparator.compare(first, second);
    return comparator.compare(second, first) == -comparison;
}

The property (most probably) succeeds and will give you confidence in your code. Or does it? Natural scepticism makes you check some statistics:

@Property(edgeCases = EdgeCasesMode.NONE)
boolean comparing_strings_is_symmetric(@ForAll String first, @ForAll String second) {
    int comparison = comparator.compare(first, second);
    String comparisonRange = comparison < 0 ? "<0" : comparison > 0 ? ">0" : "=0";
    String empty = first.isEmpty() || second.isEmpty() ? "empty" : "not empty";
    Statistics.collect(comparisonRange, empty);
    return comparator.compare(second, first) == -comparison;
}

The following output

[comparing strings is symmetric] (1000) statistics = 
    <0 not empty (471) : 47,10 %
    >0 not empty (456) : 45,60 %
    <0 empty     ( 37) :  3,70 %
    >0 empty     ( 35) :  3,50 %
    =0 empty     (  1) :  0,10 %

reveals that our generated test data is missing one combination: Comparison value of 0 for non-empty strings. In theory a generic String arbitrary could generate the same non-empty string but it’s highly unlikely. This is where we have to think about raising the probability of the same value being generated more often:

@Property
boolean comparing_strings_is_symmetric(@ForAll("pair") Tuple2<String, String> pair) {
    String first = pair.get1();
    String second = pair.get2();
    int comparison = comparator.compare(first, second);
    return comparator.compare(second, first) == -comparison;
}

@Provide
Arbitrary<Tuple2<String, String>> pair() {
    return Arbitraries.strings().injectDuplicates(0.1).tuple2();
}

This will cover the missing case and will reveal a bug in the comparator. Mind that you have to make sure that the same generator instance is being used for the two String values - using tuple2() does that.

Filtering

If you want to include only part of all the values generated by an arbitrary, use Arbitrary.filter(Predicate<? super T> filterPredicate). The following arbitrary will filter out all even numbers from the stream of generated integers:

@Provide 
Arbitrary<Integer> oddNumbers() {
  return Arbitraries.integers().filter(aNumber -> aNumber % 2 != 0);
}

Keep in mind that your filter condition should not be too restrictive. If the generator fails to find a suitable value after 10000 trials, the current property will be abandoned by throwing an exception.

Mapping

Sometimes it’s easier to start with an existing arbitrary and use its generated values to build other objects from them. In that case, use Arbitrary.map(Function<T, U> mapper). The following example uses generated integers to create numerical Strings:

@Provide 
Arbitrary<String> fiveDigitStrings() {
  return Arbitraries.integers(10000, 99999).map(aNumber -> String.valueOf(aNumber));
}

You could generate the same kind of values by constraining and filtering a generated String. However, the shrinking target would probably be different. In the example above, shrinking will move towards the lowest allowed number, that is 10000.

Mapping over Elements of Collection

ListArbitrary and SetArbitrary provide you with a convenient way to map over each element of a collection and still keep the generated collection. This is useful when the mapping function needs access to all elements of the list to do its job:

The following example will generate a list of integers and enrich the elements with the number of occurrences of the element within the list:

@Property
void elementsAreCorrectlyCounted(@ForAll("elementsWithOccurrence") List<Tuple2<Integer, Long>> list) {
	Assertions.assertThat(list).allMatch(t -> t.get2() <= list.size());
}

@Provide
Arbitrary<List<Tuple2<Integer, Long>>> elementsWithOccurrence() {
	return Arbitraries.integers().between(10000, 99999).list()
					  .mapEach((all, i) -> {
						  long count = all.stream().filter(e -> e.equals(i)).count();
						  return Tuple.of(i, count);
					  });
}

Flat Mapping

Similar as in the case of Arbitrary.map(..) there are situations in which you want to use a generated value in order to create another Arbitrary from it. Sounds complicated? Have a look at the following example:

@Property
boolean fixedSizedStrings(@ForAll("listsOfEqualSizedStrings")List<String> lists) {
    return lists.stream().distinct().count() == 1;
}

@Provide
Arbitrary<List<String>> listsOfEqualSizedStrings() {
    Arbitrary<Integer> integers2to5 = Arbitraries.integers().between(2, 5);
    return integers2to5.flatMap(stringSize -> {
        Arbitrary<String> strings = Arbitraries.strings() 
                .withCharRange('a', 'z') 
                .ofMinLength(stringSize).ofMaxLength(stringSize);
        return strings.list();
    });
}

The provider method will create random lists of strings, but in each list the size of the contained strings will always be the same - between 2 and 5.

Flat Mapping with Tuple Types

In the example above you used a generated value in order to create another arbitrary. In those situations you often want to also provide the original values to your property test.

Imagine, for instance, that you’d like to test properties of String.substring(begin, end). To randomize the method call, you not only need a string but also the begin and end indices. However, both have dependencies:

@Property
void substringLength(@ForAll("stringWithBeginEnd") Tuple3<String, Integer, Integer> stringBeginEnd) {
    String aString = stringBeginEnd.get1();
    int begin = stringBeginEnd.get2();
    int end = stringBeginEnd.get3();
    assertThat(aString.substring(begin, end).length()).isEqualTo(end - begin);
}

@Provide
Arbitrary<Tuple3<String, Integer, Integer>> stringWithBeginEnd() {
    Arbitrary<String> stringArbitrary = Arbitraries.strings() 
            .withCharRange('a', 'z') 
            .ofMinLength(2).ofMaxLength(20);
    return stringArbitrary 
            .flatMap(aString -> Arbitraries.integers().between(0, aString.length()) 
                    .flatMap(end -> Arbitraries.integers().between(0, end) 
                            .map(begin -> Tuple.of(aString, begin, end))));
}

Mind the nested flat mapping, which is an aesthetic nuisance but nevertheless very useful.

Flat Mapping over Elements of Collection

Just like mapping over elements of a collection ListArbitrary and SetArbitrary provide you with a mechanism to flat-map over each element of a collection and still keep the generated collection:

Implicit Flat Mapping

Flat mapping syntax - especially when it’s nested - is a bit cumbersome to read. Starting with version 1.5.2 jqwik allows to use flat mapping implicitly. You simply add a @ForAll parameter to your provider method, the value of which will be generated using standard parameter generation. Under the hood this uses this parameter’s arbitrary and call flatMap on it.

Here’s the example from above with no explicit flat mapping:

@Property
@Report(Reporting.GENERATED)
void substringLength(@ForAll("stringWithBeginEnd") Tuple3<String, Integer, Integer> stringBeginEnd) {
	String aString = stringBeginEnd.get1();
	int begin = stringBeginEnd.get2();
	int end = stringBeginEnd.get3();
	assertThat(aString.substring(begin, end).length()).isEqualTo(end - begin);
}

@Provide
Arbitrary<String> simpleStrings() {
	return Arbitraries.strings()
					  .withCharRange('a', 'z')
					  .ofMinLength(2).ofMaxLength(20);
}

@Provide
Arbitrary<Tuple2<String, Integer>> stringWithEnd(@ForAll("simpleStrings") String aString) {
	return Arbitraries.integers().between(0, aString.length())
					  .map(end -> Tuple.of(aString, end));
}

@Provide
Arbitrary<Tuple3<String, Integer, Integer>> stringWithBeginEnd(@ForAll("stringWithEnd") Tuple2<String, Integer> stringWithEnd) {
	String aString = stringWithEnd.get1();
	int end = stringWithEnd.get2();
	return Arbitraries.integers().between(0, end)
					  .map(begin -> Tuple.of(aString, begin, end));
}

Randomly Choosing among Arbitraries

If you have several arbitraries of the same type, you can create a new arbitrary of the same type which will choose randomly one of those arbitraries before generating a value:

@Property
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    String classifier = anInt < -1000 ? "below" : anInt > 1000 ? "above" : "one";
    Statistics.collect(classifier);
    
    return anInt < -1000 //
            || Math.abs(anInt) == 1 //
            || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-2000, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 2000);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);
    
    return Arbitraries.oneOf(below1000, above1000, oneOrMinusOne);
}

In this example the statistics should also give you an equal distribution between the three types of integers.

If you don’t want to choose with equal probability - but with differing frequency - you can do that in a similar way:

@Property(tries = 100)
@Report(Reporting.GENERATED)
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    return anInt < -1000 //
               || Math.abs(anInt) == 1 //
               || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-1050, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 1050);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);

    return Arbitraries.frequencyOf(
        Tuple.of(1, below1000),
        Tuple.of(3, above1000),
        Tuple.of(6, oneOrMinusOne)
    );
}

Uniqueness Constraints

In many problem domains there exist identifying features or attributes that must not appear more than once. In those cases the multiple generation of objects can be restricted by either annotating parameters with @UniqueElements or by using one of the many uniqueness(..) configuration methods for collections and collection-like types:

The following examples demonstrates how to generate a list of Person objects whose names must be unique:

@Property
void listOfPeopleWithUniqueNames(@ForAll("people") List<Person> people) {
  List<String> names = people.stream().map(p -> p.name).collect(Collectors.toList());
  Assertions.assertThat(names).doesNotHaveDuplicates();
}

@Provide
Arbitrary<List<Person>> people() {
  Arbitrary<String> names = Arbitraries.strings().alpha().ofMinLength(3).ofMaxLength(20);
  Arbitrary<Integer> ages = Arbitraries.integers().between(0, 120);
  
  Arbitrary<Person> persons = Combinators.combine(names, ages).as((name, age) -> new Person(name, age));
  return persons.list().uniqueElements(p -> p.name);
};

Ignoring Exceptions During Generation

Once in a while, usually when combining generated values, it’s difficult to figure out in advance all the constraints that make the generation of objects valid. In a good object-oriented model, however, the objects themselves – i.e. their constructors or factory methods – take care that only valid objects can be created. The attempt to create an invalid value will be rejected with an exception.

As a good example have a look at JDK’s LocalDate class, which allows to instantiate dates using LocalDate.of(int year, int month, int dayOfMonth). In general dayOfMonth can be between 1 and 31 but trying to generate a “February 31” will throw a DateTimeException. Therefore, when you want to randomly generated dates between “January 1 1900” and “December 31 2099” you have two choices:

@Provide
Arbitrary<LocalDate> datesBetween1900and2099() {
  Arbitrary<Integer> years = Arbitraries.integers().between(1900, 2099);
  Arbitrary<Integer> months = Arbitraries.integers().between(1, 12);
  Arbitrary<Integer> days = Arbitraries.integers().between(1, 31);
  
  return Combinators.combine(years, months, days)
  	  .as(LocalDate::of)
  	  .ignoreException(DateTimeException.class);
}

If you want to ignore more than one exception type you can use Arbitrary.ignoreExceptions(Class<? extends Throwable>...)

Ignoring Exceptions in Provider Methods

If the arbitrary is created in a provider method and the exception(s) should be ignored on the outermost level, you can use the ignoreExceptions attribute of the @Provide annotation:

@Provide(ignoreExceptions = DateTimeException.class)
Arbitrary<LocalDate> datesBetween1900and2099() {
    Arbitrary<Integer> years = Arbitraries.integers().between(1900, 2099);
    Arbitrary<Integer> months = Arbitraries.integers().between(1, 12);
    Arbitrary<Integer> days = Arbitraries.integers().between(1, 31);

    return Combinators.combine(years, months, days).as(LocalDate::of);
}

Fix an Arbitrary’s genSize

Some generators (e.g. most number generators) are sensitive to the genSize value that is used when creating them. The default value for genSize is the number of tries configured for the property they are used in. If there is a need to influence the behaviour of generators you can do so by using Arbitrary.fixGenSize(int).

Combining Arbitraries

Sometimes just mapping a single stream of generated values is not enough to generate a more complicated domain object. What you want to do is to create arbitraries for parts of your domain object and then mix those parts together into a resulting combined arbitrary.

Jqwik offers provides two main mechanism to do that:

Combining Arbitraries with combine

Combinators.combine() allows you to set up a composite arbitrary from up to eight parts.

The following example generates Person instances from three arbitraries as inputs.

@Property
void validPeopleHaveIDs(@ForAll("validPeople") Person aPerson) {
    Assertions.assertThat(aPerson.getID()).contains("-");
    Assertions.assertThat(aPerson.getID().length()).isBetween(5, 24);
}

@Provide
Arbitrary<Person> validPeople() {
    Arbitrary<String> names = Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(3).ofMaxLength(21);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    return Combinators.combine(names, ages)
        .as((name, age) -> new Person(name, age));
}

class Person {
    private final String name;
    private final int age;

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String getID() {
        return name + "-" + age;
    }

    @Override
    public String toString() {
        return String.format("%s:%s", name, age);
    }
}

The property should fail, thereby shrinking the falsified Person instance to

Shrunk Sample (<n> steps)
-------------------------
  aPerson: aaaaaaaaaaaaaaaaaaaaa:100

The Combinators.combine method accepts up to 8 parameters of type Arbitrary. If you need more you have a few options:

Combining Arbitraries vs Flat Mapping

Combining arbitraries with each other can also be achieved through flat mapping. So, the validPeople provider method from above could also be written as:

@Provide
Arbitrary<Person> validPeople() {
    Arbitrary<String> names = Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(3).ofMaxLength(21);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    return names.flatMap(name -> ages.map(age -> new Person(name, age)));
}

This approach has two disadvantages, though:

  1. The more arbitraries you combine, the more nesting of flat maps will you need. This does not only look ugly, but it’s also hard to understand.
  2. Since flat mapping is about the dependency of one arbitrary on values generated by another, shrinking cannot be as aggressive. That means that in many cases using combine(..) will lead to better shrinking behaviour than nested flatMap(..) calls.

The drawback of combine is that it cannot replace flatMap in all situations. If there is a real dependency between arbitraries, you cannot just combine them. Unless filtering combinations can take care of the dependency.

Filtering Combinations

You may run into situations in which you want to combine two or more arbitraries, but not all combinations of values makes sense. Consider the example of combining a pair of values from the same domain, but the values should never be the same. Adding a filter step between combine(..) and as(..) provides you with the capability to sort out unwanted combinations:

@Property
void pairsCannotBeTwins(@ForAll("digitPairsWithoutTwins") String pair) {
    Assertions.assertThat(pair).hasSize(2);
    Assertions.assertThat(pair.charAt(0)).isNotEqualTo(pair.charAt(1));
}

@Provide
Arbitrary<String> digitPairsWithoutTwins() {
    Arbitrary<Integer> digits = Arbitraries.integers().between(0, 9);
    return Combinators.combine(digits, digits)
                      .filter((first, second) -> first != second)
                      .as((first, second) -> first + "" + second);
}

Flat Combination

If generating domain values requires to use several generated values to be used in generating another one, there’s the combination of flat mapping and combining:

@Property
boolean fullNameHasTwoParts(@ForAll("fullName") String aName) {
    return aName.split(" ").length == 2;
}

@Provide
Arbitrary<String> fullName() {
    IntegerArbitrary firstNameLength = Arbitraries.integers().between(2, 10);
    IntegerArbitrary lastNameLength = Arbitraries.integers().between(2, 10);
    return Combinators.combine(firstNameLength, lastNameLength).flatAs( (fLength, lLength) -> {
        Arbitrary<String> firstName = Arbitraries.strings().alpha().ofLength(fLength);
        Arbitrary<String> lastName = Arbitraries.strings().alpha().ofLength(fLength);
        return Combinators.combine(firstName, lastName).as((f,l) -> f + " " + l);
    });
}

Often, however, there’s an easier way to achieve the same goal which does not require the flat combination of arbitraries:

@Provide
Arbitrary<String> fullName2() {
    Arbitrary<String> firstName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    Arbitrary<String> lastName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    return Combinators.combine(firstName, lastName).as((f, l) -> f + " " + l);
}

This is not only easier to understand but it usually improves shrinking.

Combining Arbitraries with Builders

There’s an alternative way to combine arbitraries to create an aggregated object by using a builder for the aggregated object. Consider the example from above and throw a PersonBuilder into the mix:

static class PersonBuilder {

    private String name = "A name";
    private int age = 42;

    public PersonBuilder withName(String name) {
        this.name = name;
        return this;
    }

    public PersonBuilder withAge(int age) {
        this.age = age;
        return this;
    }

    public Person build() {
        return new Person(name, age);
    }
}

Then you can go about generating people in the following way:

@Provide
Arbitrary<Person> validPeopleWithBuilder() {
    Arbitrary<String> names = 
        Arbitraries.strings().withCharRange('a', 'z').ofMinLength(2).ofMaxLength(20);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    
    return Builders.withBuilder(() -> new PersonBuilder())
        .use(names).in((builder, name) -> builder.withName(name))
        .use(ages).withProbability(0.5).in((builder, age)-> builder.withAge(age))
        .build( builder -> builder.build());
}

If you don’t want to introduce an explicit builder object, you can also use a mutable POJO – e.g. a Java bean – instead:

@Provide
Arbitrary<Person> validPeopleWithPersonAsBuilder() {
	Arbitrary<String> names =
		Arbitraries.strings().withCharRange('a', 'z').ofMinLength(3).ofMaxLength(21);
	Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);

	return Builders.withBuilder(() -> new Person(null, -1))
				   .use(names).inSetter(Person::setName)
				   .use(ages).withProbability(0.5).inSetter(Person::setAge)
				   .build();
}

Have a look at Builders.withBuilder(Supplier) to check the API.

Recursive Arbitraries

Sometimes it seems like a good idea to compose arbitraries and thereby recursively calling an arbitrary creation method. Generating recursive data types is one application field but you can also use it for other stuff.

Probabilistic Recursion

Look at the following example which generates sentences by recursively adding words to a sentence:

@Property
@Report(Reporting.GENERATED)
boolean sentencesEndWithAPoint(@ForAll("sentences") String aSentence) {
	return aSentence.endsWith(".");
	// return !aSentence.contains("x"); // using this condition instead 
	                                    // should shrink to "AAAAx."
}

@Provide
Arbitrary<String> sentences() {
	return Arbitraries.lazyOf(
		() -> word().map(w -> w + "."),
		this::sentence,
		this::sentence,
		this::sentence
	);
}

private Arbitrary<String> sentence() {
	return Combinators.combine(sentences(), word())
					  .as((s, w) -> w + " " + s);
}

private StringArbitrary word() {
    return Arbitraries.strings().alpha().ofLength(5);
}

There are two things to which you must pay attention:

There is also a caveat of which you should be aware: Never use this construct if suppliers make use of variable state like method parameters or changing instance members. In those cases use lazy() as explained below.

Using lazy() instead of lazyOf()

There is an almost equivalent variant to the example above:

@Property
boolean sentencesEndWithAPoint(@ForAll("sentences") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> sentences() {
    Arbitrary<String> sentence = Combinators.combine(
        Arbitraries.lazy(this::sentences),
        word()
    ).as((s, w) -> w + " " + s);

    return Arbitraries.oneOf(
        word().map(w -> w + "."),
        sentence,
        sentence,
        sentence
    );
}

private StringArbitrary word() {
    return Arbitraries.strings().alpha().ofLength(5);
}

The disadvantage of lazy() combined with oneOf() or frequencyOf() is its worse shrinking behaviour compared to lazyOf(). Therefore, choose lazyOf() whenever you can.

Deterministic Recursion

An alternative to probabilistic recursion shown above, is to use deterministic recursion with a counter to determine the base case. If you then use an arbitrary value for the counter, the generated sentences will be very similar, and you can often forgo using Arbitraries.lazyOf() or Arbitraries.lazy():

@Property
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
    Arbitrary<Integer> length = Arbitraries.integers().between(0, 10);
    Arbitrary<String> lastWord = word().map(w -> w + ".");
    return length.flatMap(l -> deterministic(l, lastWord));
}

@Provide
Arbitrary<String> deterministic(int length, Arbitrary<String> sentence) {
    if (length == 0) {
        return sentence;
    }
    Arbitrary<String> more = Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
    return deterministic(length - 1, more);
}

Deterministic Recursion with recursive()

To further simplify this jqwik provides two helper functions:

@Property
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
	Arbitrary<String> lastWord = word().map(w -> w + ".");

	return Arbitraries.recursive(
		() -> lastWord,
		this::prependWord,
		0, 10
	);
}

private Arbitrary<String> prependWord(Arbitrary<String> sentence) {
    return Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
}

Using Arbitraries Directly

Most of the time arbitraries are used indirectly, i.e. jqwik uses them under the hood to inject generated values as parameters. There are situations, though, in which you might want to generate values directly.

Generating a Single Value

Getting a single random value out of an arbitrary is easy and can be done with Arbitrary.sample():

Arbitrary<String> strings = Arbitraries.of("string1", "string2", "string3");
String aString = strings.sample();
assertThat(aString).isIn("string1", "string2", "string3");

Among other things, this allows you to use jqwik’s generation functionality with other test engines like Jupiter. Mind that jqwik uses a default genSize of 1000 under the hood and that the Random object will be either taken from the current property’s context or freshly instantiated if used outside a property.

Generating a Stream of Values

Getting a stream of generated values is just as easy with Arbitrary.sampleStream():

List<String> values = Arrays.asList("string1", "string2", "string3");
Arbitrary<String> strings = Arbitraries.of(values);
Stream<String> streamOfStrings = strings.sampleStream().limit(100);

assertThat(streamOfStrings).allMatch(values::contains);

Generating all possible values

There are a few cases when you don’t want to generate individual values from an arbitrary but use all possible values to construct another arbitrary. This can be achieved through Arbitrary.allValues().

Return type is Optional<Stream<T>> because jqwik can only perform this task if exhaustive generation is doable.

Iterating through all possible values

You can also use an arbitrary to iterate through all values it specifies. Use Arbitrary.forEachValue(Consumer action). for that purpose. This only works when exhaustive generation is possible. In other cases the attempt to iterate will result in an exception.

This is typically useful when your test requires to assert some fact for all values of a given (sub)set of objects. Here’s a contrived example:

@Property
void canPressAnyKeyOnKeyboard(@ForAll Keyboard keyboard, @ForAll Key key) {
    keyboard.press(key);
    assertThat(keyboard.isPressed(key));

    Arbitrary<Key> unpressedKeys = Arbitraries.of(keyboard.allKeys()).filter(k -> !k.equals(key));
    unpressedKeys.forEachValue(k -> assertThat(keyboard.isPressed(k)).isFalse());
}

In this example a simple for loop over allKeys() would also work. In more complicated scenarios jqwik will do all the combinations and filtering for you.

Using Arbitraries Outside Jqwik Lifecycle

All the methods mentioned in this chapter can be used outside a property, which also means outside jqwik’s lifecycle control. Probably the most prominent reason to do that is to experiment with arbitraries and value generation in a Java console or a main method. Another reason can be to use jqwik’s data generation capabilities for testing data in Jupiter or Cucumber tests.

In principal, there’s no problem with that approach. However, some generators are expensive to create and will therefore be cached. Other generators require some data persistence across generation iterations to work as expected. All this data will fill up your heap space and never be released, because jqwik cannot know, if you’re done with using a specific generator or not.

In order to mitigate that, there’s an experimental API that allows you to simulate a small part of jqwik’s property lifecycle. Currently this API consists of a few static methods on class net.jqwik.api.sessions.JqwikSession:

Mind that there’s currently no way to use nested sessions, spread the same session across threads or use more than one session concurrently.

Contract Tests

When you combine type variables with properties defined in superclasses or interfaces you can do some kind of contract testing. That means that you specify the properties in a generically typed interface and specify the concrete class to instantiate in a test container implementing the interface.

The following example was influenced by a similar feature in junit-quickcheck. Here’s the contract:

interface ComparatorContract<T> {
	Comparator<T> subject();

	@Property
	default void symmetry(@ForAll("anyT") T x, @ForAll("anyT") T y) {
		Comparator<T> subject = subject();

		Assertions.assertThat(signum(subject.compare(x, y))).isEqualTo(-signum(subject.compare(y, x)));
	}

	@Provide
	Arbitrary<T> anyT();
}

And here’s the concrete test container that can be run to execute the property with generated Strings:

class StringCaseInsensitiveProperties implements ComparatorContract<String> {

	@Override public Comparator<String> subject() {
		return String::compareToIgnoreCase;
	}

	@Override
	@Provide
	public Arbitrary<String> anyT() {
		return Arbitraries.strings().alpha().ofMaxLength(20);
	}
}

What we can see here is that jqwik is able to figure out the concrete type of type variables when they are used in subtypes that fill in the variables.

Stateful Testing

The approach described here has been freshly introduced in version 1.7.0. It is still marked “experimental” but will probably be promoted to “maintained” in one of the next minor versions of jqwik. You can also read about the old way of stateful testing. Since both approaches have an interface called Action, be careful to import the right one!

Despite its bad reputation state is an important concept in object-oriented languages like Java. We often have to deal with stateful objects or components whose state can be changed through methods. Applying the concept of properties to stateful objects or data is not a new idea. Jqwik provides the tools for you to explore and implement these ideas. Those tools are available in package net.jqwik.api.state.

State Machines

One, slightly formal, way to look at stateful objects are state machines. A state machine has an internal state and actions that change the internal state. Some actions have preconditions to constrain when they can be invoked. Also, state machines can have invariants that should never be violated regardless of the sequence of performed actions.

To make this abstract concept concrete, let’s look at a simple stack implementation:

public class MyStringStack {
	public void push(String element) { ... }
	public String pop() { ... }
	public void clear() { ... }
	public boolean isEmpty() { ... }
	public int size() { ... }
	public String top() { ... }
}

Specifying Actions

Jqwik’s new Action type covers the idea that an action can be represented by “arbitrary” transformers; arbitrary in the sense that in the context of property-based testing there’s some variation to it, e.g. a push onto a string stack can have any String as parameter. And as mentioned above, actions can be restricted by preconditions.

package net.jqwik.api.state;
interface Action<S> {
    default boolean precondition(S state) {
        return true;
    }

    interface Independent<S> extends Action<S> {
        Arbitrary<Transformer<S>> transformer();
    }

    interface Dependent<S> extends Action<S> {
        Arbitrary<Transformer<S>> transformer(S state);
    }
}

interface Transformer<S> extends Function<S, S> {}

What makes the abstraction a bit more complicated than desirable is the fact that the range of possible transformers may or may not depend on the previous state. That’s why there exist the two subtypes of Action: Independent and Dependent. Both types of actions can have a precondition to state when they can be applied. Leaving the precondition out means that the action can be applied at any time.

Given that the precondition is fulfilled,

Both transformer methods return an Arbitrary of Transformer instances, which means that they describe a range of possible transformations; jqwik will pick one of them at random as it does with any other Arbitrary. Mind that a transformer will not only invoke the transformation but will often check the correct state changes and postcondition(s) as well.

In our simple stack example at least three actions can be identified:

There are different ways to implement actions. Sometimes one is obviously simpler than the other. In other cases it’s a matter of taste - e.g. about preferring functions over classes, or the other way round.

Formulating Stateful Properties

Now that we have a set of actions, we can formulate the fundamental property of stateful systems:

For any valid sequence of actions all required state changes (aka postconditions) should be fulfilled.

Let’s translate that into jqwik’s language:

@Property
void checkMyStack(@ForAll("myStackActions") ActionChain<MyStringStack> chain) {
  chain.run();
}

@Provide
Arbitrary<ActionChain<MyStringStack>> myStackActions() {
  return ActionChain.startWith(MyStringStack::new)
                    .withAction(new PushAction())
                    .withAction(pop())
                    .withAction(new ClearAction());
}

The interesting API elements are

Running Stateful Properties

To give jqwik something to falsify, we broke the implementation of clear() so that it won’t clear everything if there are more than two elements on the stack:

public void clear() {
    // Wrong implementation to provoke falsification for stacks with more than 2 elements
    if (elements.size() > 2) {
        elements.remove(0);
    } else {
        elements.clear();
    }
}

Running the property should now produce a result similar to:

MyStringStackExamples:checkMyStack = 
  org.opentest4j.AssertionFailedError:
    Run failed after the following actions: [
        push(AAAAA)
        push(AAAAA)
        push(AAAAA)
        clear  
    ]
    final state: [AAAAA, AAAAA]
    [stack is empty] 
    Expecting value to be true but was false

The error message shows the sequence of actions that led to the failing postcondition. Moreover, you can notice that the sequence of actions has been shrunk to the minimal failing sequence.

Number of actions

jqwik will vary the number of generated actions according to the number of tries of your property. For the default of 1000 tries a sequence will have 32 actions. If need be you can specify the maximum number of actions explicitly using withMaxTransformations(int):

@Provide
Arbitrary<ActionChain<MyStringStack>> myStackActions() {
    return ActionChain.startWith(MyStringStack::new)
                      .withAction(new PushAction())
                      .withAction(pop())
                      .withAction(new ClearAction())
                      .withMaxTransformations(10);
}

The minimum number of generated actions in a sequence is 1 since checking an empty sequence does not make sense.

There’s also the possibility to use a potentially infinite chain, which then requires to explicitly add an action with an endOfChain() transformer:

@Provide
Arbitrary<ActionChain<MyStringStack>> myStackActions() {
    return ActionChain.startWith(MyStringStack::new)
                      .withAction(new PushAction())
                      .withAction(pop())
                      .withAction(new ClearAction())
                      .withAction(Action.just(Transformer.endOfChain()))
                      .infinite();
}

Check Invariants

We can also add invariants to our sequence checking property:

@Property
void checkMyStackWithInvariant(@ForAll("myStackActions") ActionChain<MyStringStack> chain) {
    chain
      .withInvariant("greater", stack -> assertThat(stack.size()).isGreaterThanOrEqualTo(0))
      .withInvariant("less", stack -> assertThat(stack.size()).isLessThan(5)) // Does not hold!
      .run();
}

If we first fix the bug in MyStringStack.clear() our property should eventually fail with the following result:

net.jqwik.engine.properties.state.InvariantFailedError:
  Invariant 'less' failed after the following actions: [
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)  
  ]
  final state: [AAAAA, AAAAA, AAAAA, AAAAA, AAAAA]
  Expecting actual:
    5
  to be less than:
    5

Rerunning Falsified Action Chains

As described in the chapter about rerunning falsified properties jqwik has different options for rerunning falsified properties.

Due to the fact that action chains are generated one action after the other, recreating the exact same sample of a chain is usually not possible. That’s why AfterFailureMode.SAMPLE_ONLY and AfterFailureMode.SAMPLE_FIRST will just start with the same random seed, which leads to the same sequence of chains, but not start with the last failing sample chain. A warning will be logged in such cases.

Stateful Testing (Old Approach)

As of version 1.7.0 jqwik comes with a new approach to stateful testing. What is described in this chapter will probably be deprecated in one of the next minor versions. Since both approaches have an interface called Action, be careful to import the right one!

Despite its bad reputation state is an important concept in object-oriented languages like Java. We often have to deal with stateful objects or components whose state can be changed through methods.

Thinking in a more formal way we can look at those objects as state machines and the methods as actions that move the object from one state to another. Some actions have preconditions to constrain when they can be invoked and some objects have invariants that should never be violated regardless of the sequence of performed actions.

To make this abstract concept concrete, let’s look at a simple stack implementation:

public class MyStringStack {
	public void push(String element) { ... }
	public String pop() { ... }
	public void clear() { ... }
	public boolean isEmpty() { ... }
	public int size() { ... }
	public String top() { ... }
}

Specify Actions

We can see at least three actions with their preconditions and expected state changes:

Check Postconditions

The fundamental property that jqwik should try to falsify is:

For any valid sequence of actions all required state changes
(aka postconditions) should be fulfilled.

We can formulate that quite easily as a jqwik property:

class MyStringStackProperties {

	@Property
	void checkMyStack(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
		actions.run(new MyStringStack());
	}

	@Provide
	Arbitrary<ActionSequence<MyStringStack>> sequences() {
		return Arbitraries.sequences(Arbitraries.oneOf(push(), pop(), clear()));
	}

	private Arbitrary<Action<MyStringStack>> push() {
		return Arbitraries.strings().alpha().ofLength(5).map(PushAction::new);
	}

	private Arbitrary<Action<MyStringStack>> clear() {
		return Arbitraries.just(new ClearAction());
	}

	private Arbitrary<Action<MyStringStack>> pop() {
		return Arbitraries.just(new PopAction());
	}
}

The interesting API elements are

To give jqwik something to falsify, we broke the implementation of clear() so that it won’t clear everything if there are more than two elements on the stack:

public void clear() {
    // Wrong implementation to provoke falsification for stacks with more than 2 elements
    if (elements.size() > 2) {
        elements.remove(0);
    } else {
        elements.clear();
    }
}

Running the property should now produce a result similar to:

org.opentest4j.AssertionFailedError: 
  Run failed after following actions:
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      clear
    final state: ["AAAAA", "AAAAA"]

Number of actions

jqwik will vary the number of generated actions according to the number of tries of your property. For the default of 1000 tries a sequence will have 32 actions. If need be you can specify the number of actions to generate using either the fluent interface or the @Size annotation:

@Property
// check stack with sequences of 7 actions:
void checkMyStack(@ForAll("sequences") @Size(7) ActionSequence<MyStringStack> actions) {
    actions.run(new MyStringStack());
}

The minimum number of generated actions in a sequence is 1 since checking an empty sequence does not make sense.

Check Invariants

We can also add invariants to our sequence checking property:

@Property
void checkMyStackWithInvariant(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
    actions
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isGreaterThanOrEqualTo(0))
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isLessThan(5))
        .run(new MyStringStack());
}

If we first fix the bug in MyStringStack.clear() our property should eventually fail with the following result:

org.opentest4j.AssertionFailedError: 
  Run failed after following actions:
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
      push(AAAAA)
    final state: ["AAAAA", "AAAAA", "AAAAA", "AAAAA", "AAAAA"]

Assumptions

If you want to constrain the set of generated values in a way that embraces more than one parameter, filtering does not work. What you can do instead is putting one or more assumptions at the beginning of your property.

The following property works only on strings that are not equal:

@Property
boolean comparingUnequalStrings(
        @ForAll @StringLength(min = 1, max = 10) String string1,
        @ForAll @StringLength(min = 1, max = 10) String string2
) {
    Assume.that(!string1.equals(string2));

    return string1.compareTo(string2) != 0;
}

This is a reasonable use of Assume.that(boolean condition) because most generated value sets will pass through.

Have a look at a seemingly similar example:

@Property
boolean findingContainedStrings(
        @ForAll @StringLength(min = 1, max = 10) String container,
        @ForAll @StringLength(min = 1, max = 5) String contained
) {
    Assume.that(container.contains(contained));

    return container.indexOf(contained) >= 0;
}

Despite the fact that the property condition itself is correct, the property will most likely fail with the following message:

org.opentest4j.AssertionFailedError: 
    Property [findingContainedStrings] exhausted after [1000] tries and [980] rejections

tries = 1000 
checks = 20 
generation = RANDOMIZED
after-failure = SAMPLE_FIRST
when-fixed-seed = ALLOW
edge-cases#mode = MIXIN 
seed = 1066117555581106850

The problem is that - given a random generation of two strings - only in very few cases one string will be contained in the other. jqwik will report a property as exhausted if the ratio between generated and accepted parameters is higher than 5. You can change the maximum discard ratio by specifying a parameter maxDiscardRatio in the @Property annotation. That’s why changing to @Property(maxDiscardRatio = 100) in the previous example will probably result in a successful property run, even though only a handful cases - of 1000 generated - will actually be checked.

In many cases turning up the accepted discard ration is a bad idea. With some creativity we can often avoid the problem by generating out test data a bit differently. Look at this variant of the above property, which also uses Assume.that() but with a much lower discard ratio:

@Property
boolean findingContainedStrings_variant(
        @ForAll @StringLength(min = 5, max = 10) String container,
        @ForAll @IntRange(min = 1, max = 5) int length,
        @ForAll @IntRange(min = 0, max = 9) int startIndex
) {
    Assume.that((length + startIndex) <= container.length());

    String contained = container.substring(startIndex, startIndex + length);
    return container.indexOf(contained) >= 0;
}

Result Shrinking

If a property could be falsified with a generated set of values, jqwik will try to “shrink” this sample in order to find a “smaller” sample that also falsifies the property.

Try this property:

@Property
boolean stringShouldBeShrunkToAA(@ForAll @AlphaChars String aString) {
    return aString.length() > 5 || aString.length() < 2;
}

The test run result should look something like:

AssertionFailedError: Property [stringShouldBeShrunkToAA] falsified with sample {0="aa"}

tries = 38 
checks = 38 
...
Shrunk Sample (5 steps)
-------------------------
  aString: "AA"

Original Sample
---------------
  aString: "RzZ"

In this case the original sample could be any string between 2 and 5 chars, whereas the final sample should be exactly AA since this is the shortest failing string and A has the lowest numeric value of all allowed characters.

Integrated Shrinking

jqwik’s shrinking approach is called integrated shrinking, as opposed to type-based shrinking which most property-based testing tools use. The general idea and its advantages are explained here.

Consider a somewhat more complicated example:

@Property
boolean shrinkingCanTakeAWhile(@ForAll("first") String first, @ForAll("second") String second) {
    String aString = first + second;
    return aString.length() > 5 || aString.length() < 2;
}

@Provide
Arbitrary<String> first() {
    return Arbitraries.strings()
        .withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(10)
        .filter(string -> string.endsWith("h"));
}

@Provide
Arbitrary<String> second() {
    return Arbitraries.strings()
        .withCharRange('0', '9')
        .ofMinLength(0).ofMaxLength(10)
        .filter(string -> string.length() >= 1);
}

Shrinking still works, although there’s quite a bit of filtering and string concatenation happening:

AssertionFailedError: Property [shrinkingCanTakeLong] falsified with sample {0="a", 1="000"}}

checks = 20 
tries = 20 
...
Shrunk Sample (3 steps)
-----------------------
  first: "a"
  second: "000"

Original Sample
---------------
  first: "h"
  second: "901"

This example also shows that sometimes there is no single “smallest example”. Depending on the starting random seed, this property will shrink to either {0="a", 1="000"}, {0="ah", 1="00"} or {0="aah", 1="0"}, all of which are considered to be the smallest possible for jqwik’s current way of measuring a sample’s size.

Switch Shrinking Off

Sometimes shrinking takes a really long time or won’t finish at all (usually a jqwik bug!). In those cases you can switch shrinking off for an individual property:

@Property(shrinking = ShrinkingMode.OFF)
void aPropertyWithLongShrinkingTimes(
	@ForAll List<Set<String>> list1, 
	@ForAll List<Set<String>> list2
) {	... }

Switch Shrinking to Full Mode

Sometimes you can find a message like

shrinking bound reached = after 1000 steps.

in your testrun’s output. This happens in rare cases when jqwik has not found the end of its search for simpler falsifiable values after 1000 iterations. In those cases you can try

@Property(shrinking = ShrinkingMode.FULL)

to tell jqwik to go all the way, even if it takes a million steps, even if it never ends…

Change the Shrinking Target

By default shrinking of numbers will move towards zero (0). If zero is outside the bounds of generation the closest number to zero - either the min or max value - is used as a target for shrinking. There are cases, however, when you’d like jqwik to choose a different shrinking target, usually when the default value of a number is not 0.

Consider generating signals with a standard frequency of 50 hz that can vary by plus/minus 5 hz. If possible, shrinking of falsified scenarios should move towards the standard frequency. Here’s how the provider method might look:

@Provide
Arbitrary<List<Signal>> signals() {
	Arbitrary<Long> frequencies = 
	    Arbitraries
            .longs()
            .between(45, 55)
            .shrinkTowards(50);

	return frequencies.map(f -> Signal.withFrequency(f)).list().ofMaxSize(1000);
}

Currently shrinking targets are supported for all number types.

Collecting and Reporting Statistics

In many situations you’d like to know if jqwik will really generate the kind of values you expect and if the frequency and distribution of certain value classes meets your testing needs. Statistics.collect() is made for this exact purpose.

In the most simple case you’d like to know how often a certain value is being generated:

@Property
void simpleStats(@ForAll RoundingMode mode) {
    Statistics.collect(mode);
}

will create an output similar to that:

[MyTest:simpleStats] (1000) statistics = 
    FLOOR       (158) : 16 %
    HALF_EVEN   (135) : 14 %
    DOWN        (126) : 13 %
    UP          (120) : 12 %
    HALF_UP     (118) : 12 %
    CEILING     (117) : 12 %
    UNNECESSARY (117) : 12 %
    HALF_DOWN   (109) : 11 %

More typical is the case in which you’ll classify generated values into two or more groups:

@Property
void integerStats(@ForAll int anInt) {
    Statistics.collect(anInt > 0 ? "positive" : "negative");
}

[MyTest:integerStats] (1000) statistics = 
    negative (506) : 51 %
    positive (494) : 49 %

You can also collect the distribution in more than one category and combine those categories:

@Property
void combinedIntegerStats(@ForAll int anInt) {
    String posOrNeg = anInt > 0 ? "positive" : "negative";
    String evenOrOdd = anInt % 2 == 0 ? "even" : "odd";
    String bigOrSmall = Math.abs(anInt) > 50 ? "big" : "small";
    Statistics.collect(posOrNeg, evenOrOdd, bigOrSmall);
}

[MyTest:combinedIntegerStats] (1000) statistics = 
    negative even big   (222) : 22 %
    positive even big   (201) : 20 %
    positive odd big    (200) : 20 %
    negative odd big    (194) : 19 %
    negative even small ( 70) :  7 %
    positive odd small  ( 42) :  4 %
    negative odd small  ( 38) :  4 %
    positive even small ( 33) :  3 %

And, of course, you can combine different generated parameters into one statistical group:

@Property
void twoParameterStats(
    @ForAll @Size(min = 1, max = 10) List<Integer> aList,
    @ForAll @IntRange(min = 0, max = 10) int index
) {
    Statistics.collect(aList.size() > index ? "index within size" : null);
}

[MyTest:twoParameterStats] (1000) statistics = 
    index within size (507) : 51 %

As you can see, collected null values are not being reported.

Here are a couple of examples to try out.

Labeled Statistics

If you want more than one statistic in a single property, you must give them labels for differentiation:

@Property
void labeledStatistics(@ForAll @IntRange(min = 1, max = 10) Integer anInt) {
    String range = anInt < 3 ? "small" : "large";
    Statistics.label("range").collect(range);
    Statistics.label("value").collect(anInt);
}

produces the following reports:

[MyTest:labeledStatistics] (1000) range = 
    large (783) : 78 %
    small (217) : 22 %

[MyTest:labeledStatistics] (1000) value = 
    1  (115) : 12 %
    5  (109) : 11 %
    10 (105) : 11 %
    4  (103) : 10 %
    2  (102) : 10 %
    3  ( 99) : 10 %
    6  ( 97) : 10 %
    8  ( 92) :  9 %
    7  ( 91) :  9 %
    9  ( 87) :  9 %

Statistics Report Formatting

There is a @StatisticsReport annotation that allows to change statistics report formats or to even switch it off. The annotation can be used on property methods or on container classes.

The value attribute is of type StatisticsReportMode.OFF and can have one of:

When using labeled statistics you can set mode and format for each label individually by using the annotation attribute @StatisticsReport.value.

Switch Statistics Reporting Off

You can switch off statistics report as simple as that:

@Property
@StatisticsReport(StatisticsReport.StatisticsReportMode.OFF)
void queryStatistics(@ForAll int anInt) {
    Statistics.collect(anInt);
}

Or you can just switch it off for properties that do not fail:

@Property
@StatisticsReport(onFailureOnly = true)
void queryStatistics(@ForAll int anInt) {
    Statistics.collect(anInt);
}

Histograms

jqwik comes with two report formats to display collected data as histograms: Histogram and NumberRangeHistogram.

Histogram displays the collected raw data as a histogram:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = Histogram.class)
void integers(@ForAll("gaussians") int aNumber) {
    Statistics.collect(aNumber);
}

@Provide
Arbitrary<Integer> gaussians() {
    return Arbitraries
            .integers()
            .between(0, 20)
            .shrinkTowards(10)
            .withDistribution(RandomDistribution.gaussian());
}

[HistogramExamples:integers] (1000) statistics = 
       # | label | count | 
    -----|-------|-------|---------------------------------------------------------------------------------
       0 |     0 |    13 | ■■■■
       1 |     1 |    13 | ■■■■
       2 |     2 |    15 | ■■■■■
       3 |     3 |     6 | ■■
       4 |     4 |    10 | ■■■
       5 |     5 |    22 | ■■■■■■■
       6 |     6 |    49 | ■■■■■■■■■■■■■■■■
       7 |     7 |    60 | ■■■■■■■■■■■■■■■■■■■■
       8 |     8 |   102 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
       9 |     9 |   100 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |    10 |   233 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    11 |   114 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      12 |    12 |    74 | ■■■■■■■■■■■■■■■■■■■■■■■■■
      13 |    13 |    64 | ■■■■■■■■■■■■■■■■■■■■■
      14 |    14 |    43 | ■■■■■■■■■■■■■■
      15 |    15 |    32 | ■■■■■■■■■■
      16 |    16 |    16 | ■■■■■
      17 |    17 |     8 | ■■
      18 |    18 |     7 | ■■
      19 |    20 |    19 | ■■■■■■

NumberRangeHistogram clusters the collected raw data into ranges:

@Property(generation = GenerationMode.RANDOMIZED)
@StatisticsReport(format = NumberRangeHistogram.class)
void integersInRanges(@ForAll @IntRange(min = -1000, max = 1000) int aNumber) {
    Statistics.collect(aNumber);
}

[HistogramExamples:integersInRanges] (1000) statistics = 
       # |         label | count | 
    -----|---------------|-------|---------------------------------------------------------------------------------
       0 | [-1000..-900[ |    20 | ■■■■■
       1 |  [-900..-800[ |    17 | ■■■■
       2 |  [-800..-700[ |    16 | ■■■■
       3 |  [-700..-600[ |     8 | ■■
       4 |  [-600..-500[ |    12 | ■■■
       5 |  [-500..-400[ |    14 | ■■■
       6 |  [-400..-300[ |    17 | ■■■■
       7 |  [-300..-200[ |    46 | ■■■■■■■■■■■
       8 |  [-200..-100[ |    59 | ■■■■■■■■■■■■■■
       9 |     [-100..0[ |   315 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      10 |      [0..100[ |   276 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
      11 |    [100..200[ |    47 | ■■■■■■■■■■■
      12 |    [200..300[ |    49 | ■■■■■■■■■■■■
      13 |    [300..400[ |    25 | ■■■■■■
      14 |    [400..500[ |    14 | ■■■
      15 |    [500..600[ |    13 | ■■■
      16 |    [600..700[ |    15 | ■■■
      17 |    [700..800[ |    14 | ■■■
      18 |    [800..900[ |    11 | ■■
      19 |   [900..1000] |    12 | ■■■

Both types can be subclassed to override behaviour like the number of buckets, the maximum drawing range of the bar, the order of elements, the label of a bucket and the header of the label column.

Make Your Own Statistics Report Format

In order to format statistics to your own liking you have to create an implementation of type StatisticsReportFormat and

@Property
@StatisticsReport(format = MyStatisticsFormat.class)
void statisticsWithHandMadeFormat(@ForAll Integer anInt) {
    String range = anInt < 0 ? "negative" : anInt > 0 ? "positive" : "zero";
    Statistics.collect(range);
}

class MyStatisticsFormat implements StatisticsReportFormat {
    @Override
    public List<String> formatReport(List<StatisticsEntry> entries) {
        return entries.stream()
    	              .map(e -> String.format("%s: %d", e.name(), e.count()))
    	              .collect(Collectors.toList());
    }
}

Running this property should produce a report similar to that:

[StatisticsExamples:statisticsWithHandMadeFormat] (1000) statistics = 
    negative: 520
    positive: 450
    zero: 30

Checking Coverage of Collected Statistics

Just looking at the statistics of generated values might not be sufficient. Sometimes you want to make sure that certain scenarios are being covered by your generators and fail a property otherwise. In jqwik you do that by first collecting statistics and then specifying coverage conditions for those statistics.

Check Percentages and Counts

The following example does that for generated values of enum RoundingMode:

@Property(generation = GenerationMode.RANDOMIZED)
void simpleStats(@ForAll RoundingMode mode) {
    Statistics.collect(mode);

    Statistics.coverage(coverage -> {
        coverage.check(RoundingMode.CEILING).percentage(p -> p > 5.0);
        coverage.check(RoundingMode.FLOOR).count(c -> c > 2);
    });
}

The same thing is possible for values collected with a specific label and in a fluent API style.

@Property(generation = GenerationMode.RANDOMIZED)
void labeledStatistics(@ForAll @IntRange(min = 1, max = 10) Integer anInt) {
    String range = anInt < 3 ? "small" : "large";
	
    Statistics.label("range")
              .collect(range)
              .coverage(coverage -> coverage.check("small").percentage(p -> p > 20.0));
    Statistics.label("value")
              .collect(anInt)
              .coverage(coverage -> coverage.check(0).count(c -> c > 0));
}

Start by looking at Statistics.coverage() to see all the options you have for checking percentages and counts.

Check Ad-hoc Query Coverage

Instead of classifying values at collection time you have the possibility to collect the raw data and use a query when doing coverage checking:

@Property
@StatisticsReport(StatisticsReport.StatisticsReportMode.OFF)
void queryStatistics(@ForAll int anInt) {
    Statistics.collect(anInt);
	
    Statistics.coverage(coverage -> {
    Predicate<List<Integer>> isZero = params -> params.get(0) == 0;
        coverage.checkQuery(isZero).percentage(p -> p > 5.0);
    });
}

In those cases you probably want to switch off reporting, otherwise the reports might get very long - and without informative value.

Check Coverage of Regex Pattern

Another option - similar to ad-hoc querying - is the possibility to check coverage of a regular expression pattern:

@Property
@StatisticsReport(StatisticsReport.StatisticsReportMode.OFF)
void patternStatistics(@ForAll @NumericChars String aString) {
    Statistics.collect(aString);
	
    Statistics.coverage(coverage -> {
        coverage.checkPattern("0.*").percentage(p -> p >= 10.0);
    });
}

Mind that only single values of type CharSequence, which includes String, can be checked against a pattern. All other types will not match the pattern.

Providing Default Arbitraries

Sometimes you want to use a certain, self-made Arbitrary for one of your own domain classes, in all of your properties, and without having to add @Provide method to all test classes. jqwik enables this feature by using Java’s java.util.ServiceLoader mechanism. All you have to do is:

jqwik will then add an instance of your arbitrary provider into the list of its default providers. Those default providers are considered for every test parameter annotated with @ForAll that has no explicit value. By using this mechanism you can also replace the default providers packaged into jqwik.

Simple Arbitrary Providers

A simple provider is one that delivers arbitraries for types without type variables. Consider the class Money:

public class Money {
	public BigDecimal getAmount() {
		return amount;
	}

	public String getCurrency() {
		return currency;
	}

	public Money(BigDecimal amount, String currency) {
		this.amount = amount;
		this.currency = currency;
	}

	public Money times(int factor) {
		return new Money(amount.multiply(new BigDecimal(factor)), currency);
	}
}

If you register the following class MoneyArbitraryProvider:

package my.own.provider;

public class MoneyArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Money.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		Arbitrary<BigDecimal> amount = Arbitraries.bigDecimals()
				  .between(BigDecimal.ZERO, new BigDecimal(1_000_000_000))
				  .ofScale(2);
		Arbitrary<String> currency = Arbitraries.of("EUR", "USD", "CHF");
		return Collections.singleton(Combinators.combine(amount, currency).as(Money::new));
	}
}

in file META-INF/services/net.jqwik.api.providers.ArbitraryProvider with such an entry:

my.own.provider.MoneyArbitraryProvider

The following property will run without further ado - regardless the class you put it in:

@Property
void moneyCanBeMultiplied(@ForAll Money money) {
    Money times2 = money.times(2);
    Assertions.assertThat(times2.getCurrency()).isEqualTo(money.getCurrency());
    Assertions.assertThat(times2.getAmount())
        .isEqualTo(money.getAmount().multiply(new BigDecimal(2)));
}

Arbitrary Providers for Parameterized Types

Providing arbitraries for generic types requires a little bit more effort since you have to create arbitraries for the “inner” types as well. Let’s have a look at the default provider for java.util.Optional<T>:

public class OptionalArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Optional.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		TypeUsage innerType = targetType.getTypeArguments().get(0);
		return subtypeProvider.apply(innerType).stream()
			.map(Arbitrary::optional)
			.collect(Collectors.toSet());
	}
}

Mind that provideFor returns a set of potential arbitraries. That’s necessary because the subtypeProvider might also deliver a choice of subtype arbitraries. Not too difficult, is it?

Arbitrary Provider Priority

When more than one provider is suitable for a given type, jqwik will randomly choose between all available options. That’s why you’ll have to take additional measures if you want to replace an already registered provider. The trick is to override a provider’s priority() method that returns 0 by default:

public class AlternativeStringArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isAssignableFrom(String.class);
	}

	@Override
	public int priority() {
		return 1;
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		return Collections.singleton(Arbitraries.just("A String"));
	}
}

If you register this class as arbitrary provider any @ForAll String will be resolved to "A String".

Create your own Annotations for Arbitrary Configuration

All you can do to constrain default parameter generation is adding another annotation to a parameter or its parameter types. What if the existing annotations do not suffice for your needs? Is there a way to enhance the set of constraint annotations? Yes, there is!

The mechanism you can plug into is similar to what you do when providing your own default arbitrary providers. That means:

  1. Create an implementation of an interface, in this case ArbitraryConfigurator.
  2. Register the implementation using using Java’s java.util.ServiceLoader mechanism.

jQwik will then call your implementation for every parameter that is annotated with @ForAll and any additional annotation. That also means that parameters without an additional annotation will not and cannot be affected by a configurator.

Arbitrary Configuration Example: @Odd

To demonstrate the idea let’s create an annotation @Odd which will constrain any integer generation to only generate odd numbers. First things first, so here’s the @Odd annotation:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER, ElementType.TYPE_USE })
@Retention(RetentionPolicy.RUNTIME)
public @interface Odd {
}

On its own this annotation will not have any effect. You also need an arbitrary configurator:

public class OddConfigurator implements ArbitraryConfigurator {

    @Override
    public <T> Arbitrary<T> configure(Arbitrary<T> arbitrary, TypeUsage targetType) {
        if (!targetType.isOfType(Integer.class) && !targetType.isOfType(int.class)) {
            return arbitrary;
        }
        return targetType.findAnnotation(Odd.class)
                         .map(odd -> arbitrary.filter(number -> Math.abs(((Integer) number) % 2) == 1))
                         .orElse(arbitrary);
    }
}

If you now register the implementation, the following example will work:

@Property
boolean oddIntegersOnly(@ForAll @Odd int aNumber) {
    return Math.abs(aNumber % 2) == 1;
}

However, this implementation is rather onerous since it has to check for supported target types and annotation; it also has to cast the arbitrary’s value to Integer. Let’s see if we can do better.

Using Arbitrary Configurator Base Class

Deriving your implementation from ArbitraryConfiguratorBase will largely simplify things:

public class OddConfigurator extends ArbitraryConfiguratorBase {

	public Arbitrary<Integer> configure(Arbitrary<Integer> arbitrary, Odd odd) {
		return arbitrary.filter(number -> Math.abs(number % 2) == 1);
	}
}

The nice thing about ArbitraryConfiguratorBase is that it will take care of all the boilerplate code for you. It will check for supported target types and annotations and will also that the actual arbitrary can be cast to the arbitrary parameter type. Your subclass can have any number of configure methods under the following constraints:

With this knowledge we can easily enhance our OddConfigurator to also support BigInteger values:

public class OddConfigurator extends ArbitraryConfiguratorBase {
    public Arbitrary<Integer> configureInteger(Arbitrary<Integer> arbitrary, Odd odd) {
        return arbitrary.filter(number -> Math.abs(number % 2) == 1);
    }

    public Arbitrary<BigInteger> configureBigInteger(Arbitrary<BigInteger> arbitrary, Odd odd) {
        return arbitrary.filter(number -> {
            return number.remainder(BigInteger.valueOf(2)).abs().equals(BigInteger.ONE);
        });
    }
}

You can combine @Odd with other annotations like @Positive or @Range or another self-made configurator. In this case the order of configurator application might play a role, which can be influenced by overriding the order() method of a configurator.

Domain and Domain Context

Until now you have seen two ways to specify which arbitraries will be created for a given parameter:

In many cases both approaches can be tedious to set up or require constant repetition of the same annotation value. There’s another way that allows you to group a number of arbitrary providers (and also arbitrary configurators) in a single place, called a DomainContext and tell a property method or container to only use providers and configurators from those domain contexts that are explicitly stated in a @Domain(Class<? extends DomainContext>) annotation.

As for ways to implement domain context classes have a look at DomainContext and DomainContextBase.

In subclasses of DomainContextBase you have several options to specify arbitrary providers, arbitrary configurators and reporting formats:

A DomainContext implementation class can itself have @Domain annotations, which are then used to add to the property’s set of domains.

You can override method DomainContext.initialize(PropertyLifecycleContext context), which will be called once for each property to which this context is applied. Since the lifecycle of DomainContext instances is not specified, do not rely on storing or caching any information in member variables. Instead, use jqwik’s Storage Mechanism to persist data if needed.

Domain example: American Addresses

Let’s say that US postal addresses play a crucial role in the software that we’re developing. That’s why there are a couple of classes that represent important domain concepts: Street, State, City and Address. Since we have to generate instances of those classes for our properties, we collect all arbitrary provision code in one place:

public class AmericanAddresses extends DomainContextBase {

	@Provide
	Arbitrary<Street> streets() {
		Arbitrary<String> streetName = capitalizedWord(30);
		Arbitrary<String> streetType = Arbitraries.of("Street", "Avenue", "Road", "Boulevard");
		return Combinators.combine(streetName, streetType).as((n, t) -> n + " " + t).map(Street::new);
	}

	@Provide
	Arbitrary<Integer> streetNumbers() {
		return Arbitraries.integers().between(1, 999);
	}

	@Provide
	Arbitrary<State> states() {
		return Arbitraries.of(State.class);
	}

	@Provide
	Arbitrary<City> cities() {
		Arbitrary<String> name = capitalizedWord(25);
		Arbitrary<State> state = Arbitraries.defaultFor(State.class);
		Arbitrary<String> zip = Arbitraries.strings().numeric().ofLength(5);
		return Combinators.combine(name, state, zip).as(City::new);
	}

	@Provide
	Arbitrary<Address> addresses() {
		Arbitrary<Street> streets = Arbitraries.defaultFor(Street.class);
		Arbitrary<City> cities = Arbitraries.defaultFor(City.class);
		return Combinators.combine(streets, streetNumbers(), cities).as(Address::new);
	}

	private Arbitrary<String> capitalizedWord(int maxLength) {
		Arbitrary<Character> capital = Arbitraries.chars().range('A', 'Z');
		Arbitrary<String> rest = Arbitraries.strings().withCharRange('a', 'z').ofMinLength(1).ofMaxLength(maxLength - 1);
		return Combinators.combine(capital, rest).as((c, r) -> c + r);
	}
}

Now it’s rather easy to use the arbitraries provided therein for your properties:

class AddressProperties {

	@Property
	@Domain(AmericanAddresses.class)
	void anAddressWithAStreetNumber(@ForAll Address anAddress, @ForAll int streetNumber) {
	}

	@Property
	@Domain(AmericanAddresses.class)
	void globalDomainIsNotPresent(@ForAll Address anAddress, @ForAll String anyString) {
	}

	@Property
	@Domain(DomainContext.Global.class)
	@Domain(AmericanAddresses.class)
	void globalDomainCanBeAdded(@ForAll Address anAddress, @ForAll String anyString) {
	}
}

The first two properties above will resolve their arbitraries solely through providers specified in AmericanAddresses, whereas the last one also uses the default (global) context. Keep in mind that the inner part of the return type Arbitrary<InnerPart> is used to determine the applicability of a provider method. It’s being used in a covariant way, i.e., Arbitrary<String> is also applicable for parameter @ForAll CharSequence charSequence.

Since AmericanAddresses does not configure any arbitrary provider for String parameters, the property method globalDomainIsNotPresent(..) will fail, whereas globalDomainCanBeAdded(..) will succeed because it has the additional @Domain(DomainContext.Global.class) annotation. You could also add this line to class AmericanAddresses itself, which would then automatically bring the global context to all users of this domain class:

@Domain(DomainContext.Global.class)
public class AmericanAddresses extends DomainContextBase {
   ...
}

The reason that you have to jump through these hoops is that domains are conceived to give you perfect control about how objects of a certain application domain are being created. That means, that by default they do not inherit the global context.

Generation from a Type’s Interface

Some domain classes are mostly data holders. They come with constructors or factory methods to create them and you might want to create different instances by “just” filling the constructors’ parameters with values that are themselves generated. Using the building blocks you’ve seen until now requires the use of Arbitrary.map() or even Combinators.combine(...).as(...) to invoke the relevant constructor(s) and/or factories yourself. There’s a simpler way, though…

Consider a simple Person class:

public class Person {

	private final String name;
	private final int age;

	public Person(String name, int age) {
		if (name == null || name.trim().isEmpty())
			throw new IllegalArgumentException();
		if (age < 0 || age > 130)
			throw new IllegalArgumentException();

		this.name = name;
		this.age = age;
	}

	@Override
	public String toString() {
		return String.format("%s (%d)", name, age);
	}
}

A first step to use arbitrarily generated Person objects without having to write a lot of jqwik-specific boiler plate code could look like that:

@Property
void aPersonsIsAlwaysValid(@ForAll @UseType Person aPerson) {
    Assertions.assertThat(aPerson.name).isNotBlank();
    Assertions.assertThat(aPerson.age).isBetween(0, 130);
}

Notice the annotation @UseType which tells jqwik to use the type information of Person to generate it. By default the framework will use all public constructors and all public, static factory methods in the class in order to generate instances. Whenever there’s an exception during generation they will be ignored; that way you’ll only get valid instances.

While the creation of a Person instance requires only basic Java types - String and int - that already have default arbitraries available, @UseType is also applied to nested types without default generators. That’s why class Party:

public class Party {

	final String name;
	final Set<Person> people;

	public Party(String name, Set<Person> people) {
		this.name = name;
		this.people = people;
	}
}

can also be generated in the same way:

@Property
void aPartyOfPeopleCanBeGenerated(@ForAll @UseType Party aParty) {
	Assertions.assertThat(aParty.name).isNotBlank();
	Assertions.assertThat(aParty.people).allMatch(
		person -> !person.name.isEmpty()
	);
}

This recursive application of @UseType is switched on by default, but can also be switched off: @UseType(enableRecursion=false).

To learn about all configuration options have a look at the complete example and check the following api entry points:

Generation of Edge Cases

It’s well-known that many programming bugs and specification gaps happen at the border of allowed value ranges. For example, in the domain of integer numbers the minimum (Integer.MIN_VALUE) and maximum (Integer.MAX_VALUE) belong in the set of those edge cases. Many people use the term a bit more loosely and also include other special values that tend to have a higher chance of revealing implementation problems, like 0 for numbers or an empty string.

jqwik has special treatment for edge cases. Most base type arbitraries come with their set of edge cases. You can find out about edge cases by asking an arbitrary about it. Run the following example

@Example
void printEdgeCases() {
    System.out.println(Arbitraries.integers().edgeCases());
    System.out.println(Arbitraries.strings().withCharRange('a', 'z').edgeCases());
    System.out.println(Arbitraries.floats().list().edgeCases());
}

and you will see this output:

EdgeCases[-2, -1, 0, 2, 1, -2147483648, 2147483647, 2147483647, 2147483646]
EdgeCases["a", "z", ""]
EdgeCases[[], [0.0], [1.0], [-1.0], [0.01], [-0.01], [-3.4028235E38], [3.4028235E38]]

You may notice that edge cases are not just hard-coded values but also make use of underlying arbitraries’ edge cases to arrive at new ones. That’s why a list of floats arbitrary has single element lists of floats as edge cases. Edge cases are also being combined and permuted when Combinators are used. Also, most methods from Arbitrary - like map(), filter() and flatMap() - provide sensible edge cases behaviour. Thus, your self-made domain-specific arbitraries get edge cases automatically.

jqwik makes use of edge cases in two ways:

  1. Whenever an arbitrary is asked to produce a value it will mix-in edge cases from time to time.
  2. By default jqwik will mix the combination of permutations of edge cases of a property’s parameters with purely randomized generation of parameters. You can even try all edge case combinations first as the next property shows.
@Property(edgeCases = EdgeCasesMode.FIRST)
void combinedEdgeCasesOfTwoParameters(
    @ForAll List<Integer> intList,
    @ForAll @IntRange(min = -100, max = 0) int anInt
) {
    String parameters = String.format("%s, %s", intList, anInt);
    System.out.println(parameters);
}

Run it and have a look at the output.

Configuring Edge Case Injection

How jqwik handles edge cases generation can be controlled with an annotation property and a configuration parameter.

To switch it off for a single property, use:

@Property(edgeCases = EdgeCasesMode.NONE)
void combinedEdgeCasesOfTwoParameters(
    @ForAll List<Integer> intList,
    @ForAll @IntRange(min = -100, max = 0) int anInt
) {
    // whatever you do   
}

If you want to suppress edge case generation for a single arbitrary that’s also possible: Just use Arbitrary.withoutEdgeCases(). Running the following property will regularly create empty lists - because - this is one of the default list edge cases, but it will not create integer values of 0, 1, -1 etc. with higher probability.

@Property
void noNumberEdgeCases(@ForAll List<@From("withoutEdgeCases") Integer> intList) {
  System.out.println(intList);
}

@Provide
Arbitrary<Integer> withoutEdgeCases() {
  return Arbitraries.integers().withoutEdgeCases();}

Configuring Edge Cases Themselves

Besides switching edge cases completely off, you can also filter some edge cases out, include only certain ones or add new ones. This is done through Arbitrary.edgeCases(config). Here’s an example that shows how to add a few “special” strings to a generator:

@Property(edgeCases = EdgeCasesMode.FIRST)
void stringsWithSpecialEdgeCases(@ForAll("withSpecialEdgeCases") String aString) {
  System.out.println(aString);
}

@Provide
Arbitrary<String> withSpecialEdgeCases() {
  return Arbitraries.strings()
          .alpha().ofMinLength(1).ofMaxLength(10)
          .edgeCases(stringConfig -> {
            stringConfig.add("hello", "hallo", "hi");
          });
}

The output should start with:

A
Z
a
z
hello
hallo
hi

followed by a lot of random strings.

Mind that the additional edge cases - in this case "hello", "hallo" and "hi" - are within the range of the underlying arbitrary. That means that they could also be generated randomly, albeit with a much lower probability.

Caveat: Values that are outside the range of the underlying arbitrary are not allowed as edge cases. For implementation reasons, arbitraries cannot warn you about forbidden values, and the resulting behaviour is undefined.

If you want to add values from outside the range of the underlying arbitrary, use Arbitraries.oneOf() - and maybe combine it with explicit edge case configuration:

@Provide
Arbitrary<String> withSpecialEdgeCases() {
    StringArbitrary strings = Arbitraries.strings().alpha().ofMinLength(1).ofMaxLength(10);
    return Arbitraries.oneOf(strings, Arbitraries.of("", "0", "1"))
                      .edgeCases(config -> config.add("", "0", "1")); // <-- To really raise the probability of these edge cases
}

Exhaustive Generation

Sometimes it is possible to run a property method with all possible value combinations. Consider the following example:

@Property
boolean allSquaresOnChessBoardExist(
    @ForAll @CharRange(from = 'a', to = 'h') char column,
    @ForAll @CharRange(from = '1', to = '8') char row
) {
    return new ChessBoard().square(column, row).isOnBoard();
}

The property is supposed to check that all valid squares in chess are present on a new chess board. If jqwik generated the values for column and row randomly, 1000 tries might or might not produce all 64 different combinations. Why not change strategies in cases like that and just iterate through all possible values?

This is exactly what jqwik will do:

Exhaustive generation is considered for:

Data-Driven Properties

In addition to the usual randomized generation of property parameters you have also the possibility to feed a property with preconceived or deterministically generated parameter sets. Why would you want to do that? One reason might be that you are aware of some problematic test cases but they are rare enough that jqwik’s randomization strategies don’t generate them (often enough). Another reason could be that you’d like to feed some properties with prerecorded data - maybe even from production. And last but not least there’s a chance that you want to check for a concrete result given a set of input parameters.

Feeding data into a property is quite simple:

@Data
Iterable<Tuple2<Integer, String>> fizzBuzzExamples() {
    return Table.of(
        Tuple.of(1, "1"),
        Tuple.of(3, "Fizz"),
        Tuple.of(5, "Buzz"),
        Tuple.of(15, "FizzBuzz")
    );
}

@Property
@FromData("fizzBuzzExamples")
void fizzBuzzWorks(@ForAll int index, @ForAll String result) {
    Assertions.assertThat(fizzBuzz(index)).isEqualTo(result);
}

All you have to do is annotate the property method with @FromData("dataProviderReference"). The method you reference must be annotated with @Data and return an object of type Iterable<? extends Tuple>. The Table class is just a convenient way to create such an object, but you can return any collection or create an implementation of your own.

Keep in mind that the Tuple subtype you choose must conform to the number of @ForAll parameters in your property method, e.g. Tuple.Tuple3 for a method with three parameters. Otherwise jqwik will fail the property and tell you that the provided data is inconsistent with the method’s parameters.

Data points are fed to the property in their provided order. The tries parameter of @Property will constrain the maximum data points being tried. Unlike parameterized tests in JUnit4 or Jupiter, jqwik will report only the first falsified data point. Thus, fixing the first failure might lead to another falsified data point later on. There is also no shrinking being done for data-driven properties since jqwik has no information about the constraints under which the external data was conceived or generated.

Rerunning Falsified Properties

When you rerun properties after they failed, they will - by default - use the previous random seed so that the next run will generate the exact same sequence of parameter data and thereby expose the same failing behaviour. This simplifies debugging and regression testing since it makes a property’s falsification stick until the problem has been fixed.

If you want to, you can change this behaviour for a given property like this:

@Property(afterFailure = AfterFailureMode.PREVIOUS_SEED)
void myProperty() { ... }

The afterFailure property can have one of four values:

You can also determine the default behaviour of all properties by setting the jqwik.failures.after.default parameter in the configuration file to one of those enum values.

jqwik Configuration

jqwik uses JUnit’s configuration parameters to configure itself.

The simplest form is a file junit-platform.properties in your classpath in which you can configure a few basic parameters:

jqwik.database = .jqwik-database             # The database file in which to store data of previous runs.
                                             # Set to empty to fully disable test run recording.
jqwik.tries.default = 1000                   # The default number of tries for each property
jqwik.maxdiscardratio.default = 5            # The default ratio before assumption misses make a property fail
jqwik.reporting.onlyfailures = false         # Set to true if only falsified properties should be reported
jqwik.reporting.usejunitplatform = false     # Set to true if you want to use platform reporting
jqwik.failures.runfirst = false              # Set to true if you want to run the failing tests from the previous run first
jqwik.failures.after.default = SAMPLE_FIRST  # Set default behaviour for falsified properties:
                                             # PREVIOUS_SEED, SAMPLE_ONLY, SAMPLE_FIRST or RANDOM_SEED
jqwik.generation.default = AUTO              # Set default behaviour for generation:
                                             # AUTO, RANDOMIZED, or EXHAUSTIVE
jqwik.edgecases.default = MIXIN              # Set default behaviour for edge cases generation:
                                             # FIRST, MIXIN, or NONE
jqwik.shrinking.default = BOUNDED            # Set default shrinking behaviour:
                                             # BOUNDED, FULL, or OFF
jqwik.shrinking.bounded.seconds = 10         # The maximum number of seconds to shrink if
                                             # shrinking behaviour is set to BOUNDED
jqwik.seeds.whenfixed = ALLOW                # How a test should act when a seed is fixed. Can set to ALLOW, WARN or FAIL
                                             # Useful to prevent accidental commits of fixed seeds into source control.

Besides the properties file there is also the possibility to set properties in Gradle or Maven Surefire.

Legacy Configuration in jqwik.properties File

Prior releases of jqwik used a custom jqwik.properties file. Since version 1.6.0 this is no longer supported.

Additional Modules

jqwik comes with a few additional modules:

Web Module

This module’s artefact name is jqwik-web. It’s supposed to provide arbitraries, default generation and annotations for web related types. Currently it supports the generation of

This module is part of jqwik’s default dependencies.

Email Address Generation

To generate email addresses you can either

An email address consists of two parts: local-part and host. The complete email address is therefore local-part@host. The local-part can be unquoted or quoted (in double quotes), which allows for more characters to be used. The host can be a standard domain name, but also an IP (v4 or v6) address, surrounded by square brackets [].

For example, valid email addresses are:

abc@example
abc@example.com
" "@example.example
"admin@server"@[192.168.201.0]
admin@[32::FF:aBc:79a:83B:FFFF:345]

By default, only addresses with unquoted local part and domain hosts are generated (e.g. me@myhost.com), because many - if not most - applications and web forms only accept those.

The @Email annotation comes with a few configuration attributes:

You can use it as follows:

@Property
void defaultEmailAddresses(@ForAll @Email String email) {
    assertThat(email).contains("@");
}

@Property
void restrictedEmailAddresses(@ForAll @Email(quotedLocalPart = true, ipv4Host = true) String email) {
    assertThat(email).contains("@");
}

Web Domain Generation

To generate web domain names

Here’s an example:

@Property
void topLevelDomainCannotHaveSingleLetter(@ForAll @WebDomain String domain) {
	int lastDot = domain.lastIndexOf('.');
	String tld = domain.substring(lastDot + 1);
	Assertions.assertThat(tld).hasSizeGreaterThan(1);
}

Time Module

This module’s artefact name is jqwik-time. It’s supposed to provide arbitraries, default generation and annotations for date and time types.

This module is part of jqwik’s default dependencies.

The module provides:

Generation of Dates

Default Generation of Dates

Default generation currently is supported for LocalDate, Year, YearMonth, DayOfWeek, MonthDay and Period. Here’s a small example:

@Property
void generateLocalDatesWithAnnotation(@ForAll @DateRange(min = "2019-01-01", max = "2020-12-31") LocalDate localDate) {
  assertThat(localDate).isBetween(
    LocalDate.of(2019, 1, 1),
    LocalDate.of(2020, 12, 31)
  );
}

The following annotations can be used to constrain default generation of the enumerated types:

@DateRange, @MonthDayRange, @YearMonthRange and @PeriodRange use the ISO format for date strings. Examples: 2013-05-25, --05-25, 2013-05 and P1Y2M15D.

Programmatic Generation of Dates

Programmatic generation of dates and date-related types always starts with a static method call on class Dates. For example:

@Property
void generateLocalDates(@ForAll("dates") LocalDate localDate) {
  assertThat(localDate).isAfter(LocalDate.of(2000, 12, 31));
}

@Provide
Arbitrary<LocalDate> dates() {
  return Dates.dates().atTheEarliest(LocalDate.of(2001, 1, 1));
}

Here’s the list of available methods:

LocalDateArbitrary
CalendarArbitrary
DateArbitrary
YearArbitrary
YearMonthArbitrary
MonthDayArbitrary
PeriodArbitrary

Generation of Times

Default Generation of Times

Default generation currently is supported for LocalTime, OffsetTime, ZoneOffset, TimeZone, ZoneId and Duration. Here’s a small example:

@Property
void generateLocalTimesWithAnnotation(@ForAll @TimeRange(min = "01:32:21", max = "03:49:32") LocalTime localTime) {
    assertThat(time).isAfterOrEqualTo(LocalTime.of(1, 32, 21));
    assertThat(time).isBeforeOrEqualTo(LocalTime.of(3, 49, 32));
}

The following annotations can be used to constrain default generation of the enumerated types:

@TimeRange, @OffsetRange and @DurationRange use the standard format of their classes. Examples:

Programmatic Generation of Times

Programmatic generation of times always starts with a static method call on class Times. For example:

@Property
void generateLocalTimes(@ForAll("times") LocalTime localTime) {
  assertThat(localTime).isAfter(LocalTime.of(13, 53, 21));
}

@Provide
Arbitrary<LocalTime> times() {
  return Times.times().atTheEarliest(LocalTime.of(13, 53, 22));
}

Here’s the list of available methods:

LocalTimeArbitrary
OffsetTimeArbitrary
ZoneOffsetArbitrary
DurationArbitrary

Generation of DateTimes

Default Generation of DateTimes

Default generation currently is supported for LocalDateTime, Instant, OffsetDateTime and ZonedDateTime. Here’s a small example:

@Property
void generateLocalDateTimesWithAnnotation(@ForAll @DateTimeRange(min = "2019-01-01T01:32:21", max = "2020-12-31T03:11:11") LocalDateTime localDateTime) {
  assertThat(localDateTime).isBetween(
    LocalDateTime.of(2019, 1, 1, 1, 32, 21),
    LocalDateTime.of(2020, 12, 31, 3, 11, 11)
  );
}

The following annotations can be used to constrain default generation of the enumerated types:

@DateTimeRange, @InstantRange, @DateRange, @TimeRange and @OffsetRange use the standard format of their classes. Examples:

Programmatic Generation of DateTimes

Programmatic generation of date times always starts with a static method call on class DateTimes. For example:

@Property
void generateLocalDateTimes(@ForAll("dateTimes") LocalDateTime localDateTime) {
  assertThat(localDateTime).isAfter(LocalDateTime.of(2013, 5, 25, 19, 48, 32));
}

@Provide
Arbitrary<LocalDateTime> dateTimes() {
  return DateTimes.dateTimes().atTheEarliest(LocalDateTime.of(2013, 5, 25, 19, 48, 33));
}

Here’s the list of available methods:

LocalDateTimeArbitrary
InstantArbitrary
OffsetDateTimeArbitrary
ZonedDateTimeArbitrary

Kotlin Module

This module’s artefact name is jqwik-kotlin. It’s supposed to simplify and streamline using jqwik in Kotlin projects.

This module is not in jqwik’s default dependencies. It’s usually added as a test-implementation dependency.

Adding this module will add dependencies on:

Table of contents:

Build Configuration for Kotlin

Apart from adding this module to the dependencies in test scope, there’s a few other things you should configure for a seamless jqwik experience in Kotlin:

Here’s the jqwik-related part of the Gradle build file for a Kotlin project:

dependencies {
    ...
    testImplementation("net.jqwik:jqwik-kotlin:1.9.1")
}

tasks.withType<Test>().configureEach {
    useJUnitPlatform()
}

tasks.withType<KotlinCompile> {
    compilerOptions {
        freeCompilerArgs = listOf(
            "-Xnullability-annotations=@org.jspecify.annotations:strict",
            "-Xemit-jvm-type-annotations" // Enable annotations on type variables
        )
        apiVersion = org.jetbrains.kotlin.gradle.dsl.KotlinVersion.KOTLIN_2_0
        languageVersion = org.jetbrains.kotlin.gradle.dsl.KotlinVersion.KOTLIN_2_0
        javaParameters = true // Get correct parameter names in jqwik reporting
    }
}

Differences to Java Usage

Kotlin is very compatible with Java, but a few things do not work or do not work as expected. Here are a few of those which I noticed to be relevant for jqwik:

Generation of Nullable Types

Top-level nullable Kotlin types are recognized, i.e., null’s will automatically be generated with a probability of 5%. If you want a different probability you have to use @WithNull(probability).

@Property
fun alsoGenerateNulls(@ForAll nullOrString: String?) {
    println(nullOrString)
}

Note that the detection of nullable types only works for top-level types. Using @ForAll list: List<String?> will not result in null values within the list. Nullability for type parameters must be explicitly set through @WithNull:

@Property(tries = 100)
fun generateNullsInList(@ForAll list: List<@WithNull String>) {
    println(list)
}

Support for Coroutines

In order to test regular suspend functions or coroutines this module offers two options:

suspend fun echo(string: String): String {
    delay(100)
    return string
}

@Property(tries = 10)
fun useRunBlocking(@ForAll s: String) = runBlockingProperty {
    assertThat(echo(s)).isEqualTo(s)
}

@Property(tries = 10)
fun useRunBlockingWithContext(@ForAll s: String) = runBlockingProperty(EmptyCoroutineContext) {
    assertThat(echo(s)).isEqualTo(s)
}

@Property(tries = 10)
suspend fun useSuspend(@ForAll s: String) {
    assertThat(echo(s)).isEqualTo(s)
}

Both variants do nothing more than starting the body of the property method asynchronously and waiting for all coroutines to finish. That means e.g. that delays will require the full amount of specified delay time.

If you need more control over the dispatchers to use or the handling of delays, you should consider using kotlinx.coroutines testing support. This will require to add a dependency on org.jetbrains.kotlinx:kotlinx-coroutines-test.

Support for Kotlin Collection Types

Kotlin has its own variations of collection types, e.g. (kotlin.collections.List and kotlin.collections.MutableList) that are - under the hood - instances of their corresponding, mutable Java type. Using those types in ForAll-parameters works as expected.

This is also true for

Support for Kotlin Functions

Kotlin provides a generic way to specify functional types, e.g. (String, String) -> Int specifies a function with two String parameters that returns an Int. You can use those function types as ForAll parameters:

@Property
fun generatedFunctionsAreStable(@ForAll aFunc: (String) -> Int) {
    assertThat(aFunc("hello")).isEqualTo(aFunc("hello"))
}

Moreover, there are a few top-level functions to create arbitraries for functional types: anyFunction0(Arbitrary<R>) … anyFunction4(Arbitrary<R>). Look at this code example:

@Property
fun providedFunctionsAreAlsoStable(@ForAll("myIntFuncs") aFunc: (String, String) -> Int) {
    assertThat(aFunc("a", "b")).isBetween(10, 1000)
}

@Provide
fun myIntFuncs(): Arbitrary<(String, String) -> Int> = anyFunction2(Int.any(10..1000))

Supported Kotlin-only Types

The Kotlin standard library comes with a lot of types that don’t have an equivalent in the JDK. Some of them are already supported directly:

IntRange
Sequence<T>

jqwik will never create infinite sequences by itself.

Pair<A, B>
Triple<A, B, C>

Kotlin Singletons

Since Kotlin object <name> {...} definitions are compiled to Java classes there is no reason that they cannot be used as test containers. As a matter of fact, this module adds a special feature on top: Using object instead of class will always use the same instance for running all properties and examples:

// One of the examples will fail
object KotlinSingletonExample {

    var lastExample: String = ""

    @Example
    fun example1() {
        Assertions.assertThat(lastExample).isEmpty()
        lastExample = "example1"
    }

    @Example
    fun example2() {
        Assertions.assertThat(lastExample).isEmpty()
        lastExample = "example2"
    }
}

Mind that IntelliJ in its current version does not mark functions in object definitions as runnable test functions. You can, however, run an object-based test container from the project view and the test runner itself.

Convenience Functions for Kotlin

Some parts of the jqwik API are hard to use in Kotlin. That’s why this module offers a few extension functions and top-level functions to ease the pain.

Extensions for Built-in Kotlin Types
Arbitrary Extensions
SizableArbitrary Extensions

In addition to ofMinSize(..) and ofMaxSize(..) all sizable arbitraries can now be configured using ofSize(min..max).

StringArbitrary Extensions

In addition to ofMinLength(..) and ofMaxLength(..) a StringArbitrary can now be configured using ofLength(min..max).

Jqwik Tuples in Kotlin

Jqwik’s Tuple types can be used to assign multi values, e.g.

val (a, b) = Tuple.of(1, 2)

will assign 1 to variable a and 2 to variable b.

Type-based Arbitraries

Getting a type-based generator using the Java API looks a bit awkward in Kotlin: Arbitraries.forType(MyType::class.java). There’s a more Kotlinish way to do the same: anyForType<MyType>().

anyForType<MyType>() is limited to concrete classes. For example, it cannot handle sealed class or interface by looking for sealed subtypes. anyForSubtypeOf<MyInterface>() exists for such situation.

sealed interface Character
sealed interface Hero : Character
class Knight(val name: String) : Hero
class Wizard(val name: String) : Character

val arbitrary =  anyForSubtypeOf<Character>()

In the previous example, the created arbitrary provides arbitrarily any instances of Knight or Wizard. The arbitrary is recursively based on any sealed subtype. Under the hood, it uses anyForType<Subtype>() for each subtype. However, this can be customized subtype by subtype, by providing a custom arbitrary:

anyForSubtypeOf<Character> {
    provide<Wizard> { Arbitraries.of(Wizard("Merlin"),Wizard("Élias de Kelliwic’h")) }
}

More over, like anyForType<>(), anyForSubtypeOf<>() can be applied recursively (default is false): anyForSubtypeOf<SealedClass>(enableArbitraryRecursion = true).

Diverse Convenience Functions
Combinator DSL

The combinator DSL provides another, potentially more convenient, wrapper for Combinators.combine.

Instead of a list of arguments, it uses kotlin’s property delegates to refer to the arbitraries that are being combined:

combine {
    val first by Arbitraries.strings()
    val second by Arbitraries.strings()
    // ...

    combineAs {
        "first: ?first, second: ?second"
    }
}

Note that accessing the first or second properties in the example above outside the combineAs block would result in an error.

In the background, this is equivalent to:

combine(listOf(Arbitraries.strings(), Arbitraries.strings())) { values ->
    "first: ?{values[0]}, second: ?{values[1]}"
}

It is also possible to use filters within the combinator DSL:

combine {
    val first by Arbitraries.strings()
    val second by Arbitraries.strings()

    filter { first.isNotEmpty() }
    filter { first != second }

    combineAs {
        // 'first' will never be empty or equal to 'second'

        "first" + "second"
    }
}

And you can also achieve the flatCombine behaviour, by using flatCombineAs:

combine {
    val first by Arbitraries.strings()
    val second by Arbitraries.strings()
    // ...

    flatCombineAs {
        Arbitraries.just("first: ?first, second: ?second")
    }
}

Quirks and Bugs

Testing Module

This module’s artefact name is jqwik-testing. It provides a few helpful methods and classes for generator writers to test their generators - including edge cases and shrinking.

This module is not in jqwik’s default dependencies. It’s usually added as a test-implementation dependency.

Advanced Topics

Implement your own Arbitraries and Generators

Looking at jqwik’s most prominent interfaces – Arbitrary and RandomGenerator – you might think that rolling your own implementations of these is a reasonable thing to do. I’d like to tell you that it never is, but I’ve learned that “never” is a word you should never use. There’s just too many things to consider when implementing a new type of Arbitrary to make it work smoothly with the rest of the framework.

Therefore, use the innumerable features to combine existing arbitraries into your special one. If your domain arbitrary must implement another interface - e.g. for configuration -, subclassing net.jqwik.api.arbitraries.ArbitraryDecorator is the way to go. An example would come in handy now…

Imagine you have to roll your own complex number type:

public class ComplexNumber {
    ...
	public ComplexNumber(double rational, double imaginary) {
		this.rational = rational;
		this.imaginary = imaginary;
	}
}

And you want to be able to tell the arbitrary whether or not the generated number should have an imaginary part:

@Property
void rationalNumbers(@ForAll("rationalOnly") ComplexNumber number) {
    ...
}

@Provide
Arbitrary<ComplexNumber> rationalOnly() {
	return new ComplexNumberArbitrary().withoutImaginaryPart();
}

Here’s how you could implement a configurable ComplexNumberArbitrary type:

public class ComplexNumberArbitrary extends ArbitraryDecorator<ComplexNumber> {

	private boolean withImaginaryPart = true;

	@Override
	protected Arbitrary<ComplexNumber> arbitrary() {
		Arbitrary<Double> rationalPart = Arbitraries.doubles();
		Arbitrary<Double> imaginaryPart = withImaginaryPart ? Arbitraries.doubles() : Arbitraries.just(0.0);
		return Combinators.combine(rationalPart, imaginaryPart).as(ComplexNumber::new);
	}

	public ComplexNumberArbitrary withoutImaginaryPart() {
		this.withImaginaryPart = false;
		return this;
	}
}

Overriding ArbitraryDecorator.arbitrary() is where you can apply the knowledge of the previous chapters. If you get stuck figuring out how to create an arbitrary with the desired behaviour either ask on stack overflow or open a Github issue.

Lifecycle Hooks

Similar to Jupiter’s Extension Model jqwik provides a means to extend and change the way how properties and containers are being configured, run and reported on. The API – interfaces, classes and annotations – for accessing those lifecycle hooks lives in the package net.jqwik.api.lifecycle and is – as of this release – are now mostly in the API evolution status MAINTAINED.

Principles of Lifecycle Hooks

There are a few fundamental principles that determine and constrain the lifecycle hook API:

  1. There are several types of lifecycle hooks, each of which is an interface that extends net.jqwik.api.lifecycle.LifecycleHook.
  2. A concrete lifecycle hook is an implementation of one or more lifecycle hook interfaces.
  3. You can add a concrete lifecycle hook to a container class or a property method with the annotation @AddLifecycleHook. By default, a lifecycle hook is only added to the annotated element, not to its children. However, you can override this behaviour by either:
    • Override LifecycleHook.propagateTo()
    • Use the annotation attribute @AddLifecycleHook.propagateTo()
  4. To add a global lifecycle use Java’s java.util.ServiceLoader mechanism and add the concrete lifecylcle hook class to file META-INF/services/net.jqwik.api.lifecycle.LifecycleHook. Do not forget to override LifecycleHook.propagateTo() if the global hook should be applied to all test elements.
  5. In a single test run there will only be a single instance of each concrete lifecycle hook implementation. That’s why you have to use jqwik’s lifecycle storage mechanism if shared state across several calls to lifecycle methods is necessary.
  6. Since all instances of lifecycle hooks are created before the whole test run is started, you cannot use non-static inner classes of test containers to implement lifecycle interfaces.
  7. If relevant, the order in which hook methods are being applied is determined by dedicated methods in the hook interface, e.g. BeforeContainerHook.beforeContainerProximity().

Mind that much of what you can do with hooks can also be done using the simpler mechanisms of annotated lifecycle methods or a property lifecycle class. You usually start to consider using lifecycle hooks when you want to reuse generic behaviour in many places or even across projects.

Lifecycle Hook Types

All lifecycle hook interfaces extend net.jqwik.api.lifecycle.LifecycleHook which has two methods that may be overridden:

jqwik currently supports eight types of lifecycle hooks:

Lifecycle Execution Hooks

With these hooks you can determine if a test element will be run at all, and what potential actions should be done before or after running it.

SkipExecutionHook

Implement SkipExecutionHook to filter out a test container or property method depending on some runtime condition.

Given this hook implementation:

public class OnMacOnly implements SkipExecutionHook {
    @Override
    public SkipResult shouldBeSkipped(final LifecycleContext context) {
        if (System.getProperty("os.name").equals("Mac OS X")) {
            return SkipResult.doNotSkip();
        }
        return SkipResult.skip("Only on Mac");
    }
}

The following property will only run on a Mac:

@Property
@AddLifecycleHook(OnMacOnly.class)
void macSpecificProperty(@ForAll int anInt) {
}
BeforeContainerHook

Implement BeforeContainerHook for a hook that’s supposed to do some work exactly once before any of its property methods and child containers will be run. This is typically used to set up a resource to share among all properties within this container.

AfterContainerHook

Implement AfterContainerHook for a hook that’s supposed to do some work exactly once after all of its property methods and child containers have been run. This is typically used to tear down a resource that has been shared among all properties within this container.

AroundContainerHook

AroundContainerHook is a convenience interface to implement both BeforeContainerHook and AfterContainerHook in one go. This is typically used to set up and tear down a resource that is intended to be shared across all the container’s children.

Here’s an example that shows how to start and stop an external server once for all properties of a test container:

@AddLifecycleHook(ExternalServerResource.class)
class AroundContainerHookExamples {
    @Example
    void example1() {
        System.out.println("Running example 1");
    }
    @Example
    void example2() {
        System.out.println("Running example 2");
    }
}

class ExternalServerResource implements AroundContainerHook {
    @Override
    public void beforeContainer(final ContainerLifecycleContext context) {
        System.out.println("Starting server...");
    }
  
    @Override
    public void afterContainer(final ContainerLifecycleContext context) {
        System.out.println("Stopping server...");
    }
}

Running this example should output

Starting server...

Running example 1

Running example 2

Stopping server...

If you wanted to do something before and/or after the whole jqwik test run, using a container hook and registering it globally is probably the easiest way.

AroundPropertyHook

AroundPropertyHook comes in handy if you need to define behaviour that should “wrap” the execution of a property, i.e., do something directly before or after running a property - or both. Since you have access to an object that describes the final result of a property you can also change the result, e.g. make a failed property successful or vice versa.

Here is a hook implementation that will measure the time spent on running a property and publish the result using a Reporter:

@Property(tries = 100)
@AddLifecycleHook(MeasureTime.class)
void measureTimeSpent(@ForAll Random random) throws InterruptedException {
    Thread.sleep(random.nextInt(50));
}

class MeasureTime implements AroundPropertyHook {
    @Override
    public PropertyExecutionResult aroundProperty(PropertyLifecycleContext context, PropertyExecutor property) {
        long before = System.currentTimeMillis();
        PropertyExecutionResult executionResult = property.execute();
        long after = System.currentTimeMillis();
        context.reporter().publish("time", String.format("%d ms", after - before));
        return executionResult;
    }
}

The additional output from reporting is concise:

timestamp = ..., time = 2804 ms
AroundTryHook

Wrapping the execution of a single try can be achieved by implementing AroundTryHook. This hook can be used for a lot of things. An incomplete list:

The following example shows how to fail if a single try will take longer than 100 ms:

@Property(tries = 10)
@AddLifecycleHook(FailIfTooSlow.class)
void sleepingProperty(@ForAll Random random) throws InterruptedException {
    Thread.sleep(random.nextInt(101));
}

class FailIfTooSlow implements AroundTryHook {
    @Override
    public TryExecutionResult aroundTry(
        final TryLifecycleContext context,
        final TryExecutor aTry,
        final List<Object> parameters
    ) {
        long before = System.currentTimeMillis();
        TryExecutionResult result = aTry.execute(parameters);
        long after = System.currentTimeMillis();
        long time = after - before;
        if (time >= 100) {
            String message = String.format("%s was too slow: %s ms", context.label(), time);
            return TryExecutionResult.falsified(new AssertionFailedError(message));
        }
        return result;
    }
}

Since the sleep time is chosen randomly the property will fail from time to time with the following error:

org.opentest4j.AssertionFailedError: sleepingProperty was too slow: 100 ms
InvokePropertyMethodHook

This is an experimental hook, which allows to change the way how a method - represented by a java.lang.reflect.Method object - is being invoked through reflection mechanisms.

ProvidePropertyInstanceHook

This is an experimental hook, which allows to change the way how the instance of a container class is created or retrieved.

Other Hooks

ResolveParameterHook

Besides the well-known @ForAll-parameters, property methods and annotated lifecycle methods can take other parameters as well. These can be injected by concrete implementations of ResolveParameterHook.

Consider this stateful Calculator:

public class Calculator {
    private int result = 0;
  
    public int result() {
        return result;
    }
  
    public void plus(int addend) {
        result += addend;
    }
}

When going to check its behaviour with properties you’ll need a fresh calculator instance in each try. This can be achieved by adding a resolver hook that creates a freshly instantiated calculator per try.

@AddLifecycleHook(CalculatorResolver.class)
class CalculatorProperties {
    @Property
    void addingANumberTwice(@ForAll int aNumber, Calculator calculator) {
        calculator.plus(aNumber);
        calculator.plus(aNumber);
        Assertions.assertThat(calculator.result()).isEqualTo(aNumber * 2);
    }
}

class CalculatorResolver implements ResolveParameterHook {
    @Override
    public Optional<ParameterSupplier> resolve(
        final ParameterResolutionContext parameterContext,
        final LifecycleContext lifecycleContext
    ) {
        return Optional.of(optionalTry -> new Calculator());
    }
    @Override
    public PropagationMode propagateTo() {
        // Allow annotation on container level
        return PropagationMode.ALL_DESCENDANTS;
    }
}

There are a few constraints regarding parameter resolution of which you should be aware:

RegistrarHook

Use RegistrarHook if you need to apply several hook implementations that implement the desired behaviour together but cannot be implemented in a single class. For example, more than one implementation of the same hook type is needed, but those implementations have a different proximity or require a different propagation mode.

This is really advanced stuff, the mechanism of which will probably evolve or change in the future. If you really really want to see an example, look at JqwikSpringExtension

Lifecycle Storage

As described above one of the fundamental principles is that there will be only a single instance of any lifecycle hook implementation during runtime. Since – depending on configuration and previous rung – containers and properties are not run in a strict sequential order this guarantee comes with a drawback: You cannot use a hook instance’s member variables to hold state that should be shared across all tries of a property or across all properties of a container or across different lifecycle phases of a single try. That’s when lifecycle storage management enters the stage in the form of type net.jqwik.api.lifecycle.Store.

A Store object…

You create a store like this:

Store<MyObject> myObjectStore = Store.create("myObjectStore", Lifespan.PROPERTY, () -> new MyObject());

And you retrieve a store similarly:

Store<MyObject> myObjectStore = Store.get("myObjectStore");

A store with the same identifier can only be created once, that’s why there are also convenience methods for creating or retrieving it:

Store<MyObject> myObjectStore = Store.getOrCreate("myObjectStore", Lifespan.PROPERTY, () -> new MyObject());

You now have the choice to use or update the shared state:

Store<MyObject> myObjectStore = ...;

myObjectStore.get().doSomethingWithMyObject();
myObjectStore.update(old -> {
    old.changeState();
    return old;
});

Let’s look at an example…

TemporaryFileHook

The following hook implementation gives you the capability to access one (and only one) temporary file per try using parameter resolution:

class TemporaryFileHook implements ResolveParameterHook {

    public static final Tuple.Tuple2 STORE_IDENTIFIER = Tuple.of(TemporaryFileHook.class, "temporary files");

    @Override
    public Optional<ParameterSupplier> resolve(
        ParameterResolutionContext parameterContext,
        LifecycleContext lifecycleContext
    ) {
        if (parameterContext.typeUsage().isOfType(File.class)) {
            return Optional.of(ignoreTry -> getTemporaryFileForTry());
        }
        return Optional.empty();
    }

    private File getTemporaryFileForTry() {
        Store<ClosingFile> tempFileStore =
            Store.getOrCreate(
                STORE_IDENTIFIER, Lifespan.TRY,
                () -> new ClosingFile(createTempFile())
            );
        return tempFileStore.get().file();
    }

    private File createTempFile() {
        try {
            return File.createTempFile("temp", ".txt");
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    private record ClosingFile(File file) implements Store.CloseOnReset {
        @Override
        public void close() {
            file.delete();
        }
    }
}

There are a few interesting things going on:

With this information you can probably figure out how the following test container works – especially why the assertion in @AfterTry-method assertFileNotEmpty() succeeds.

@AddLifecycleHook(value = TemporaryFileHook.class, propagateTo = PropagationMode.ALL_DESCENDANTS)
class TemporaryFilesExample {
    @Property(tries = 10)
    void canWriteToFile(File anyFile, @ForAll @AlphaChars @StringLength(min = 1) String fileContents) throws Exception {
        assertThat(anyFile).isEmpty();
        writeToFile(anyFile, fileContents);
        assertThat(anyFile).isNotEmpty();
    }
  
    @AfterTry
    void assertFileNotEmpty(File anyFile) {
        assertThat(anyFile).isNotEmpty();
    }
  
    private void writeToFile(File anyFile, String contents) throws IOException {
        BufferedWriter writer = new BufferedWriter(new FileWriter(anyFile));
        writer.write(contents);
        writer.close();
    }
}

API Evolution

In agreement with the JUnit 5 platform jqwik uses the @API Guardian project to communicate version and status of all parts of its API. The different types of status are:

-STABLE: Intended for features that will not be changed in a backwards-incompatible way in the current major version (1.*).

-MAINTAINED: Intended for features that will not be changed in a backwards-incompatible way for at least the current minor release of the current major version. If scheduled for removal, it will be demoted to DEPRECATED first.

-EXPERIMENTAL: Intended for new, experimental features where we are looking for feedback. Use this element with caution; it might be promoted to MAINTAINED or STABLE in the future, but might also be removed without prior notice, even in a patch.

-DEPRECATED: Should no longer be used; might disappear in the next minor release.

-INTERNAL: Must not be used by any code other than jqwik itself. Might be removed without prior notice.

Since annotation @API has runtime retention you find the actual API status in an element’s source code, its Javadoc but also through reflection. If a certain element, e.g. a method, is not annotated itself, then it carries the status of its containing class.

Release Notes

Read this version’s release notes.