Sponsor the Team View on GitHub

jqwik Latest Release: 1.9.2

Property-Based Testing in Java

The jqwik User Guide 1.2.1

Table of Contents

How to Use

jqwik is an alternative test engine for the JUnit 5 platform. That means that you can use it either stand-alone or combine it with any other JUnit 5 engine, e.g. Jupiter (the standard engine) or Vintage (aka JUnit 4). All you have to do is add all needed engines to your testCompile dependencies as shown in the gradle file below.

The latest release of jqwik is deployed to Maven Central.

Snapshot releases can be fetched from https://oss.sonatype.org/content/repositories/snapshots.

Gradle

Since version 4.6, Gradle has built-in support for the JUnit platform. Set up is rather simple; here are the relevant parts of a project’s build.gradle file:

repositories {
    ...
    mavenCentral()

    # For snapshot releases only:
    maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }

}

ext.junitPlatformVersion = '1.5.2'
ext.junitJupiterVersion = '5.5.2'

ext.jqwikVersion = '1.2.1'

test {
	useJUnitPlatform {
		includeEngines "jqwik"

		// includeTags "fast", "medium"
		// excludeTags "slow"
	}

	include '**/*Properties.class'
	include '**/*Test.class'
	include '**/*Tests.class'
}

dependencies {
    ...

    // aggregate jqwik dependency
    testCompile "net.jqwik:jqwik:${jqwikVersion}"

    // Add if you also want to use the Jupiter engine or Assertions from it
    testCompile("org.junit.jupiter:junit-jupiter-engine:5.5.2")

    // Add any other test library you need...
    testCompile("org.assertj:assertj-core:3.12.2")

}

With version 1.0.0 net.jqwik:jqwik has become an aggregating module to simplify jqwik integration for standard users. If you want to be more explicit about the real dependencies you can replace this dependency with

    testCompile "net.jqwik:jqwik-api:${jqwikVersion}"
    testRuntime "net.jqwik:jqwik-engine:${jqwikVersion}"

See the Gradle section in JUnit 5’s user guide for more details on how to configure Gradle for the JUnit 5 platform.

Seeing jqwik Reporting in Gradle Output

Since Gradle does not yet support JUnit platform reporting (see this Github issue) jqwik has switched to do its own reporting by default. This behaviour can be configured through parameter useJunitPlatformReporter (default: false).

If you want to see jqwik’s reports in the output use Gradle’s command line option --info:

> gradle clean test --info
...
mypackage.MyClassProperties > myPropertyMethod STANDARD_OUT
    timestamp = 2019-02-28T18:01:14.302, MyClassProperties:myPropertyMethod = 
                                  |-----------------------jqwik-----------------------
    tries = 1000                  | # of calls to property
    checks = 1000                 | # of not rejected calls
    generation-mode = RANDOMIZED  | parameters are randomly generated
    seed = 1685744359484719817    | random seed to reproduce generated values

Maven

Configure the surefire plugin as described in the Maven section in JUnit 5’s user guide and add the following dependency to your pom.xml file:

<dependencies>
    ...
    <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik</artifactId>
        <version>1.2.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Snapshot Releases

Snapshot releases are available through Sonatype’s snapshot repositories.

Adding

https://oss.sonatype.org/content/repositories/snapshots

as a maven repository will allow you to use jqwik’s snapshot release which contains all the latest features.

Project without Build Tool

I’ve never tried it but using jqwik without gradle or some other tool to manage dependencies should also work. You will have to add at least the following jars to your classpath:

Creating an Example-based Test

Just annotate a public, protected or package-scoped method with @Example. Example-based tests work just like plain JUnit-style test cases and are not supposed to take any parameters.

A test case method must

Here is a test class with two example-based tests:

import static org.assertj.core.api.Assertions.*;

import net.jqwik.api.*;
import org.assertj.core.data.*;

class ExampleBasedTests {
	
	@Example
	void squareRootOf16is4() { 
		assertThat(Math.sqrt(16)).isCloseTo(4.0, Offset.offset(0.01));
	}

	@Example
	boolean add1plu3is4() {
		return (1 + 3) == 4;
	}
}

Creating a Property

Properties are the core concept of property-based testing.

You create a Property by annotating a public, protected or package-scoped method with @Property. In contrast to examples a property method is supposed to have one or more parameters, all of which must be annotated with @ForAll.

At test runtime the exact parameter values of the property method will be filled in by jqwik.

Just like an example test a property method has to

If not specified differently, jqwik will run 1000 tries, i.e. a 1000 different sets of parameter values and execute the property method with each of those parameter sets. The first failed execution will stop value generation and be reported as failure - usually followed by an attempt to shrink the falsified parameter set.

Here are two properties whose failures might surprise you:

import net.jqwik.api.*;
import org.assertj.core.api.*;

class PropertyBasedTests {

	@Property
	boolean absoluteValueOfAllNumbersIsPositive(@ForAll int anInteger) {
		return Math.abs(anInteger) >= 0;
	}

	@Property
	void lengthOfConcatenatedStringIsGreaterThanLengthOfEach(
		@ForAll String string1, @ForAll String string2
	) {
		String conc = string1 + string2;
		Assertions.assertThat(conc.length()).isGreaterThan(string1.length());
		Assertions.assertThat(conc.length()).isGreaterThan(string2.length());
	}
}

Currently jqwik cannot deal with parameters that are not annotated with ‘@ForAll’. However, this might change in future versions.

Optional @Property Parameters

The @Property annotation has a few optional values:

Additional Reporting

You can switch on additional reporting aspects by adding a @Report(Reporting[]) annotation to a property method.

The following reporting aspects are available:

Assertions

jqwik does not come with any assertions, so you have to use one of the third-party assertion libraries, e.g. Hamcrest or AssertJ.

If you have Jupiter in your test dependencies anyway, you can also use the static methods in org.junit.jupiter.api.Assertions.

Lifecycle

Method Lifecycle

The current lifecycle of jqwik test methods is rather simple:

import net.jqwik.api.*;

class TestsWithLifecycle implements AutoCloseable {

	TestsWithLifecycle() {
		System.out.println("Before each");
	}

	@Example void anExample() {
		System.out.println("anExample");
	}

	@Property(tries = 5)
	void aProperty(@ForAll String aString) {
		System.out.println("anProperty: " + aString);
	}

	@Override
	public void close() throws Exception {
		System.out.println("After each");
	}
}

In this example both the constructor and close() will be called twice times: Once for anExample() and once for aProperty(...).

Other Lifecycles

Currently jqwik does not have special support for a lifecycle per test container, per test try or even package. Later versions of jqwik might possible bring more features in that field. Create an issue on github with your concrete needs.

Grouping Tests

Within a containing test class you can group other containers by embedding another non-static and non-private inner class and annotating it with @Group. Grouping examples and properties is a means to improve the organization and maintainability of your tests.

Groups can be nested and there lifecycle is also nested, that means that the lifecycle of a test class is also applied to inner groups of that container. Have a look at this example:

import net.jqwik.api.*;

class TestsWithGroups {

	@Property
	void outer(@ForAll String aString) {
	}

	@Group
	class Group1 {
		@Property
		void group1Property(@ForAll String aString) {
		}

		@Group
		class Subgroup {
			@Property
			void subgroupProperty(@ForAll String aString) {
			}
		}
	}

	@Group
	class Group2 {
		@Property
		void group2Property(@ForAll String aString) {
		}
	}
}

Naming and Labeling Tests

Using Java-style camel case naming for your test container classes and property methods will sometimes lead to hard to read display names in your test reports and your IDE. Therefore, jqwik provides a simple way to insert spaces into the displayed name of your test container or property: just add underscores (_), which are valid Java identifier characters. Each underscore will be replaced by a space for display purposes.

If you want to tweak display names even more, test container classes, groups, example methods and property methods can be labeled using the annotation @Label("a label"). This label will be used to display the element in test reports or within the IDE. In the following example, every test relevant element has been labeled:

@Label("Naming")
class NamingExamples {

	@Property
	@Label("a property")
	void aPropertyWithALabel() { }

	@Group
	@Label("A Group")
	class GroupWithLabel {
		@Example
		@Label("an example with äöüÄÖÜ")
		void anExampleWithALabel() { }
	}

    @Group
    class Group_with_spaces {
        @Example
        void example_with_spaces() { }
    }

}

Labels can consist of any characters and don’t have to be unique - but you probably want them to be unique within their container.

Tagging Tests

Test container classes, groups, example methods and property methods can be tagged using the annotation @Tag("a-tag"). You can have many tags on the same element.

Those tag can be used to filter the set of tests run by the IDE or the build tool. Tags are handed down from container (class or group) to its children (test methods or groups).

Have a look at the following example. Including the tag integration-test will include all tests of the class.

@Tag("integration-test")
class TaggingExamples {

	@Property
	@Tag("fast")
	void aFastProperty() { }

	@Example
	@Tag("slow") @Tag("involved")
	void aSlowTest() { }
}

Tags must follow certain rules as described here

Disabling Tests

From time to time you might want to disable a test or all tests in a container temporarily. You can do that by adding the @Disabled annotation to a property method or a container class.

import net.jqwik.api.Disabled

@Disabled("for whatever reason")
class DisablingExamples {

	@Property
	@Disabled
	void aDisabledProperty() { }

}

Disabled properties will be reported by IDEs and build tools as “skipped” together with the reason - if one has been provided.

Be careful not to use the Jupiter annotation with the same name. Jqwik will refuse to execute methods that have Jupiter annotations.

Default Parameter Generation

jqwik tries to generate values for those property method parameters that are annotated with @ForAll. If the annotation does not have a value parameter, jqwik will use default generation for the following types:

If you use @ForAll with a value, e.g. @ForAll("aMethodName"), the method referenced by "aMethodName" will be called to provide an Arbitrary of the required type (see Parameter Provider Methods).

Constraining Default Generation

Default parameter generation can be influenced and constrained by additional annotations, depending on the requested parameter type.

Allow Null Values

Unique Values

String Length

Character Sets

When generating chars any unicode character might be generated.

When generating Strings, however, Unicode “noncharacters” and “private use characters” will not be generated unless you explicitly include them using @Chars or @CharRange (see below).

You can use the following annotations to restrict the set of allowed characters and even combine several of them:

They work for generated Strings and Characters.

List, Set, Stream, Map and Array Size

Integer Constraints

Decimal Constraints

Constraining parameterized types

When you want to constrain the generation of contained parameter types you can annotate the parameter type directly, e.g.:

@Property
void aProperty(@ForAll @Size(min= 1) List<@StringLength(max=10) String> listOfStrings) {
}

will generate lists with a minimum size of 1 filled with Strings that have 10 characters max.

Providing variable types

While checking properties of generically typed classes or functions, you often don’t care about the exact type of variables and therefore want to express them with type variables. jqwik can also handle type variables and wildcard types. The handling of upper and lower bounds works mostly as you would expect it.

Consider the following examples:

class VariableTypedPropertyExamples {

	@Property
	<T> boolean unboundedGenericTypesAreResolved(@ForAll List<T> items, @ForAll T newItem) {
		items.add(newItem);
		return items.contains(newItem);
	}

	@Property
	<T extends Serializable & Comparable> void someBoundedGenericTypesCanBeResolved(@ForAll List<T> items, @ForAll T newItem) {
	}

	@Property
	void someWildcardTypesWithUpperBoundsCanBeResolved(@ForAll List<? extends Serializable> items) {
	}

}

In the case of unbounded type variables or an unbounded wildcard type, jqwik will create instanced of a special class (WildcardObject) under the hood.

In the case of bounded type variables and bounded wildcard types, jqwik will check if any registered arbitrary provider can provide suitable arbitraries and choose randomly between those.

There is, however, a potentially unexpected behaviour, when the same type variable is used in more than one place and can be resolved by more than one arbitrary. In this case it can happen that the variable does not represent the same type in all places. You can see this above in property method someBoundedGenericTypesCanBeResolved() where items might be a list of Strings but newItem of some number type - and all that in the same call to the method!

Self-Made Annotations

You can make your own annotations instead of using jqwik’s built-in ones. BTW, ‘@Example’ is nothing but a plain annotation using @Property as “meta”-annotation.

The following example provides an annotation to constrain String or Character generation to German letters only:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
@NumericChars
@AlphaChars
@Chars({'ä', 'ö', 'ü', 'Ä', 'Ö', 'Ü', 'ß'})
@Chars({' ', '.', ',', ';', '?', '!'})
@StringLength(min = 10, max = 100)
public @interface GermanText { }

@Property(tries = 10) @Reporting(Reporting.GENERATED)
void aGermanText(@ForAll @GermanText String aText) {}

The drawback of self-made annotations is that they do not forward their parameters to meta-annotations, which constrains their applicability to simple cases.

Customized Parameter Generation

Sometimes the possibilities of adjusting default parameter generation through annotations is not enough. You want to control the creation of values programmatically. The means to do that are provider methods.

Parameter Provider Methods

Look at the following example:

@Property
boolean concatenatingStringWithInt(
    @ForAll("shortStrings") String aShortString,
    @ForAll("10 to 99") int aNumber
) {
    String concatenated = aShortString + aNumber;
    return concatenated.length() > 2 && concatenated.length() < 11;
}

@Provide
Arbitrary<String> shortStrings() {
    return Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(8);
}

@Provide("10 to 99")
Arbitrary<Integer> numbers() {
    return Arbitraries.integers().between(10, 99);
}

The String value of the @ForAll annotation serves as a reference to a method within the same class (or one of its superclasses or owning classes). This reference refers to either the method’s name or the String value of the method’s @Provide annotation.

The providing method has to return an object of type @Arbitrary<T> where T is the static type of the parameter to be provided. Optionally the provider method can take tow optional parameters:

These two objects can be used to get detailed information about the parameter, like annotations and embedded type parameters, and to resolve other types, usually from type parameters embedded in the original parameter. Use with care!

Parameter provision usually starts with a static method call to Arbitraries, maybe followed by one or more filtering, mapping or combining actions.

Providing Arbitraries for Embedded Types

There is an alternative syntax to @ForAll("methodRef") using a From annotation:

@Property
boolean concatenatingStringWithInt(
    @ForAll @From("shortStrings") String aShortString,
    @ForAll @From("10 to 99") int aNumber
) { ... }

Why this redundancy? Well, @From becomes a necessity when you want to provide the arbitrary of an embedded type parameter. Consider this example:

@Property
boolean joiningListOfStrings(@ForAll List<@From("shortStrings") String> listOfStrings) {
    String concatenated = String.join("", listOfStrings);
    return concatenated.length() <= 8 * listOfStrings.size();
}

Here, the list is created using the default list arbitrary, but the String elements are generated using the arbitrary from the method shortStrings.

Static Arbitraries methods

The starting point for generation usually is a static method call on class Arbitraries.

Generate values yourself

Select values randomly

Select randomly with Weights

If you have a set of values to choose from with weighted probabilities, use ` Arbitraries.frequency(...):

@Property
void abcdWithFrequencies(@ForAll("abcdWeighted") String aString) {
    Statistics.collect(aString);
}

@Provide
Arbitrary<String> abcdWeighted() {
    return Arbitraries.frequency(
        Tuple.of(1, "a"),
        Tuple.of(5, "b"),
        Tuple.of(10, "c"),
        Tuple.of(20, "d")
    );
}

The first value of the tuple specifies the frequency of a particular value in relation to the sum of all frequencies. In the given example the sum is 36, thus "a" will be generated with a probability of 1/36 whereas "d" has a generation probability of 20/36 (= 5/9).

Shrinking moves towards the start of the frequency list.

Integers

Decimals

Characters and Strings

java.util.Random

Constants

Create

Shuffling Permutations

Default Types

Collections, Streams, Arrays and Optional

Generating types who have generic type parameters, requires to start with an Arbitrary instance for the generic type. You can create the corresponding collection arbitrary from there:

Collecting Values in a List

If you do not want any random combination of values in your list - as can be done with Arbitrary.list() - you have the possibility to collect random values in a list until a certain condition is fulfilled. Arbitrary.collect(Predicate condition) is what you need in those cases.

Imagine you need a list of integers the sum of which should be at least 1000. Here’s how you could do that:

Arbitrary<Integer> integers = Arbitraries.integers().between(1, 100);
Arbitrary<List<Integer>> collected = integers.collect(list -> sum(list) >= 1000);

Maps

Generating instances of type Map is a bit different since two arbitraries are needed, one for the key and one for the value. Therefore you have to use Arbitraries.maps(...) like this:

@Property
void mapsFromNumberToString(@ForAll("numberMaps")  Map<Integer, String> map) {
    Assertions.assertThat(map.keySet()).allMatch(key -> key >= 0 && key <= 1000);
    Assertions.assertThat(map.values()).allMatch(value -> value.length() == 5);
}

@Provide
Arbitrary<Map<Integer, String>> numberMaps() {
    Arbitrary<Integer> keys = Arbitraries.integers().between(1, 100);
    Arbitrary<String> values = Arbitraries.strings().alpha().ofLength(5);
    return Arbitraries.maps(keys, values);
}

For generating individual Map.Entry instances there is Arbitraries.entries(...).

Functional Types

Interfaces that have a single (non default) method are considered to be Functional types; they are sometimes called SAM types for “single abstract method”. If a functional type is used as a @ForAll-parameter jqwik will automatically generate instances of those functions. The generated functions have the following characteristics:

Let’s look at an example:

@Property
void fromIntToString(@ForAll Function<Integer, @StringLength(5) String> function) {
    assertThat(function.apply(42)).hasSize(5);
    assertThat(function.apply(1)).isEqualTo(function.apply(1));
}

This works for any interface-based functional types, even your own. If you register a default provider for a functional type with a priority of 0 or above, it will take precedence.

If the functions need some specialized arbitrary for return values or if you want to fix the function’s behaviour for some range of values, you can define the arbitrary manually:

@Property
void emptyStringsTestFalse(@ForAll("predicates") Predicate<String> predicate) {
    assertThat(predicate.test("")).isFalse();
}

@Provide
Arbitrary<Predicate<String>> predicates() {
    return Functions
        .function(Predicate.class)
        .returns(Arbitraries.of(true, false))
        .when(parameters -> parameters.get(0).equals(""), parameters -> false);
}

In this example the generated predicate will always return false when given an empty String and randomly choose between true and false in all other cases.

Fluent Configuration Interfaces

Most specialized arbitrary interfaces provide special methods to configure things like size, length, boundaries etc. Have a look at the Java doc for the following types:

Here are a two examples to give you a hint of what you can do:

@Provide
Arbitrary<String> alphaNumericStringsWithMinLength5() {
    return Arbitraries.strings().ofMinLength(5).alpha().numeric();
}

@Provide
Arbitrary<List<Integer>> fixedSizedListOfPositiveIntegers() {
    return Arbitraries.integers().greaterOrEqual(0).list().ofSize(17);
}

Generate null values

Predefined generators will never create null values. If you want to allow that, call Arbitrary.injectNull(double probability). The following provider method creates an arbitrary that will return a null String in about 1 of 100 generated values.

@Provide 
Arbitrary<String> stringsWithNull() {
  return Arbitraries.strings(0, 10).injectNull(0.01);
}

Filtering

If you want to include only part of all the values generated by an arbitrary, use Arbitrary.filter(Predicate<T> filterPredicate). The following arbitrary will filter out all even numbers from the stream of generated integers:

@Provide 
Arbitrary<Integer> oddNumbers() {
  return Arbitraries.integers().filter(aNumber -> aNumber % 2 != 0);
}

Keep in mind that your filter condition should not be too restrictive. If the generator fails to find a suitable value after 10000 trials, the current property will be abandoned by throwing an exception.

Creating unique values

If you want to make sure that all the values generated by an arbitrary are unique, use Arbitrary.unique(). The following arbitrary will generate integers between 1 and 1000 but never the same integer twice:

@Provide
Arbitrary<Integer> oddNumbers() {
  return Arbitraries.integers().between(1, 1000).unique();
}

This means that a maximum of 1000 values can be generated. If the generator fails to find a yet unseen value after 10000 trials, the current property will be abandoned by throwing an exception.

Mapping

Sometimes it’s easier to start with an existing arbitrary and use its generated values to build other objects from them. In that case, use Arbitrary.map(Function<T, U> mapper). The following example uses generated integers to create numerical Strings:

@Provide 
Arbitrary<String> fiveDigitStrings() {
  return Arbitraries.integers(10000, 99999).map(aNumber -> String.valueOf(aNumber));
}

You could generate the same kind of values by constraining and filtering a generated String. However, the shrinking target would probably be different. In the example above, shrinking will move towards the lowest allowed number, that is 10000.

Flat Mapping

Similar as in the case of Arbitrary.map(..) there are situations in which you want to use a generated value in order to create another Arbitrary from it. Sounds complicated? Have a look at the following example:

@Property
boolean fixedSizedStrings(@ForAll("listsOfEqualSizedStrings")List<String> lists) {
    return lists.stream().distinct().count() == 1;
}

@Provide
Arbitrary<List<String>> listsOfEqualSizedStrings() {
    Arbitrary<Integer> integers2to5 = Arbitraries.integers().between(2, 5);
    return integers2to5.flatMap(stringSize -> {
        Arbitrary<String> strings = Arbitraries.strings() //
                .withCharRange('a', 'z') //
                .ofMinLength(stringSize).ofMaxLength(stringSize);
        return strings.list();
    });
}

The provider method will create random lists of strings, but in each list the size of the contained strings will always be the same - between 2 and 5.

Flat Mapping with Tuple Types

In the example above you used a generated value in order to create another arbitrary. In those situations you often want to also provide the original values to your property test.

Imagine, for instance, that you’d like to test properties of String.substring(begin, end). To randomize the method call, you not only need a string but also the begin and end indices. However, both have dependencies:

@Property
void substringLength(@ForAll("stringWithBeginEnd") Tuple3<String, Integer, Integer> stringBeginEnd) {
    String aString = stringBeginEnd.get1();
    int begin = stringBeginEnd.get2();
    int end = stringBeginEnd.get3();
    Assertions.assertThat(aString.substring(begin, end).length())
        .isEqualTo(end - begin);
}

@Provide
Arbitrary<Tuple3<String, Integer, Integer>> stringWithBeginEnd() {
    Arbitrary<String> stringArbitrary = Arbitraries.strings() //
            .withCharRange('a', 'z') //
            .ofMinLength(2).ofMaxLength(20);
    return stringArbitrary //
            .flatMap(aString -> Arbitraries.integers().between(0, aString.length()) //
                    .flatMap(end -> Arbitraries.integers().between(0, end) //
                            .map(begin -> Tuple.of(aString, begin, end))));
}

Mind the nested flat mapping, which is an aesthetic nuisance but nevertheless very useful.

Randomly Choosing among Arbitraries

If you have several arbitraries of the same type, you can create a new arbitrary of the same type which will choose randomly one of those arbitraries before generating a value:

@Property
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    String classifier = anInt < -1000 ? "below" : anInt > 1000 ? "above" : "one";
    Statistics.collect(classifier);
    
    return anInt < -1000 //
            || Math.abs(anInt) == 1 //
            || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-2000, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 2000);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);
    
    return Arbitraries.oneOf(below1000, above1000, oneOrMinusOne);
}

In this example the statistics should also give you an equal distribution between the three types of integers.

If you don’t want to choose with equal probability - but with differing frequency - you can do that in a similar way:

@Property(tries = 100)
@Report(Reporting.GENERATED)
boolean intsAreCreatedFromOneOfThreeArbitraries(@ForAll("oneOfThree") int anInt) {
    return anInt < -1000 //
               || Math.abs(anInt) == 1 //
               || anInt > 1000;
}

@Provide
Arbitrary<Integer> oneOfThree() {
    IntegerArbitrary below1000 = Arbitraries.integers().between(-1050, -1001);
    IntegerArbitrary above1000 = Arbitraries.integers().between(1001, 1050);
    Arbitrary<Integer> oneOrMinusOne = Arbitraries.samples(-1, 1);

    return Arbitraries.frequencyOf(
        Tuple.of(1, below1000),
        Tuple.of(3, above1000),
        Tuple.of(6, oneOrMinusOne)
    );
}

Combining Arbitraries

Sometimes just mapping a single stream of generated values is not enough to generate a more complicated domain object. In those cases you can combine several arbitraries to a single result arbitrary using Combinators.combine() with up to eight arbitraries. Create an issue on github if you need more than eight.

The following example generates Person instances from three arbitraries as inputs.

@Property
void validPeopleHaveIDs(@ForAll("validPeople") Person aPerson) {
    Assertions.assertThat(aPerson.getID()).contains("-");
    Assertions.assertThat(aPerson.getID().length()).isBetween(5, 24);
}

@Provide
Arbitrary<Person> validPeople() {
    Arbitrary<String> names = Arbitraries.strings().withCharRange('a', 'z')
        .ofMinLength(3).ofMaxLength(21);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    return Combinators.combine(names, ages)
        .as((name, age) -> new Person(name, age));
}

class Person {
    private final String name;
    private final int age;

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String getID() {
        return name + "-" + age;
    }

    @Override
    public String toString() {
        return String.format("%s:%s", name, age);
    }
}

The property should fail, thereby shrinking the falsified Person instance to [aaaaaaaaaaaaaaaaaaaaa:100].

The Combinators.combine method accepts up to 8 parameters of type Arbitrary. If you need more you have a few options:

Combining Arbitraries with Builder

There’s an alternative way to combine arbitraries to create an aggregated object by using a builder for the aggregated object. Consider the example from above and throw a PersonBuilder into the mix:

static class PersonBuilder {

    private String name = "A name";
    private int age = 42;

    public PersonBuilder withName(String name) {
        this.name = name;
        return this;
    }

    public PersonBuilder withAge(int age) {
        this.age = age;
        return this;
    }

    public Person build() {
        return new Person(name, age);
    }
}

Then you can go about generating people in the following way:

@Provide
Arbitrary<Person> validPeopleWithBuilder() {
    Arbitrary<String> names = 
        Arbitraries.strings().withCharRange('a', 'z').ofMinLength(2).ofMaxLength(20);
    Arbitrary<Integer> ages = Arbitraries.integers().between(0, 130);
    
    return Combinators.withBuilder(() -> new PersonBuilder())
        .use(names).in((builder, name) -> builder.withName(name))
        .use(ages).in((builder, age)-> builder.withAge(age))
        .build( builder -> builder.build());
}

Have a look at Combinators.withBuilder(Supplier) and Combinators.withBuilder(Arbitrary) to check the API.

Flat Combination

If generating domain values requires to use several generated values to be used in generating another one, there’s the combination of flat mapping and combining:

@Property
boolean fullNameHasTwoParts(@ForAll("fullName") String aName) {
    return aName.split(" ").length == 2;
}

@Provide
Arbitrary<String> fullName() {
    IntegerArbitrary firstNameLength = Arbitraries.integers().between(2, 10);
    IntegerArbitrary lastNameLength = Arbitraries.integers().between(2, 10);
    return Combinators.combine(firstNameLength, lastNameLength).flatAs( (fLength, lLength) -> {
        Arbitrary<String> firstName = Arbitraries.strings().alpha().ofLength(fLength);
        Arbitrary<String> lastName = Arbitraries.strings().alpha().ofLength(fLength);
        return Combinators.combine(firstName, lastName).as((f,l) -> f + " " + l);
    });
}

Often, however, there’s an easier way to achieve the same goal which does not require the flat combination of arbitraries:

@Provide
Arbitrary<String> fullName2() {
    Arbitrary<String> firstName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    Arbitrary<String> lastName = Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(10);
    return Combinators.combine(firstName, lastName).as((f, l) -> f + " " + l);
}

This is not only easier to understand but it usually improves shrinking.

Fix an Arbitrary’s genSize

Some generators (e.g. most number generators) are sensitive to the genSize value that is used when creating them. The default value for genSize is the number of tries configured for the property they are used in. If there is a need to influence the behaviour of generators you can do so by using Arbitrary.fixGenSize(int)..

Generating all possible values

There are a few cases when you don’t want to generate individual values from an arbitrary but use all possible values to construct another arbitrary. This can be achieved through Arbitrary.allValues().

Return type is Optional<Stream<T>> because jqwik can only perform this task if exhaustive generation is doable.

Iterating through all possible values

You can also use an arbitrary to iterate through all values it specifies. Use Arbitrary.forEachValue(Consumer action). for that purpose. This only works when exhaustive generation is possible. In other cases the attempt to iterate will result in an exception.

This is typically useful when your test requires to assert some fact for all values of a given (sub)set of objects. Here’s a contrived example:

@Property
void canPressAnyKeyOnKeyboard(@ForAll Keyboard keyboard, @ForAll Key key) {
    keyboard.press(key);
    assertThat(keyboard.isPressed(key));

    Arbitrary<Key> unpressedKeys = Arbitraries.of(keyboard.allKeys()).filter(k -> !k.equals(key));
    unpressedKeys.forEachValue(k -> assertThat(keyboard.isPressed(k)).isFalse());
}

In this example a simple for loop over allKeys() would also work. In more complicated scenarios jqwik will do all the combinations and filtering for you.

Recursive Arbitraries

Sometimes it seems like a good idea to compose arbitraries and thereby recursively calling an arbitrary creation method. Generating recursive data types is one application field but you can also use it for other stuff.

Probabilistic Recursion

Look at the following example which generates sentences by recursively adding words to a sentence:

@Property
boolean sentencesEndWithAPoint(@ForAll("sentences") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> sentences() {
    Arbitrary<String> sentence = Combinators.combine( //
        Arbitraries.lazy(this::sentences), //
        word() //
    ).as((s, w) -> w + " " + s);
    return Arbitraries.oneOf( //
        word().map(w -> w + "."), //
        sentence, //
        sentence, //
        sentence //
    );
}

private StringArbitrary word() {
    return Arbitraries.strings().alpha().ofLength(5);
}

There are two things to which you must pay attention:

Deterministic Recursion

An alternative to the non-deterministic recursion shown above, is to use classical recursion with a counter to determine the base case. If you then use an arbitrary value for the counter, the generated sentences will be very similar, and there is no need for using Arbitraries.lazy() at all:

@Property(tries = 10)
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
    Arbitrary<Integer> length = Arbitraries.integers().between(0, 10);
    Arbitrary<String> lastWord = word().map(w -> w + ".");
    return length.flatMap(l -> deterministic(l, lastWord));
}

@Provide
Arbitrary<String> deterministic(int length, Arbitrary<String> sentence) {
    if (length == 0) {
        return sentence;
    }
    Arbitrary<String> more = Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
    return deterministic(length - 1, more);
}

Deterministic Recursion with recursive()

To further simplify this jqwik provides a helper function: Arbitraries.recursive(...). Using that further simplifies the example:

@Property(tries = 10)
boolean sentencesEndWithAPoint(@ForAll("deterministic") String aSentence) {
    return aSentence.endsWith(".");
}

@Provide
Arbitrary<String> deterministic() {
    Arbitrary<Integer> length = Arbitraries.integers().between(0, 10);
    Arbitrary<String> lastWord = word().map(w -> w + ".");

    return length.flatMap(l -> Arbitraries.recursive(() -> lastWord, s -> prependWord(s), l));
}

private Arbitrary<String> prependWord(Arbitrary<String> sentence) {
    return Combinators.combine(word(), sentence).as((w, s) -> w + " " + s);
}

Contract Tests

When you combine type variables with properties defined in superclasses or interfaces you can do some kind of contract testing. That means that you specify the properties in a generically typed interface and specify the concrete class to instantiate in a test container implementing the interface.

The following example was influence by a similar feature in junit-quickcheck. Here’s the contract:

interface ComparatorContract<T> {
	Comparator<T> subject();

	@Property
	default void symmetry(@ForAll("anyT") T x, @ForAll("anyT") T y) {
		Comparator<T> subject = subject();

		Assertions.assertThat(signum(subject.compare(x, y))).isEqualTo(-signum(subject.compare(y, x)));
	}

	@Provide
	Arbitrary<T> anyT();
}

And here’s the concrete test container that can be run to execute the property with generated Strings:

class StringCaseInsensitiveProperties implements ComparatorContract<String> {

	@Override public Comparator<String> subject() {
		return String::compareToIgnoreCase;
	}

	@Override
	@Provide
	public Arbitrary<String> anyT() {
		return Arbitraries.strings().alpha().ofMaxLength(20);
	}
}

What we can see here is that jqwik is able to figure out the concrete type of type variables when they are used in subtypes that fill in the variables.

Stateful Testing

Despite its bad reputation state is an important concept in object-oriented languages like Java. We often have to deal with stateful objects or components whose state can be changed through methods.

Thinking in a more formal way we can look at those objects as state machines and the methods as actions that move the object from one state to another. Some actions have preconditions to constrain when they can be invoked and some objects have invariants that should never be violated regardless of the sequence of performed actions.

To make this abstract concept concrete, let’s look at a simple stack implementation:

public class MyStringStack {
	public void push(String element) { ... }
	public String pop() { ... }
	public void clear() { ... }
	public boolean isEmpty() { ... }
	public int size() { ... }
	public String top() { ... }
}

Specify Actions

We can see at least three actions with their preconditions and expected state changes:

Check Postconditions

The fundamental property that jqwik should try to falsify is:

For any valid sequence of actions all required state changes
(aka postconditions) should be fulfilled.

We can formulate that quite easily as a jqwik property:

class MyStringStackProperties {

	@Property
	void checkMyStack(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
		actions.run(new MyStringStack());
	}

	@Provide
	Arbitrary<ActionSequence<MyStringStack>> sequences() {
		return Arbitraries.sequences(Arbitraries.oneOf(push(), pop(), clear()));
	}

	private Arbitrary<Action<MyStringStack>> push() {
		return Arbitraries.strings().alpha().ofLength(5).map(PushAction::new);
	}

	private Arbitrary<Action<MyStringStack>> clear() {
		return Arbitraries.constant(new ClearAction());
	}

	private Arbitrary<Action<MyStringStack>> pop() {
		return Arbitraries.constant(new PopAction());
	}
}

The interesting API elements are

To give jqwik something to falsify, we broke the implementation of clear() so that it won’t clear everything if there are more than two elements on the stack:

public void clear() {
    // Wrong implementation to provoke falsification for stacks with more than 2 elements
    if (elements.size() > 2) {
        elements.remove(0);
    } else {
        elements.clear();
    }
}

Running the property should now produce a result similar to:

org.opentest4j.AssertionFailedError: Run failed after following actions:
    push(AAAAA)
    push(AAAAA)
    push(AAAAA)
    clear
  final state: ["AAAAA", "AAAAA"]

Number of actions

jqwik will vary the number of generated actions according to the number of tries of your property. For the default of 1000 tries a sequence will have up to 32 actions. If need be you can specify the number of actions to generate using either the fluent interface or the @Size annotation:

@Property
// check stack with sequences of 7 actions:
void checkMyStack(@ForAll("sequences") @Size(max = 7) ActionSequence<MyStringStack> actions) {
    actions.run(new MyStringStack());
}

The minimum number of generated actions in a sequence is 1 since checking an empty sequence does not make sense.

Check Invariants

We can also add invariants to our sequence checking property:

@Property
void checkMyStackWithInvariant(@ForAll("sequences") ActionSequence<MyStringStack> actions) {
    actions
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isGreaterThanOrEqualTo(0))
        .withInvariant(stack -> Assertions.assertThat(stack.size()).isLessThan(5))
        .run(new MyStringStack());
}

If we first fix the bug in MyStringStack.clear() our property should eventually fail with the following result:

org.opentest4j.AssertionFailedError: Run failed after following actions:
    push(AAAAA)
    push(AAAAA)
    push(AAAAA)
    push(AAAAA)
    push(AAAAA)
  final state: ["AAAAA", "AAAAA", "AAAAA", "AAAAA", "AAAAA"]

Assumptions

If you want to constrain the set of generated values in a way that embraces more than one parameter, filtering does not work. What you can do instead is putting one or more assumptions at the beginning of your property.

The following property works only on strings that are not equal:

@Property
boolean comparingUnequalStrings( //
        @ForAll @StringLength(min = 1, max = 10) String string1, //
        @ForAll @StringLength(min = 1, max = 10) String string2 //
) {
    Assume.that(!string1.equals(string2));

    return string1.compareTo(string2) != 0;
}

This is a reasonable use of Assume.that(boolean condition) because most generated value sets will pass through.

Have a look at a seemingly similar example:

@Property
boolean findingContainedStrings( //
        @ForAll @StringLength(min = 1, max = 10) String container, //
        @ForAll @StringLength(min = 1, max = 5) String contained //
) {
    Assume.that(container.contains(contained));

    return container.indexOf(contained) >= 0;
}

Despite the fact that the property condition itself is correct, the property will most likely fail with the following message:

org.opentest4j.AssertionFailedError: 
    Property [findingContainedStrings] exhausted after [1000] tries and [980] rejections

tries = 1000 
checks = 20 
seed = 1066117555581106850

The problem is that - given a random generation of two strings - only in very few cases one string will be contained in the other. jqwik will report a property as exhausted if the ratio between generated and accepted parameters is higher than 5. You can change the maximum discard ratio by specifying a parameter maxDiscardRatio in the @Property annotation. That’s why changing to @Property(maxDiscardRatio = 100) in the previous example will probably result in a successful property run, even though only a handful cases - of 1000 generated - will actually be checked.

In many cases turning up the accepted discard ration is a bad idea. With some creativity we can often avoid the problem by generating out test data a bit differently. Look at this variant of the above property, which also uses Assume.that() but with a much lower discard ratio:

@Property
boolean findingContainedStrings_variant( //
        @ForAll @StringLength(min = 5, max = 10) String container, //
        @ForAll @IntRange(min = 1, max = 5) int length, //
        @ForAll @IntRange(min = 0, max = 9) int startIndex //
) {
    Assume.that((length + startIndex) <= container.length());

    String contained = container.substring(startIndex, startIndex + length);
    return container.indexOf(contained) >= 0;
}

Result Shrinking

If a property could be falsified with a generated set of values, jqwik will try to “shrink” this sample in order to find a “smaller” sample that also falsifies the property.

Try this property:

@Property
boolean stringShouldBeShrunkToAA(@ForAll @AlphaChars String aString) {
    return aString.length() > 5 || aString.length() < 2;
}

The test run result should look something like:

AssertionFailedError: Property [stringShouldBeShrunkToAA] falsified with sample ["AA"]

tries = 38 
checks = 38 
seed = -633877439388930932 
sample = ["AA"]
original-sample ["LVtyB"]

In this case the originalSample could be any string between 2 and 5 chars, whereas the final sample should be exactly AA since this is the shortest failing string and A has the lowest numeric value of all allowed characters.

Integrated Shrinking

jqwik’s shrinking approach is called integrated shrinking, as opposed to type-based shrinking which most property-based testing tools use. The general idea and its advantages are explained here.

Consider a somewhat more complicated example:

@Property
boolean shrinkingCanTakeLong(@ForAll("first") String first, @ForAll("second") String second) {
    String aString = first + second;
    return aString.length() > 5 || aString.length() < 2;
}

@Provide
Arbitrary<String> first() {
    return Arbitraries.strings()
        .withCharRange('a', 'z')
        .ofMinLength(1).ofMaxLength(10)
        .filter(string -> string.endsWith("h"));
}

@Provide
Arbitrary<String> second() {
    return Arbitraries.strings()
        .withCharRange('0', '9')
        .ofMinLength(0).ofMaxLength(10)
        .filter(string -> string.length() >= 1);
}

Shrinking still works, although there’s quite a bit of filtering and string concatenation happening:

AssertionFailedError: Property [shrinkingCanTakeLong] falsified with sample ["h", "0"]

checks = 20 
tries = 20 
seed = -5596810132893895291 
sample = ["h", "0"]
original-sample ["gh", "774"]

Switch Shrinking Off

Sometimes shrinking takes a really long time or won’t finish at all (usually a jqwik bug!). In those cases you can switch shrinking off for an individual property:

@Property(shrinking = ShrinkingMode.OFF)
void aPropertyWithLongShrinkingTimes(
	@ForAll List<Set<String>> list1, 
	@ForAll List<Set<String>> list2
) {	... }

Switch Shrinking to Full Mode

Sometimes you can find a message like

shrinking bound reached =
    steps : 1000
    original value : [blah blah blah ...]
    shrunk value   : [bl bl bl ...]

in your testrun’s output. This happens in rare cases when jqwik has not found the end of its search for simpler falsifiable values after 1000 iterations. In those cases you can try

@Property(shrinking = ShrinkingMode.FULL)

to tell jqwik to go all the way, even if it takes a million steps, even if it never ends…

Change the Shrinking Target

By default shrinking of numbers will move towards zero (0). If zero is outside the bounds of generation the closest number to zero

Consider generating signals with a standard frequency of 50 hz that can vary by plus/minus 5 hz. If possible, shrinking of falsified scenarios should move towards the standard frequency. Here’s how the provider method might look:

@Provide
Arbitrary<List<Signal>> signals() {
	Arbitrary<Long> frequencies = 
	    Arbitraries
            .longs()
            .between(45, 55)
            .shrinkTowards(50);

	return frequencies.map(f -> Signal.withFrequency(f)).list().ofMaxSize(1000);
}

Currently shrinking targets are supported for integral numbers and decimal numbers, i.e., bytes, shorts, integers, longs, floats, doubles, BigIntegers and BigDecimals.

Collecting and Reporting Statistics

In many situations you’d like to know if jqwik will really generate the kind of values you expect and if the frequency and distribution of certain value classes meets your testing needs. Statistics.collect() is made for this exact purpose.

In the most simple case you’d like to know how often a certain value is being generated:

@Property
void simpleStats(@ForAll RoundingMode mode) {
    Statistics.collect(mode);
}

will create an output similar to that:

[MyTest:simpleStats] (1000) statistics = 
    FLOOR       (158) : 16 %
    HALF_EVEN   (135) : 14 %
    DOWN        (126) : 13 %
    UP          (120) : 12 %
    HALF_UP     (118) : 12 %
    CEILING     (117) : 12 %
    UNNECESSARY (117) : 12 %
    HALF_DOWN   (109) : 11 %

More typical is the case in which you’ll classify generated values into two or more groups:

@Property
void integerStats(@ForAll int anInt) {
    Statistics.collect(anInt > 0 ? "positive" : "negative");
}

[MyTest:integerStats] (1000) statistics = 
    negative (506) : 51 %
    positive (494) : 49 %

You can also collect the distribution in more than one category and combine those categories:

@Property
void combinedIntegerStats(@ForAll int anInt) {
    String posOrNeg = anInt > 0 ? "positive" : "negative";
    String evenOrOdd = anInt % 2 == 0 ? "even" : "odd";
    String bigOrSmall = Math.abs(anInt) > 50 ? "big" : "small";
    Statistics.collect(posOrNeg, evenOrOdd, bigOrSmall);
}

[MyTest:combinedIntegerStats] (1000) statistics = 
    negative even big   (222) : 22 %
    positive even big   (201) : 20 %
    positive odd big    (200) : 20 %
    negative odd big    (194) : 19 %
    negative even small ( 70) :  7 %
    positive odd small  ( 42) :  4 %
    negative odd small  ( 38) :  4 %
    positive even small ( 33) :  3 %

And, of course, you can combine different generated parameters into one statistical group:

@Property
void twoParameterStats(
    @ForAll @Size(min = 1, max = 10) List<Integer> aList, //
    @ForAll @IntRange(min = 0, max = 10) int index //
) {
    Statistics.collect(aList.size() > index ? "index within size" : null);
}

[MyTest:twoParameterStats] (1000) statistics = 
    index within size (507) : 51 %

As you can see, collected null values are not being reported.

Here are a couple of examples to try out.

Labeled Statistics

If you want more than one statistic in a single property, you must give them labels for differentiation:

@Property
void severalStatistics(@ForAll @IntRange(min = 1, max = 10) Integer anInt) {
    String range = anInt < 3 ? "small" : "large";
    Statistics.label("range").collect(range);
    Statistics.label("value").collect(anInt);
}

produces the following reports:

[MyTest:labeledStatistics] (1000) range = 
    large (783) : 78 %
    small (217) : 22 %

[MyTest:labeledStatistics] (1000) value = 
    1  (115) : 12 %
    5  (109) : 11 %
    10 (105) : 11 %
    4  (103) : 10 %
    2  (102) : 10 %
    3  ( 99) : 10 %
    6  ( 97) : 10 %
    8  ( 92) :  9 %
    7  ( 91) :  9 %
    9  ( 87) :  9 %

Providing Default Arbitraries

Sometimes you want to use a certain, self-made Arbitrary for one of your own domain classes, in all of your properties, and without having to add @Provide method to all test classes. jqwik enables this feature by using Java’s java.util.ServiceLoader mechanism. All you have to do is:

jqwik will then add an instance of your arbitrary provider into the list of its default providers. Those default providers are considered for every test parameter annotated with @ForAll that has no explicit value. By using this mechanism you can also replace the default providers packaged into jqwik.

Simple Arbitrary Providers

A simple provider is one that delivers arbitraries for types without type variables. Consider the class Money:

public class Money {
	public BigDecimal getAmount() {
		return amount;
	}

	public String getCurrency() {
		return currency;
	}

	public Money(BigDecimal amount, String currency) {
		this.amount = amount;
		this.currency = currency;
	}

	public Money times(int factor) {
		return new Money(amount.multiply(new BigDecimal(factor)), currency);
	}
}

If you register the following class MoneyArbitraryProvider:

package my.own.provider;

public class MoneyArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Money.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		Arbitrary<BigDecimal> amount = Arbitraries.bigDecimals() //
				  .between(BigDecimal.ZERO, new BigDecimal(1_000_000_000)) //
				  .ofScale(2);
		Arbitrary<String> currency = Arbitraries.of("EUR", "USD", "CHF");
		return Collections.singleton(Combinators.combine(amount, currency).as(Money::new));
	}
}

in file META-INF/services/net.jqwik.api.providers.ArbitraryProvider with such an entry:

my.own.provider.MoneyArbitraryProvider

The following property will run without further ado - regardless the class you put it in:

@Property
void moneyCanBeMultiplied(@ForAll Money money) {
    Money times2 = money.times(2);
    Assertions.assertThat(times2.getCurrency()).isEqualTo(money.getCurrency());
    Assertions.assertThat(times2.getAmount())
        .isEqualTo(money.getAmount().multiply(new BigDecimal(2)));
}

Arbitrary Providers for Parameterized Types

Providing arbitraries for generic types requires a little bit more effort since you have to create arbitraries for the “inner” types as well. Let’s have a look at the default provider for java.util.Optional<T>:

public class OptionalArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isOfType(Optional.class);
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		TypeUsage innerType = targetType.getTypeArguments().get(0);
		return subtypeProvider.apply(innerType).stream() //
			.map(Arbitrary::optional)
			.collect(Collectors.toSet());
	}
}

Mind that provideFor returns a set of potential arbitraries. That’s necessary because the subtypeProvider might also deliver a choice of subtype arbitraries. Not too difficult, is it?

Arbitrary Provider Priority

When more than one provider is suitable for a given type, jqwik will randomly choose between all available options. That’s why you’ll have to take additional measures if you want to replace an already registered provider. The trick is to override a provider’s priority() method that returns 0 by default:

public class AlternativeStringArbitraryProvider implements ArbitraryProvider {
	@Override
	public boolean canProvideFor(TypeUsage targetType) {
		return targetType.isAssignableFrom(String.class);
	}

	@Override
	public int priority() {
		return 1;
	}

	@Override
	public Set<Arbitrary<?>> provideFor(TypeUsage targetType, SubtypeProvider subtypeProvider) {
		return Collections.singleton(Arbitraries.constant("A String"));
	}
}

If you register this class as arbitrary provider any @ForAll String will be resolved to "A String".

Create your own Annotations for Arbitrary Configuration

All you can do to constrain default parameter generation is adding another annotation to a parameter or its parameter types. What if the existing parameters do not suffice your needs? Is there a way to enhance the set of constraint annotations? Yes, there is!

The mechanism you can plug into is similar to what you do when providing your own default arbitrary providers. That means:

  1. Create an implementation of an interface, in this case ArbitraryConfigurator.
  2. Register the implementation using using Java’s java.util.ServiceLoader mechanism.

Arbitrary Configuration Example: @Odd

To demonstrate the idea let’s create an annotation @Odd which will constrain any integer generation to only generate odd numbers. First things first, so here’s the @Odd annotation together with the configurator implementation:

@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER, ElementType.TYPE_USE })
@Retention(RetentionPolicy.RUNTIME)
public @interface Odd {
}

public class OddConfigurator extends ArbitraryConfiguratorBase {
	public Arbitrary<Integer> configure(Arbitrary<Integer> arbitrary, Odd odd) {
		return arbitrary.filter(number -> Math.abs(number % 2) == 1);
	}
}

Mind that the implementation uses an abstract base class - instead of the interface itself - which simplifies implementation if you’re only interested in a single annotation.

If you now register the implementation, the following example will work:

@Property
boolean oddIntegersOnly(@ForAll @Odd int aNumber) {
    return Math.abs(aNumber % 2) == 1;
}

There are a few catches, though:

Domain and Domain Context

Until now you have seen two ways to specify which arbitraries will be created for a given parameter:

In many cases both approaches can be tedious to set up or require constant repetition of the same annotation value. There’s another way that allows you to collect a number of arbitrary providers (and also arbitrary configurators) in a single place, called a DomainContext and tell a property method or container to only use providers and configurators from those domain contexts that are explicitly stated in a @Domain(Class<? extends DomainContext>) annotation.

As for ways to implement domain context classes have a look at DomainContext and AbstractDomainContextBase.

Domain example: American Addresses

Let’s say that US postal addresses play a crucial role in the software that we’re developing. That’s why there are a couple of classes that represent important domain concepts: Street, State, City and Address. Since we have to generate instances of those classes for our properties, we collect all arbitrary provision code in AmericanAddresses. Now look at this example:

class AddressProperties {

	@Property
	@Domain(AmericanAddresses.class)
	void anAddressWithAStreetNumber(@ForAll Address anAddress, @ForAll int streetNumber) {
	}

	@Property
	@Domain(AmericanAddresses.class)
	void globalDomainNotPresent(@ForAll Address anAddress, @ForAll String anyString) {
	}

	@Property
	@Domain(DomainContext.Global.class)
	@Domain(AmericanAddresses.class)
	void globalDomainCanBeAdded(@ForAll Address anAddress, @ForAll String anyString) {
	}
}

The first two properties above will resolve their arbitraries solely through providers specified in AmericanAddresses, whereas the last one also uses the default (global) context. Since AmericanAddresses does not configure any arbitrary provider for String parameters, property method globalDomainNotPresent will fail with a CannotFindArbitraryException.

Generation from a Type’s Interface

Some domain classes are mostly data holders. They come with constructors or factory methods to create them and you might want to create different instances by “just” filling the constructors’ parameters with values that are themselves generated. Using the building blocks you’ve seen until now requires the use of Arbitrary.map() or even Combinators.combine(...).as(...) to invoke the relevant constructor(s) and/or factories yourself. There’s a simpler way, though…

Consider a simple Person class:

public class Person {

	private String name;
	private final int age;

	public Person(String name, int age) {
		if (name == null || name.trim().isEmpty())
			throw new IllegalArgumentException();
		if (age < 0 || age > 130)
			throw new IllegalArgumentException();

		this.name = name;
		this.age = age;
	}

	@Override
	public String toString() {
		return String.format("%s (%d)", name, age);
	}
}

A first step to use arbitrarily generated Person objects without having to write a lot of jqwik-specific boiler plat code could look like that:

@Property
void aPersonsIsAlwaysValid(@ForAll @UseType Person aPerson) {
    Assertions.assertThat(aPerson.name).isNotBlank();
    Assertions.assertThat(aPerson.age).isBetween(0, 130);
}

Notice the annotation @UseType which tells jqwik to use the type information of Person to generate it. By default the framework will use all public constructors and all public, static factory methods in the class in order to generate instances. Whenever there’s an exception during generation they will be ignored; that way you’ll only get valid instances.

There are quite a few ways usage and configuration options. Have a look at the complete example and check the following api entry points:

Implement your own Arbitraries and Generators

In your everyday property testing you will often get along without ever implementing an arbitrary yourself. In cases where constraining default generation through annotations does not cut it, you can use all the mechanisms to configure, (flat-)map, filter and combine the pre-implemented arbitraries.

However, there are a few circumstances when you should think about rolling your own implementation. The most important of which are:

In those - and maybe a few other cases - you can implement your own arbitrary. To get a feel for what a usable implementation looks like, you might start with having a look at some of the internal arbitraries:

Under the hood, most arbitraries use RandomGenerators for the final value generation. Since RandomGenerator is a SAM type, most implementations are just lambda expression. Start with the methods on RandomGenerators to figure out how they work.

Since the topic is rather complicated, a detailed example will one day be published in a separate article…

Exhaustive Generation

Sometimes it is possible to run a property method with all possible value combinations. Consider the following example:

@Property
boolean allSquaresOnChessBoardExist(
    @ForAll @CharRange(from = 'a', to = 'h') char column,
    @ForAll @CharRange(from = '1', to = '8') char row
) {
    return new ChessBoard().square(column, row).isOnBoard();
}

The property is supposed to check that all valid squares in chess are present on a new chess board. If jqwik generated the values for column and row randomly, 1000 tries might or might not produce all 64 different combinations. Why not change strategies in cases like that and just iterate through all possible values?

This is exactly what jqwik will do:

Exhaustive generation is considered for:

Data-Driven Properties

In addition to the usual randomized generation of property parameters you have also the possibility to feed a property with preconceived or deterministically generated parameter sets. Why would you want to do that? One reason might be that you are aware of some problematic test cases but they are rare enough that jqwik’s randomization strategies don’t generate them (often enough). Another reason could be that you’d like to feed some properties with prerecorded data - maybe even from production. And last but not least there’s a chance that you want to check for a concrete result given a set of input parameters.

Feeding data into a property is quite simple:

@Data
Iterable<Tuple2<Integer, String>> fizzBuzzExamples() {
    return Table.of(
        Tuple.of(1, "1"),
        Tuple.of(3, "Fizz"),
        Tuple.of(5, "Buzz"),
        Tuple.of(15, "FizzBuzz")
    );
}

@Property
@FromData("fizzBuzzExamples")
void fizzBuzzWorks(@ForAll int index, @ForAll String result) {
    Assertions.assertThat(fizzBuzz(index)).isEqualTo(result);
}

All you have to do is annotate the property method with @FromData("dataProviderReference"). The method you reference must be annotated with @Data and return an object of type Iterable<? extends Tuple>. The Table class is just a convenient way to create such an object, but you can return any collection or create an implementation of your own.

Keep in mind that the Tuple subtype you choose must conform to the number of @ForAll parameters in your property method, e.g. Tuple.Tuple3 for a method with three parameters. Otherwise jqwik will fail the property and tell you that the provided data is inconsistent with the method’s parameters.

Data points are fed to the property in their provided order. The tries parameter of @Property will constrain the maximum data points being tried. Unlike parameterized tests in JUnit4 or Jupiter, jqwik will report only the first falsified data point. Thus, fixing the first failure might lead to another falsified data point later on. There is also no shrinking being done for data-driven properties since jqwik has no information about the constraints under which the external data was conceived or generated.

Rerunning Falsified Properties

When you rerun properties after they failed, they will - by default - use the previous random seed so that the next run will generate the exact same parameter data and thereby expose the same failing behaviour. This simplifies debugging and regression testing since it makes a property’s falsification stick until the problem has been fixed.

If you want to, you can change this behaviour for a given property like this:

@Property(afterFailure = AfterFailureMode.RANDOM_SEED)
void myProperty() { ... }

The afterFailure property can have one of four values:

You can also determine the default behaviour of all properties by setting the defaultAfterFailure property in the configuration file to one of those enum values.

jqwik Configuration

jqwik will look for a file jqwik.properties in your classpath in which you can configure a few basic parameters:

database = .jqwik-database          # The database to store data of previous runs
defaultTries = 1000                 # The default number of tries for each property
defaultMaxDiscardRatio = 5          # The default ratio before assumption misses make a property fail
useJunitPlatformReporter = false    # Set to true if you want to use platform reporting
defaultAfterFailure = PREVIOUS_SEED # Set default behaviour for falsified properties:
                                    # PREVIOUS_SEED, SAMPLE_ONLY or SAMPLE_FIRST
reportOnlyFailures = false          # Set to true if only falsified properties should be reported
defaultGeneration = AUTO            # Set default behaviour for generation:
                                    # AUTO, RANDOMIZED, or EXHAUSTIVE

Release Notes

Read this version’s release notes.