Property Testing library, which allows for the specification and testing of properties of code.

Property-based testing is a methodology where you define general characteristics (properties) that your code should satisfy across a wide range of inputs, rather than asserting specific outputs for hardcoded inputs. This library automatically generates random inputs to test these properties, and if a failure occurs, it aggressively "shrinks" the input to find the minimal, simplest example that reproduces the bug.

Motivating Example

Imagine an e-commerce function that applies a coupon discount to a shopping cart total. A naive unit test might check applyDiscount(100, 20) == 80. A property test instead asserts universal truths about the function:

Example:

import experimental/property_testing
# import experimental/property_testing # required in other files

proc applyDiscount(total, discount: int): int =
  max(0, total - discount)

let result = runProperty:
  forAll (total: genInt(0, 1000), discount: genInt(0, 100)):
    let discounted = applyDiscount(total, discount)
    # Property 1: The discounted total is never greater than the original
    if discounted > total: return psFail
    # Property 2: The discounted total is never negative
    if discounted < 0: return psFail
    return psPass

assert result.status == psPass

Core Concepts

The library revolves around a few key types and concepts:

Properties (`forAll`): The idiomatic way to define a test is using the forAll procedures, which pair generators with a predicate function that returns a PropertyStatus (psPass, psFail, or psDiscard).
Generators (`Gen[T]`): Procedures that consume a Source of randomness to produce values of type T.
Shrinking: An automatic process that simplifies failing test cases.

Writing Property Tests

Tests are typically constructed using forAll and executed with runProperty. runProperty runs the scenario numerous times (default 256) with different seeds. If a failure (psFail) is encountered, the library automatically begins shrinking the generated inputs to find the most minimal reproducing case, which is then available in the TestResult.

Tests can return psDiscard if the generated inputs do not meet certain preconditions, effectively skipping that run without failing the test. For example, validating that a division function works correctly when the denominator is not zero.

Generators and Sources

To generate data, you build or compose Gen[T] procedures. The standard library provides many built-in generators:

Primitives: genInt, genBool, genByte, genChar, genString.
Collections: genSeq, genSet, genArray.
Ranges: genInt(min, max), genEnum.

You can compose and modify generators using combinators:

map: Transforms the output of a generator (e.g. generating even numbers by mapping x => x * 2).
filter: Discards values that don't meet a predicate. (Use sparingly, as too many retries raise FilterExhaustedError).
flatMap: Chains generators dependently.

Underlying all generation is the Source object. It provides the entropy for generators and records the sequence of choices made. This recording is what enables the library's powerful, integrated shrinking capabilities.

Integrated Shrinking

This library uses integrated shrinking (inspired by Hypothesis). Unlike traditional type-directed shrinking, this library shrinks the underlying byte stream (the Source buffer) that produced the values, rather than shrinking the typed values themselves.

This approach has several massive advantages:

You do not need to write custom shrink functions for your custom types.
Filtering and flatMap work perfectly and maintain invariants during shrinking, because the shrinking happens on the raw entropy before the combinators run.
It aggressively finds minimal examples using structural heuristics like sequence deletion, binary search on numeric ranges, and unbounded scalar lowering.

See the following posts on the hows and whys of integrated shrinking:

Imports

macros, math, options, random, hashes, sequtils, sugar, times, typetraits, enumutils, sets, strutils

Types

FilterExhaustedError = object of CatchableError: Error raised when a filter fails to find a valid value after its maximum number of retries. Source Edit
Gen[T] = proc (s: Source): T: Source Edit
Property[T] = object gen*: Gen[T] check*: proc (x: T): PropertyStatus: Represents a property to be tested, consisting of a generator and a predicate function. Source Edit
PropertyStatus = enum psPass, psFail, psDiscard, psError: Represents the result of a single property test execution. Source Edit
ScalarBytes = range[1 .. 8]: The number of bytes used to store a scalar value. Source Edit
Source = ref object rng: Rand buffer*: seq[byte] pos: int recording: bool limit*: int ## Max bytes to generate before stopping/erroring idempotent*: bool ## whether the generator consuming this source should ## be able to produce the same value given the same ## source state, i.e.: disabling exhaustiveness debug*: bool ## used for debugging: A source of randomness and a record of choices made by generators. Source Edit
SourceLimitExceededError = object of CatchableError: Error raised when a source exceeds its allocated byte limit. Source Edit
StorageKind = enum skByte, ## 1 byte sk2Bytes, ## 2 bytes sk4Bytes, ## 4 bytes sk8Bytes, ## 8 bytes skNBytes, ## N bytes skRange, ## Range of values skArray, ## Array of bytes, with size stored as a range in the ## immediately following bytes and elements thereafter skGroup ## Group of values, with size stored in the next byte: Defines the different types of data storage in the source buffer. Source Edit
TestResult[T] = object status*: PropertyStatus runCount*: int seed*: uint32 failingValue*: Option[T] failingBuffer*: seq[byte] errorMsg*: Option[string] shrunk*: bool shrunkValue*: Option[T] shrunkBuffer*: seq[byte] debugBuffer*: seq[byte]: The results of running a property test, including failure details and shrinking results if a failure was found. Source Edit

Consts

DefaultSourceLimit = 100000: Source Edit
defaultTrials = 1024: number of trials to run per property Source Edit

Procs

proc beginArray(s: Source; min, max: uint32): (uint32, uint32, int) {.
    ...raises: [SourceLimitExceededError], tags: [].}

Marks the beginning of an array of length in the range [min, max] of homogeneous elements in the stream, returning the chosen length, the absolute length from the source, and the position of the absolute length scalar in the buffer (for patching). Source Edit

proc beginFixedArray(s: Source; len: uint32): (uint32, uint32, int) {.
    ...raises: [SourceLimitExceededError], tags: [].}

Marks the beginning of a fixed-size array of length len of homogeneous elements in the stream. Source Edit

proc beginGroup(s: Source; numElements: uint8): uint8 {.
    ...raises: [SourceLimitExceededError], tags: [].}

Marks the beginning of a heterogeneous group (tuple, object) with numElements Source Edit

func bytesForRange(rangeSize: uint64): ScalarBytes {....raises: [], tags: [].}

Determines the number of bytes required to store a range of size rangeSize. Source Edit

proc chooseRange(s: Source; min, max: int64; scalarKind: ScalarStorageKind): int64 {.
    ...raises: [SourceLimitExceededError], tags: [].}

Produces a random int64 between min and max when recording, otherwise reads it from the buffer.

Mapping: Uses monotonic renumeration to map the signed range to an unsigned space centering 0.0 for high-quality shrinking.

Source Edit

proc chooseRange(s: Source; min, max: uint64; kind: ScalarStorageKind): uint64 {.
    ...raises: [SourceLimitExceededError], tags: [].}

Produces a random uint64 between min and max when recording, otherwise reads it from the buffer.

Structural Verification: Replay-phase reads verify structural compatibility via skRange tags.

Source Edit

proc chooseScalarRaw(s: Source; kind: ScalarStorageKind): uint64 {.
    ...raises: [SourceLimitExceededError], tags: [].}

Produces a random scalar of kind when recording, otherwise reads a scalar of kind from the buffer.

Structural Verification: During replay, this uses readStorageKind(kind) to ensure the buffer is structurally compatible. If the buffer is exhausted, it returns a safe default (0) without crashing.

Source Edit

proc decodeUint64(buffer: seq[byte]; pos: int; bytes: ScalarBytes): uint64 {.
    ...raises: [], tags: [].}

Reads bytes worth of bytes from buffer at position pos as a uint64. Source Edit

proc filter[T](g: sink Gen[T]; pred: sink proc (x: T): bool;
               maxRetries: int = 100): Gen[T]

Create a new generator based on g, that filters output based on the predicate procedure (pred), with maxRetries per filter attempt, raising a FilterExhaustedError if exceeded. Source Edit

proc flatMap[T, U](g: sink Gen[T]; f: sink proc (x: T): Gen[U]): Gen[U]

Takes the initial generator, and a factory function that creates a new generator based on the value generated by the initial generator.

This is useful for creating a generator that generates values based on the values generated by another generator.

Source Edit

proc float32ToOrdinal(f: float32): int32 {....raises: [], tags: [].}

Converts float f to an int32 ordinal. Source Edit

proc float64ToOrdinal(f: float64): int64 {....raises: [], tags: [].}

Maps float64 to monotonic int64 ordinal. +0.0 maps to 0. Source Edit

proc genArray[T](g: sink Gen[T]; size: static uint32): Gen[array[size, T]]

Create an array generator with element type T and static size size. Source Edit

proc genAsciiChar(): Gen[char] {....raises: [], tags: [].}

Create a char arbitrary for the ASCII range. Source Edit

proc genAsciiString(minLen: uint32 = 0; maxLen: uint32 = 100): Gen[string] {.
    ...raises: [], tags: [].}

Create an ASCII string generator. Source Edit

proc genBool(): Gen[bool] {....raises: [], tags: [].}

Create a boolean generator. Source Edit

proc genByte(): Gen[byte] {....raises: [], tags: [].}

Create a byte generator. Source Edit

proc genChar(): Gen[char] {....raises: [], tags: [].}

Create a char arbitrary for the full character range, see: genAsciiChar for the ASCII range. Source Edit

proc genChar(min, max: char): Gen[char] {....raises: [], tags: [].}

create a char arbitrary for the range [min, max]. Source Edit

proc genConst[T](v: T): Gen[T]

Create a generator that always returns v. Source Edit

proc genEnum[T: enum](): Gen[T]

Create an enum arbitrary of type T for the full uint64 range. Source Edit

proc genEnum[T: enum](min, max: T): Gen[T]

Generate an enum arbitrary of type T for the range [min, max]. Source Edit

proc genExhaustive[T](vals: sink seq[T]): Gen[T]

Creates a generator producing values that are part of vals. Source Edit

proc genFloat(min, max: float = NaN; allowNaN: bool = false;
              allowInf: bool = true; allowSubnormal: bool = true): Gen[float] {.
    ...raises: [], tags: [].}

Generate an float arbitrary for the range [min, max], with flags to control whether NaN, Inf, and Subnormal values are allowed. Source Edit

proc genFloat32(min, max: float32 = NaN; allowNaN: bool = false;
                allowInf: bool = true; allowSubnormal: bool = true): Gen[float32] {.
    ...raises: [], tags: [].}

Generate an float32 arbitrary for the range [min, max], with flags to control whether NaN, Inf, and Subnormal values are allowed. Source Edit

proc genFloat64(min, max: float64 = NaN; allowNaN: bool = false;
                allowInf: bool = true; allowSubnormal: bool = true): Gen[float64] {.
    ...raises: [], tags: [].}

Generate an float64 arbitrary for the range [min, max], with flags to control whether NaN, Inf, and Subnormal values are allowed. Source Edit

proc genFromList[T](vals: sink seq[T]; simplestIdx: int = 0): Gen[T]

Create a generator that draws from a fixed list of values, vals, with ranked shrinking targeting the simplestIdx. Source Edit

proc genInt(): Gen[int] {....raises: [], tags: [].}

Generate an int for the full range of int. Source Edit

proc genInt(min, max: int): Gen[int] {....raises: [], tags: [].}

Create an integer arbitrary for the range [min, max]. Source Edit

proc genInt8(): Gen[int8] {....raises: [], tags: [].}

Create an int8 arbitrary for the full int8 range. Source Edit

proc genInt8(min, max: int8): Gen[int8] {....raises: [], tags: [].}

Generate an int8 arbitrary for the range [min, max]. Source Edit

proc genInt16(): Gen[int16] {....raises: [], tags: [].}

Create an int16 arbitrary for the full int16 range. Source Edit

proc genInt16(min, max: int16): Gen[int16] {....raises: [], tags: [].}

Generate an int16 arbitrary for the range [min, max]. Source Edit

proc genInt32(): Gen[int32] {....raises: [], tags: [].}

Create an int32 arbitrary for the full int32 range. Source Edit

proc genInt32(min, max: int32): Gen[int32] {....raises: [], tags: [].}

Generate an int32 arbitrary for the range [min, max]. Source Edit

proc genInt64(): Gen[int64] {....raises: [], tags: [].}

Create an int64 arbitrary for the full int64 range. Source Edit

proc genInt64(min, max: int64): Gen[int64] {....raises: [], tags: [].}

Generate an int64 arbitrary for the range [min, max]. Source Edit

proc genProc1[T1, R](retGen: sink Gen[R]): Gen[proc (a: T1): R]

Create a generator for a procedure that takes one argument of type T1 and returns a value of type R produced by retGen. Source Edit

proc genProc2[T1, T2, R](retGen: sink Gen[R]): Gen[proc (a: T1; b: T2): R]

Create a generator for a procedure that takes two arguments of types T1 and T2, and returns a value of type R produced by retGen. Source Edit

proc genProc3[T1, T2, T3, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3): R]

Create a generator for a procedure that takes three arguments of types T1, T2, and T3, and returns a value of type R produced by retGen. Source Edit

proc genProc4[T1, T2, T3, T4, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4): R]

Create a generator for a procedure that takes four arguments of types T1, T2, T3, and T4, and returns a value of type R produced by retGen. Source Edit

proc genProc5[T1, T2, T3, T4, T5, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4; e: T5): R]

Create a generator for a procedure that takes five arguments of types T1, T2, T3, T4, and T5, and returns a value of type R produced by retGen. Source Edit

proc genProc6[T1, T2, T3, T4, T5, T6, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4; e: T5; f: T6): R]

Create a generator for a procedure that takes six arguments of types T1 through T6, and returns a value of type R produced by retGen. Source Edit

proc genProc7[T1, T2, T3, T4, T5, T6, T7, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4; e: T5; f: T6; g: T7): R]

Create a generator for a procedure that takes seven arguments of types T1 through T7, and returns a value of type R produced by retGen. Source Edit

proc genProc8[T1, T2, T3, T4, T5, T6, T7, T8, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4; e: T5; f: T6; g: T7; h: T8): R]

Create a generator for a procedure that takes eight arguments of types T1 through T8, and returns a value of type R produced by retGen. Source Edit

proc genProc9[T1, T2, T3, T4, T5, T6, T7, T8, T9, R](retGen: sink Gen[R]): Gen[
    proc (a: T1; b: T2; c: T3; d: T4; e: T5; f: T6; g: T7; h: T8; i: T9): R]

Create a generator for a procedure that takes nine arguments of types T1 through T9, and returns a value of type R produced by retGen. Source Edit

proc genProc10[T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, R](retGen: sink Gen[R]): Gen[proc (
    a: T1; b: T2; c: T3; d: T4; e: T5; f: T6; g: T7; h: T8; i: T9; j: T10): R]

Create a generator for a procedure that takes ten arguments of types T1 through T10, and returns a value of type R produced by retGen. Source Edit

proc genProc[R](retGen: sink Gen[R]): Gen[proc (): R]

Create a generator for a procedure that takes no arguments and returns a value of type R produced by retGen. Source Edit

proc genSeq[T](g: sink Gen[T]; minLen: uint32 = 0; maxLen: uint32 = 100): Gen[
    seq[T]]

Create a sequence generator with element type T and length in the range [minLen, maxLen]. Source Edit

proc genSet[T: enum](minLen: uint16 = 0; exclude: set[T] = {}): Gen[set[T]]

Create a set generator for the enum type T excluding the values in exclude. Source Edit

proc genString(minLen: uint32 = 0; maxLen: uint32 = 100;
               charGen: sink Gen[char] = genChar()): Gen[string] {....raises: [],
    tags: [].}

Create a string generator for the range [minLen, maxLen] using the given char generator. Source Edit

proc genTuple[T: tuple](gen: sink T): auto

Creates a tuple generator from a tuple of generators, with each element in the produced tuple coming from the input generator at the corresponding index. Source Edit

proc genUint8(): Gen[uint8] {....raises: [], tags: [].}

Create an uint8 arbitrary for the full uint8 range. Source Edit

proc genUint8(min, max: uint8): Gen[uint8] {....raises: [], tags: [].}

Generate an uint8 arbitrary for the range [min, max]. Source Edit

proc genUint16(): Gen[uint16] {....raises: [], tags: [].}

Create an uint16 arbitrary for the full uint16 range. Source Edit

proc genUint16(min, max: uint16): Gen[uint16] {....raises: [], tags: [].}

Generate an uint16 arbitrary for the range [min, max]. Source Edit

proc genUint32(): Gen[uint32] {....raises: [], tags: [].}

Create an uint32 arbitrary for the full uint32 range. Source Edit

proc genUint32(min, max: uint32): Gen[uint32] {....raises: [], tags: [].}

Generate an uint32 arbitrary for the range [min, max]. Source Edit

proc genUint64(): Gen[uint64] {....raises: [], tags: [].}

Create an uint64 arbitrary for the full uint64 range. Source Edit

proc genUint64(min, max: uint64): Gen[uint64] {....raises: [], tags: [].}

Generate an uint64 arbitrary for the range [min, max]. Source Edit

proc genVoidProc(): Gen[proc ()] {....raises: [], tags: [].}

Create a generator for a procedure that takes no arguments and returns nothing. Source Edit

proc getScalarBytes(kind: ScalarStorageKind): ScalarBytes {....raises: [], tags: [].}

Determines the number of bytes that kind requires. Source Edit

proc map[T, U](g: sink Gen[T]; f: sink proc (x: T): U): Gen[U]

Create a new generator based on g, using f to map values of the base generator. Source Edit

proc newSource(buffer: sink seq[byte]): Source {....raises: [], tags: [].}

Creates a source for replaying/shrinking with a fixed buffer. Source Edit

proc newSource(seed: uint32; limit: int = DefaultSourceLimit;
               idempotent: bool = false; debug: bool = false): Source {.
    ...raises: [], tags: [].}

Creates a source for recording. Source Edit

proc ordinalToFloat32(o: int32): float32 {....raises: [], tags: [].}

Converts ordinal value o to a float32. Source Edit

proc ordinalToFloat64(o: int64): float64 {....raises: [], tags: [].}

Converts ordinal value o to a float64. Source Edit

proc ordinalToRank(o: int64): uint64 {....raises: [], tags: [].}

Converts an ordinal o to an uint64. Source Edit

proc rankToOrdinal(r: uint64; uMin, uMax, uSimp: uint64): uint64 {....raises: [],
    tags: [].}

Maps a non-negative rank back to a uint64 value within [uMin, uMax], centering on uSimp. Source Edit

proc readGroupLength(s: Source): uint8 {....raises: [], tags: [].}

Parses a group marker and returns the number of fields Source Edit

proc readRawByte(s: Source): byte {....raises: [], tags: [].}

Read the current byte and advances to the next byte, if there are no more recorded buffer given the position, it returns zero. Source Edit

proc readRawBytes(s: Source; bytes: ScalarBytes): uint64 {....raises: [], tags: [].}

Read upto bytes worth of bytes and returns them. Source Edit

proc readStorageKind(s: Source; expected: StorageKind): StorageKind {.
    ...raises: [], tags: [].}

Reads the current byte as a StorageKind and advances to the next byte.

Structural Resilience: If the buffer is not exhausted, it asserts that the read kind matches expected. If the buffer is exhausted, it returns expected without asserting, allowing the caller to return a safe default value.

Source Edit

proc recordRange(s: Source; rangeSize, offset: uint64; kind: ScalarStorageKind) {.
    ...raises: [SourceLimitExceededError], tags: [].}

Records a structural range marker (skRange) followed by its metadata. Source Edit

proc rngNextBytes(s: Source; bytes: ScalarBytes): uint64 {....raises: [], tags: [].}

Generate and return bytes worth RNG bytes. Source Edit

proc runProperty[T](p: Property[T]; trials: int = defaultTrials;
                    seed: uint32 = 0; maxDiscards: int = -1; debug: bool = false): TestResult[
    T]

Runs the property p for trials iterations, using seed as the base seed. Returns a TestResult containing the status of the test, the number of trials run, the seed used, and the failing value and buffer if a failure was found. If seed is 0, the current time is used as the seed base. If debug is true, the property will record the first failing buffer in p.debugBuffer. Source Edit

proc sample[T](g: Gen[T]; source: Source; count: int): seq[T]

Sample a generator, g, with a given source for a count number of elements, returning a sequence of that length. Source Edit

proc skipNode(buffer: seq[byte]; startPos: int): int {....raises: [], tags: [].}

Parses the structural StorageKind at startPos and returns the index immediately after the fully encoded node (including all nested children). Source Edit

proc skipNodes(s: Source; count: int) {....raises: [], tags: [].}

Advances the source position past count nodes in the stream. Source Edit

proc treeRepr(buffer: seq[byte]): string {....raises: [], tags: [].}

Returns a string representation of the buffer tree, stored in buffer, mostly used for debugging and exploration. Source Edit

proc writeRawByte(s: Source; b: byte) {....raises: [SourceLimitExceededError],
                                        tags: [].}

When recording writes the byte b to the buffer, otherwise ignores it. Source Edit

proc writeRawBytes(s: Source; val: uint64; bytes: ScalarBytes) {.
    ...raises: [SourceLimitExceededError], tags: [].}

Writes upto bytes worth of bytes from val. Source Edit

proc writeStorageKind(s: Source; kind: StorageKind) {.
    ...raises: [SourceLimitExceededError], tags: [].}

Writes the storage kind, kind, to the buffer and advances to the next byte. Source Edit

Iterators

iterator candidates(buffer: seq[byte]): seq[byte] {....raises: [], tags: [].}: Take the buffer and produces a set of candidate buffers that are shrinks of the original. Source Edit

Macros

macro forAll(gens: untyped; check: untyped): untyped: forAll takes a tuple of generators and a check procedure body, and returns a Property[T] where T is a tuple type of all the generators. Source Edit
macro genVoidProcN(T: varargs[typedesc]): untyped: Create a generator for a procedure that takes arguments of types specified by T and returns nothing. Source Edit