@test_sets L op R
@test_sets L op R broken=true
@test_sets L op R skip=true

Test that the expression L op R evaluates to true, where op is an infix operator interpreted as a set-like comparison:

• L == R expands to issetequal(L, R)
• L != R or L ≠ R expands to !issetequal(L, R)
• L ⊆ R or L ⊂ R expands to issubset(L, R)
• L ⊇ R or L ⊃ R expands to issubset(R, L)
• L ⊊ R expands to issubset(L, R) && !issetequal(L, R)
• L ⊋ R expands to issubset(R, L) && !issetequal(L, R)
• L ∩ R or L || R expands to isdisjoint(L, R)

Same return behaviour as Test.@test, namely: if executed inside a @testset, returns a Pass Result if expanded expression evaluates to true, a Fail Result if it evaluates to false, and an Error Result if it could not be evaluated. If executed outside a @testset, throws an exception instead of returning Fail or Error.

You can use any L and R that work with the expanded expressions above (including tuples, arrays, sets, dictionaries, strings, and more generable iterables). The ∅ symbol can also be used as shorthand for Set().

The only additional limitation is that setdiff(L, R) and intersect(L, R) must also work, since they are used to generate informative failure messages in some cases.

See also: Base.issetequal, Base.issubset, Base.isdisjoint, Base.setdiff, Base.intersect.

Examples

julia> @test_sets (1, 2) == (2, 1, 1, 1)
Test Passed

julia> @test_sets ∅ ⊆ 1:5
Test Passed

julia> @test_sets 1:3 ⊇ 5
Test Failed at none:1
  Expression: 1:3 ⊇ 5
   Evaluated: L is not a superset of R.
              R ∖ L has 1 element:  [5]

julia> @test_sets [1, 2, 3] ∩ [2, 3, 4]
Test Failed at none:1
  Expression: [1, 2, 3] ∩ [2, 3, 4] == ∅
   Evaluated: L and R are not disjoint.
              L ∩ R has 2 elements: [2, 3]

julia> @test_sets "baabaa" ≠ 'a':'b'
Test Failed at none:1
  Expression: "baabaa" ≠ 'a':'b'
   Evaluated: L and R are equal.
              L = R has 2 elements: ['b', 'a']

The macro supports broken=cond and skip=cond keywords, with similar behavior to Test.@test:

Examples

julia> @test_sets [1] ⊆ [2, 3] broken=true
Test Broken
  Expression: [1] ⊆ [2, 3]

julia> @test_sets [1] ⊆ [1, 2] broken=true
Error During Test at none:1
 Unexpected Pass
 Expression: [1] ⊆ [1, 2]
 Got correct result, please change to @test if no longer broken.

julia> @test_sets [1] ⊆ [2, 3] skip=true
Test Broken
  Skipped: [1] ⊆ [2, 3]

@test_all ex
@test_all f(args...) key=val ...
@test_all ex broken=true
@test_all ex skip=true

Test that the expression all(ex) evaluates to true. Does not short-circuit at the first false value, so that all false elements are shown in case of failure.

Same return behaviour as Test.@test, namely: if executed inside a @testset, returns a Pass Result if all(ex) evaluates to true, a Fail Result if it evaluates to false or missing, and an Error Result if it could not be evaluated. If executed outside a @testset, throws an exception instead of returning Fail or Error.

Examples

julia> @test_all [1.0, 2.0] .== [1, 2]
Test Passed

julia> @test_all [1, 2, 3] .< 2
Test Failed at none:1
  Expression: all([1, 2, 3] .< 2)
   Evaluated: false
    Argument: 3-element BitVector, 2 failures:
              [2]: 2 < 2 ===> false
              [3]: 3 < 2 ===> false

Similar to @test, the @test_all f(args...) key=val... form is equivalent to writing @test_all f(args...; key=val...) which can be useful when the expression is a call using infix syntax such as vectorized approximate comparisons:

julia> v = [0.99, 1.0, 1.01];

julia> @test_all v .≈ 1 atol=0.1
Test Passed

This is equivalent to the uglier test @test_all .≈(v, 1, atol=0.1). Keyword splicing also works through any negation operator:

julia> @test_all .!(v .≈ 1) atol=0.001
Test Failed at none:1
  Expression: all(.!.≈(v, 1, atol=0.001))
   Evaluated: false
    Argument: 3-element BitVector, 1 failure:
              [2]: !≈(1.0, 1, atol=0.001) ===> false

As with @test, it is an error to supply more than one expression unless the first is a call (possibly broadcast . syntax) and the rest are assignments (k=v).

The macro supports broken=true and skip=true keywords, with similar behavior to Test.@test:

julia> @test_all [1, 2] .< 2 broken=true
Test Broken
  Expression: all([1, 2] .< 2)

julia> @test_all [1, 2] .< 3 broken=true
Error During Test at none:1
 Unexpected Pass
 Expression: all([1, 2] .< 3)
 Got correct result, please change to @test if no longer broken.

julia> @test_all [1, 2, 3] .< 2 skip=true
Test Broken
  Skipped: all([1, 2, 3] .< 2)

set_max_print_failures(n=10)

Globaly sets the maximum number of individual failures that will be printed in a failed @test_all test to n. If n is nothing, all failures are printed. If n == 0, only a summary is printed.

By default, if there are more than n=10 failing elements in a @test_all, the macro only shows messages for the first and last 5. Calling this function changes n globally for all subsequent tests, or until the function is called again.

The function returns the previous value of n so that it can be restored if desired.

Examples

julia> @test_all 1:3 .== 0
Test Failed at none:1
  Expression: all(1:3 .== 0)
   Evaluated: false
    Argument: 3-element BitVector, 3 failures:
              [1]: 1 == 0 ===> false
              [2]: 2 == 0 ===> false
              [3]: 3 == 0 ===> false

julia> set_max_print_failures(0);

julia> @test_all 1:3 .== 0
Test Failed at none:1
  Expression: all(1:3 .== 0)
   Evaluated: false
    Argument: 3-element BitVector, 3 failures

disable_failure_styling()

Globally disable ANSI color styling in macro failure messages.

All tests macros that are run after this function will print failure messages in plain text (or until enable_failure_styling() is called).

The function can be called when the module is loaded, e.g. in a runtests.jl file, to disable styling throughout a test suite. Alternatively, it can be called at the beginning of a specific @testset and renabled at the end to disable styling for that specific test set.

See also enable_failure_styling.

enable_failure_styling()

Globally enable ANSI color styling in macro failure messages.

All test macros that are run after this function will print failure messages with ANSI color styling for readability (or until disable_failure_styling() is called).

See also disable_failure_styling.