Errors & Reporting¶

Structured errors¶

A failing assertion raises AssertionFailure, a subclass of AssertionError that carries structured data. Existing except AssertionError handlers keep working unchanged.

from assertpy2 import AssertionFailure, assert_that

try:
    assert_that(1).is_equal_to(2)
except AssertionFailure as e:
    print(e.actual)     # 1
    print(e.expected)   # 2

Catching the subclass is what gives a type checker the structured fields: except AssertionError still runs, but types e as the base class, which declares none of them.

For comparisons, a DiffResult with path-level entries is attached:

try:
    assert_that({"a": 1, "b": 2}).is_equal_to({"a": 1, "b": 99})
except AssertionFailure as e:
    assert e.diff is not None   # every comparison failure carries one, other failures do not
    entry = e.diff.entries[0]
    print(entry.path, entry.actual, entry.expected)   # b 2 99

Paths a program can follow¶

entry.path is written for a person and cannot be read back. A mapping key goes through str(), so {3: "a"} and {"3": "a"} produce the same text, and a key holding a dot or a bracket has no grammar to parse it with.

entry.steps is the same location in the form a program can act on: a tuple of Step, each holding the key, index, field name, set member or line number itself, untouched.

from typing import Any

data = {"users": [{"roles": {7: "admin"}}]}

try:
    assert_that(data).is_equal_to({"users": [{"roles": {7: "guest"}}]})
except AssertionFailure as e:
    assert e.diff is not None
    entry = e.diff.entries[0]
    print(entry.path)                                  # users[0].roles.7
    print([(s.kind, s.value) for s in entry.steps])
    # [('key', 'users'), ('index', 0), ('key', 'roles'), ('key', 7)]

    cursor: Any = e.actual   # `actual` is `object`: the library does not know your payload's shape
    for step in entry.steps:
        cursor = cursor[step.value]
    print(cursor)                                      # admin

An empty steps means the difference is the whole value, which is the entry whose path renders as .. It is also empty where the path is a label rather than a location: a containment failure reports missing and extra, and an item that is not in a collection has no position in it.

A sequence whose two sides have shifted apart is the one case where an index alone is ambiguous, since the index spaces no longer agree. There the step names its side (actual or expected), matching the actual[2] / expected[1] the path renders.

The diff is also rendered into the failure message, so it travels with str(e) wherever the exception surfaces - unittest, a plain script, an AssertionError in a CI log:

try:
    assert_that({"a": 1, "b": 2}).is_equal_to({"a": 1, "b": 99})
except AssertionError as e:
    print(e)
    # Expected <{.., 'b': 2}> to be equal to <{.., 'b': 99}>, but was not.
    # diff (dict):
    #   b:
    #     - 2
    #     + 99

Each .. in that message stands for a run of parts that matched, and stands where that run was, so a one-field change in a wide object reads as {.., 'b': 2, ..} rather than as both objects in full.

Sequences collapse the same way once they grow past a line or so, which keeps a single changed element out of a forty-item dump:

try:
    assert_that(list(range(40))).is_equal_to([*range(27), 999, *range(28, 40)])
except AssertionError as e:
    print(e)
    # Expected <[.., 27, ..]> to be equal to <[.., 999, ..]>, but was not.
    # diff (sequence):
    #   [27]:
    #     - 27
    #     + 999

Sequences are aligned before they are compared, so an element inserted or removed is reported as itself rather than as a mismatch at every index after it:

try:
    assert_that([0, *range(1, 40)]).is_equal_to(list(range(1, 40)))
except AssertionError as e:
    print(e)
    # Expected <[0, ..]> to be equal to <[..]>, but was not.
    # diff (sequence):
    #   actual[0]: - 0

Comparing position by position would have called all forty elements different. A one-sided entry names the sequence its index belongs to, because once the two sides have shifted their index spaces no longer agree.

The alignment is used only where a sequence changed length, and then only where it reads shorter than the plain index comparison. A reversal, a coordinate pair and anything else of unchanged length keep the positional reading, so a comparison that could not benefit does not pay for the attempt. Sequences over a thousand elements are not aligned at all.

A multi-line value is collapsed by line, which matters most: every line of it costs a row of terminal, and the message carries the value twice.

try:
    assert_that("line 1\nline 2\nline 3\nline 4").is_equal_to("line 1\nline 2\nline THREE\nline 4")
except AssertionError as e:
    print(e)
    # Expected <.., line 3: line 3, ..> to be equal to <.., line 3: line THREE, ..>, but was not.

Short values are printed whole, since collapsing them would hide context to save a few characters.

When almost nothing matches there is little to collapse, so the message names the first few differences and counts the rest as ... and N more.

Either way every difference stays in the diff, so the shorter message loses nothing.

Matcher-based assertions (matches_structure(), satisfies(), each()) attach a DiffResult with kind='match', where each entry's expected holds the failed predicate's description.

assert_conforms() attaches one of the same kind. A validation failure has an entry per pydantic error. An exact=True failure has an entry per undeclared field, holding the value the payload sent there with absent == "expected", or a single entry for the part that cannot be checked, with the reason in expected. The steps lead into the payload. Two kinds of step hold an object of their own rather than a key or an index into the one before. item holds a member of a collection that keeps no positions, or the mapping key a find lies inside. json stands where the value is JSON text (Json[...]) and holds what the text decodes to, the object the steps after it lead into. A reader takes step.value at both:

def resolve(value, steps):
    for step in steps:
        value = step.value if step.kind in ("item", "json") else value[step.value]
    return value

resolve(payload, entry.steps) is then the very object entry.actual holds, text or not, for a payload validation left as it was sent. Where a validator changed the payload in place, the payload is returned to what validation left, so an entry that holds one of its dicts, lists or tuples holds a copy of what was sent instead. An entry inside JSON text is not copied: it keeps the decoded value reached by following the steps after its json step.

Under pytest the plugin attaches this same diff to the failure as its own section instead, keeping the message itself to a single line so the diff is never shown twice. The section travels with the failure, so the terminal, a JUnit report and an IDE runner such as PyCharm's all show it, and so does a failing fixture. When failures arrive as an exception group, as two fixtures failing in teardown do, each one gets its own section, numbered in the order the traceback prints them: Structured Diff (2 of 3). A soft block's failure gets the sections of each failure it collected, as What a soft block hands back shows. The terminal colors it as it draws the failure, when it takes color at all, so a JUnit report, an IDE runner and a CI log without forced color all get plain text. The plugin is auto-registered through the pytest11 entry point and needs no configuration.

See Rich pytest diffs for supported types and configuration.

Rich pytest diffs¶

When is_equal_to() or contains()/contains_exactly() fail, the DiffResult on the exception is rendered by the plugin as diff sections on the failure.

Type	Diff kind	How it works
`list`, `tuple`	`sequence`	Element-by-element, recursive into nested dicts, lists, dataclasses, namedtuples, attrs classes, and models
`set`, `frozenset`	`set`	Extra and missing items
`str`	`string`	Line-by-line, with difflib carets marking the exact intra-line change
`dict`	`dict`	Key-by-key, recursive into nested dicts, lists, dataclasses, namedtuples, attrs classes, and models
`dataclass`	`dataclass`	Field-by-field, handles differing types with overlapping fields
`namedtuple`	`namedtuple`	Field-by-field comparison
Pydantic model	`model`	Field-by-field on the values the fields and extras hold, not `model_dump()`, recursive into nested models
attrs class	`attrs`	Field-by-field, recursive into nested fields
other	`scalar`	Single actual-vs-expected entry
`contains` family	`contains`	Missing and extra items, plus the repeat counts for a duplicate failure
matcher mismatch	`match`	`satisfies()`, `each()`, `all_satisfy()`, `any_satisfy()`, `satisfies_exactly()`, `zip_satisfies()`, `matches_structure()`, `all_fields_satisfy()`, `assert_conforms()`: path + failed predicate

--- AssertionFailure ---
  actual:   [{'id': 1, 'name': 'Alice'}, {'id': 2, 'name': 'Bob'}]
  expected: [{'id': 1, 'name': 'Alice'}, {'id': 2, 'name': 'Robert'}]

That section carries the values the assertion named. Every failure holds the value under test on failure.actual, but most messages open with it, so a contains_key() failure shows the diff without repeating its subject above it.

For a failed is_equal_to() under ignore= or include= the two rows are the views that were compared: without the keys left out, and a dataclass or a model as the fields read of it. failure.actual stays the value you passed. Beside tolerance=, comparators=, strict_types= or ignore_null= the rows stay the two values.

The diff for that failure, and the other diff shapes, render like this.

The comparison is recursive, so for a very large or deeply nested value it walks the whole object graph.

When the payload is huge and you care about a few fields, extract those and assert on them instead. It is faster and the failure stays focused. See When full structural comparison is too much.

What each diff kind looks like¶

Value diffs (sequence, dict, dataclass, namedtuple, attrs, Pydantic model, string, scalar) show the path with the removal in red and the addition in green. This is the diff for the example above:

Colored sequence diff: [1].name with the removal in red and the addition in green

String values go finer than line-by-line. Each changed line is diffed within the line, with difflib carets (? ^^^) pointing at the exact span, the same guides pytest's own assertion rewriting uses.

bytes and bytearray take that path too, pointed at through their b'...' form:

Colored string diff: the changed word marked with difflib carets, removal in red and addition in green

The carets appear only when the two lines still resemble each other, at difflib's own 0.75 similarity cutoff.

Past that the two lines are printed plainly. A guide underlining most of the line says nothing the pair of values does not already say.

Set and contains show extra items in red and missing items in green:

Colored set diff: extra items in red, missing items in green

Match (the matcher-driven assertions listed in the table above) shows each field's path and the predicate that failed, with the actual value in red. A field assert_conforms(..., exact=True) found undeclared has no predicate, and is shown as path: - value.

Every mismatch is listed, not just the first. There is no green, since a predicate has no "addition".

Colored match diff: each field's path, the failed predicate, and the actual value in red

Nested structures are diffed recursively and report the exact path to the differing value (for example [1].name). A value that leads back into itself is read as far as the two sides differ, and the way back adds no row. Where == holds two values apart and nothing inside them differs, as with two instances of a class that compares by identity, the row is the pair itself.

Two values can render to the same text and still not be equal, most often because they differ only in type.

The message then tags each with its type, so assert_that("1").is_equal_to(1) reads Expected <1:str> to be equal to <1:int>, but was not. rather than a baffling <1> / <1>.

Note

Cycle detection covers every walker this library owns:

the diff rendering, and the comparison under an option (ignore, include, strict_types, tolerance, comparators), which treat a pair met again inside itself as equal rather than recursing, so two values are compared as the trees they unfold into. A key option applies at its own level of a cyclic value as it does of a flat one: ignore="id" leaves out the id of the value compared, not the id of what it leads back to
a contract snapshot, which records a revisited node as <circular ref>
a value snapshot, which fails with a message naming the cycle, since json cannot represent one

The bare equality check is not ours: it follows Python's own ==, so two structurally equal cyclic graphs raise RecursionError exactly as a plain assert a == b would.

The comparison settings are named back to you¶

ignore and include appear inside the failure sentence. The rest of the comparison settings get a line of their own, so a failure under tolerance, comparators, ignore_null or strict_types says which of them were in force:

from assertpy2 import assert_that

try:
    assert_that({"price": 1.0}).is_equal_to({"price": 9.0}, tolerance=0.001, strict_types=True)
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # compared with tolerance=0.001, strict_types=True

Nothing is added on an ordinary failure, where every setting sits at its default. The line goes below the sentence rather than into it, so the original text stays a prefix and a match= written against it keeps matching.

Some failures say why, not only what¶

A diff says what differs. There are failures where that leaves the hard part undone: two strings that render identically and differ in a trailing space, or a comparison that could never have passed because a NaN was in it. Where the whole difference has one explanation, it gets a line:

from assertpy2 import assert_that

try:
    assert_that({"user": {"name": "bob "}}).is_equal_to({"user": {"name": "bob"}})
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # every difference here is one of surrounding whitespace

The line appears only when it accounts for every entry in the diff. A half-explanation is worse than none: you would act on it and land back at the same failure. So a value that is simply wrong gets nothing, and neither does a failure where one field differs by whitespace and another by its content.

The one a JSON payload produces more than any other is a field that came back as text. The diff shows - 7 against + '7' on every row, and the line says it once:

from assertpy2 import assert_that

payload = {"id": 7, "quantity": 3}

try:
    assert_that(payload).is_equal_to({"id": "7", "quantity": "3"})
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # every difference here is the same text against a value of another type

A single value compared on its own says it in the sentence instead, tagging each side with its type (Expected <7:int> to be equal to <7:str>), so the line stays away rather than repeating it.

NaN is stated first whatever else differs, since no value on the other side could have made that comparison pass:

from assertpy2 import assert_that

try:
    assert_that(float("nan")).is_equal_to(float("nan"))
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # a NaN takes part in this comparison, and a NaN is equal to nothing, not even itself

The other failure that leaves a reader stuck is two values that print the same and are not equal. A class that defines no __eq__ compares by identity, so an expected value your test built is never equal to the one the code returned, however well they agree:

from assertpy2 import assert_that


class Order:
    def __init__(self, id, total):
        self.id, self.total = id, total


try:
    assert_that(Order(7, 10.0)).is_equal_to(Order(7, 10.0))
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # these values compare with object's __eq__, so equality is identity and no two separate instances are equal

It is said before anything about the values, for the same reason NaN is. No value on the other side would have made the comparison pass, so a line about how the two differ would send you to fix something that cannot help.

Exceptions are the everyday case, since they carry identity equality too. Your own comparators= entry over those values silences it, the verdict there being your predicate's rather than the type's.

The same holds one level down. A row of the diff whose two sides print the same is never left to stand alone. Two sides of two classes are each printed with its class, and with its module where the classes share a name:

from assertpy2 import assert_that


class Kept(dict):
    pass


try:
    assert_that({"a": Kept(x=1)}).is_equal_to({"a": {"x": 1}}, strict_types=True)
except AssertionError as failure:
    print("\n".join(str(failure).splitlines()[-2:]))
    #     - {'x': 1}:Kept
    #     + {'x': 1}:dict

Two sides of one class that print the same get a line instead. Where the class leaves __eq__ to object and no comparators= took part, the line says so of the row: a difference here prints the same on both sides, and the class of the two leaves __eq__ to object, so two separate instances are never equal. Anywhere else it says what is known, that the repr does not show the difference, and lists the attributes of the two that differ:

from assertpy2 import assert_that


class Money:
    def __init__(self, amount, currency):
        self.amount, self.currency = amount, currency

    def __eq__(self, other):
        return (self.amount, self.currency) == (other.amount, other.currency)

    def __repr__(self):
        return f"Money({self.amount})"


try:
    assert_that({"total": Money(5, "EUR")}).is_equal_to({"total": Money(5, "USD")})
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # a difference here prints the same on both sides, so what holds the two apart is not in their repr (attributes that differ: currency)

Where the differences repeat down a sequence, a line says where they sit. A place is the path with every position written [*], and every entry of the diff is counted, the ones past the fiftieth that are not printed included:

from assertpy2 import assert_that

rows = [{"id": n, "name": f"user{n}", "updated_at": f"10:00:{n:02d}"} for n in range(51)]
other = [dict(row, updated_at=f"11:00:{n:02d}") for n, row in enumerate(rows)]
other[50]["name"] = "renamed"

try:
    assert_that(rows).is_equal_to(other)
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # the 52 differences here are at <[*].updated_at> (51) and <[*].name> (1)

It is said for three or more differences at one to three places, under whatever line says why. It names where and recommends nothing: whether updated_at belongs under ignore= is your call.

A failed contains() is explained the same way, and so is every assertion that looks for something. Where one fact about the item not found accounts for it, a line says so in the words of the assertion that failed: the item is a NaN, a candidate reads the same and is of another plain type, or a candidate prints the same and their class compares by identity. That holds for contains(), contains_key(), contains_only(), contains_exactly(), contains_exactly_in_any_order(), contains_only_once(), contains_sequence(), contains_in_order(), is_in(), is_subset_of(), contains_value() and contains_entry().

All of them but the two that walk in order, the keys and the entry also name the closest row with what differs, where a row shares a key with the item: for dicts and for dataclasses, attrs instances, named tuples and models. Both are said of one item not found. Several would each need their own.

Where the thing not found is a key, and the value holds one that reads almost the same, the line names it. contains_key(), contains() on a mapping, contains_entry() and has_<name>() say it, the last of an attribute as well:

Expected <{'user_id': 2, 'email': 'a@b'}> to contain key <userId>, but did not.
a key reads the same but for how its words are joined: <user_id>

Three things are looked for, the narrowest first. A key is the same once both are lowercased. A key is the same once both are lowercased and what joins their words is taken out (a space, an underscore, a dot, a hyphen), userId beside user_id. One spelling alone is almost the same: two neighbouring letters swapped, a letter too many or too few, or the two nearly one text, a difflib ratio of 0.85 once lowercased and unjoined, which a long name reaches with a few letters dropped or replaced. The first two are facts. The third is a likeness and is worded as one. Where two spellings come that close nothing is said, and keys that differ only in case or joining are one spelling, named together: the first three, and how many more there are.

The line is said of one key not found: where several are missing none is named. Only keys that are plain text are read, and of an object only the public names it holds. The likeness is looked for among names of a hundred characters at most. A has_<name>() asked under not_ for a name that is not there fails as it does without not_, the line included. Over the keys of three real payloads, a key asked for with one slip was named in 99.3% of 2529 cases, and no other key ever was.

contains_exactly() and contains_exactly_in_any_order() count. Where everything missing is one item the value does hold, the failure says the two counts of that item instead: <'7'> is held 1 time and was asked for 2 times. With nothing missing, where everything left over is one item that was asked for, it says the two counts of that one.

import json

from assertpy2 import assert_that

payload = json.loads('{"ids": [7, 8]}')

try:
    assert_that(payload["ids"]).contains("7")
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # an element reads the same as the item not found and is of another type: int, not str

try:
    assert_that("7").is_in(*payload["ids"])
except AssertionError as failure:
    print(str(failure).splitlines()[1])
    # a given item reads the same as the value and is of another type: int, not str

A failed contains_entry() says what the key holds, or that the key is not there. A failed has_<name>() fails as is_equal_to() on the value it read: the failure carries that value, the operand and their diff, and two values that print the same are told apart by class. Two dicts are printed as is_equal_to() prints them, the keys that match collapsed to .., and so are two lists or two plain tuples too long to print whole, unless the two collapsed texts read alike or the values cannot be compared a second time: then both are printed as texts, cut around the place they part. Where the two cannot be read a second time the diff is one entry, the pair itself.

A text held against a text is explained the same way. Where the relation asked holds once one thing is set aside, the line names that thing:

Expected <Total: 42 EUR> to contain item <Total: 42>, but did not.
the value holds it once every run of whitespace is read as one space

The space after Total: in the value is a no-break space, which the sentence cannot show.

starts_with() and ends_with() name whitespace at that end of the value (a single \n, \r\n or \r is called a line break), case, whitespace at an end of the text given, or that the value holds the text somewhere else. contains() on a text names case, or that every run of whitespace has to be read as one space, which is what a line break, a tab, a no-break space and two spaces in a row all come to, or both. starts_with_ignoring_case(), ends_with_ignoring_case() and contains_ignoring_case() say all of that but case. is_equal_to_ignoring_case() names the whitespace that still holds the two apart, and is_equal_to_ignoring_whitespace() names case. For two texts is_equal_to() named line endings and surrounding whitespace before: it names the kind or amount of whitespace too, past those two.

One fact is said, the narrowest. Two at once get no line, a leading space and another case for starts_with(): only contains() has a line for case and whitespace together, and equality one for line endings and surrounding whitespace. These lines are said of plain text alone: a str of a class of its own, bytes and the first element of a list get none. is_equal_to() is the exception, and reads bytes and a str of any class as it did before.

Two moments say how far apart they are. is_before(), is_after() and their _or_equal_to twins give the distance on the other side, or say that the two are the same moment:

Expected <2026-01-01 12:00:00+00:00> to be before <2026-01-01 13:30:00+02:00>, but was not.
the value is 0:30:00 after the moment given

is_close_to() and is_not_close_to() on two datetimes give the distance and how far it is from the tolerance:

Expected <2026-01-01 12:00:00> to be close to <2026-01-01 12:00:07> within tolerance <0:00:05>, but was not.
the two are 0:00:07 apart, 0:00:02 more than the tolerance

The distance is datetime's own subtraction, exact to the microsecond, printed as a timedelta prints. It is said where datetime itself made the verdict. A subclass that writes its own comparison, a pandas Timestamp for one, gets no line, and one that writes its own - or +, as pendulum's does, gets the line of the four ordering assertions and not the one of closeness. The tolerance has to be exactly a timedelta.

is_less_than(), is_greater_than() and their _or_equal_to forms, which take two datetimes as well, say of them what is_before() and its kin say. is_between() says nothing.

Two datetimes that share one zone object are compared by Python on their wall clocks, which is the time between the two only while the zone keeps one offset. Across a change of its clocks it is not, and the two readings of a repeated hour compare equal. The line then speaks of the clock and not of the moments:

Expected <2026-10-25 02:30:00+02:00> to be before <2026-10-25 02:30:00+01:00>, but was not.
the two read the same on the clock they share

Two in different zones are compared as the moments they are, and the line gives the time between them. The line itself asks a zone for its offset only where the zone is datetime.timezone or zoneinfo.ZoneInfo. With a zone of another library, pytz's or dateutil's, two that share it get the line about their clock, and two in different zones get none.

A third line appears when the value came from an HTTP response, naming the request it answered:

Expected <500> to be equal to <200> on attribute <status_code>, but was not.
from GET https://api.example.com/orders/7 -> 500

It survives the step into the body, so a failure under decoded_as_json() still says which call produced it. It reads no body and starts no I/O, which is why a streaming response that has never been read is safe to assert on.

Like the settings echo above, these sit below the sentence, so anything written against the original message keeps working.

Polling failures carry a trace¶

An eventually() timeout attaches its convergence telemetry as failure.trace - a PollTrace with per-poll PollSample entries and a one-line trend summary. See Polling trace.

The comparison window of an IDE¶

pytest calls the hook pytest_assertrepr_compare for a failed assert left == right, and a test runner that opens a two-pane comparison listens to it. A failed is_equal_to() is the same comparison, so the plugin hands the two values it compared to those listeners, the value under test as left and the operand as right.

PyCharm's runner is one of them. Run from PyCharm, a failed assert_that(actual).is_equal_to(expected) gets the "Click to see difference" link a plain assert actual == expected gets. The window it opens shows the operand under Expected and the value under test under Actual, both whole. Seen in PyCharm 2026.3 on a response of forty nested rows. As for a plain assert, PyCharm also prints the two values above the link.

What is handed over is the pair the assertion compared:

Failed assertion	The two panes show
`is_equal_to()`	the value and the operand
`is_equal_to()` under `ignore=` or `include=`	the two copies without the keys left out, so a key the assertion passed over is in neither pane
`has_<name>()`	the value it read and the operand

A record in such a copy, a dataclass, an attrs class or a model, is handed over as the dict of its fields, which is what was compared. A pane then lays a record out line by line, and a dict beside a record reads in the same shape. That holds for a mapping whose keys are each exactly a str, an int, a bool, a float, a bytes or None. A mapping with any other key, one of a subclass of those among them, is handed over as it was.

The window compares two texts, as it does for a plain assert:

two values == holds equal that print differently, 1 and True, show there as a difference
two that differ and print the same, two NaNs, show as none

The failure's own diff is the one built from the comparison.

Nothing is handed over in three cases:

under tolerance=, comparators=, strict_types= or ignore_null=, where a text comparison of two values would show a difference the assertion let pass, or none where it found one
for a negation, a membership assertion or a soft_assertions() block
for a group of several failures

The failure is what it was there: its message, failure.diff and the report sections do not change.

Who hears the hook:

It is called while the report of the test is being made, before it is logged. A listener that reads the pair when the report is logged hears it, which is how PyCharm's runner reads.
Under pytest-xdist it is called in the worker, and a runner that reports from the controller does not hear it.
assertpy2_comparison = "off" turns the hand-over off.

Catching failures with their types intact¶

pytest.raises(AssertionError) types the caught exception as plain AssertionError, so a type checker flags .actual / .expected / .diff access. Catch AssertionFailure instead. It is the subclass every failure is raised as, including a soft block's aggregate and fail(), so one handler covers the library:

import pytest
from assertpy2 import AssertionFailure, assert_that

def test_diff_is_machine_readable():
    with pytest.raises(AssertionFailure) as exc_info:
        assert_that({"role": "guest"}).is_equal_to({"role": "admin"})

    failure = exc_info.value  # typed as AssertionFailure
    diff = failure.diff
    assert diff is not None   # optional on the class, present on every comparison failure
    assert_that(diff.kind).is_equal_to("dict")
    assert_that(diff.entries[0].path).is_equal_to("role")

The rich diff comes from the fluent form. The == drop-in for matchers (for example assert response == {"id": match.is_positive()}) hands rendering to pytest instead, which prints its own dict comparison without the path.

What the failures had in common¶

Forty failing tests are usually not forty problems. Where three or more of them differ at the same place, the run ends with a line saying where:

assertpy2 failure clusters:
  37 of 40 failing tests differ at user.role
      actual:   'superadmin'
      expected: 'admin'
  3 of 40 outside any cluster of 3

You get that without configuring anything. Three is the floor rather than a rule, and it moves:

[tool.pytest.ini_options]
assertpy2_failure_clusters = "5"   # failing tests a cluster must hold before it is printed, or "off"

That is a hypothesis you can act on before opening a single traceback, and it comes from the same structured diff the report sections are built from. Nothing about it is a guess: two failures share a cluster when their difference is equal, never when it looks similar. Where a difference has no place to be keyed on, what is compared is the text the values print as, so two values with the same repr count as one.

Failures are grouped by the place they differ at, not by the values they showed. A broken constant gives every failure the same pair, and grouping on values would find it. A broken formula differs at the same field with a value of its own per test, and grouping on values would scatter it into forty clusters of one. Where a cluster's values disagree, the summary says so rather than printing the first pair as though it explained all of them:

  12 of 12 failing tests differ at order.total
      actual:   118.4 and 11 other values
      expected: 120.0

A row index is generalised away, so users[0].role and users[7].role are one difference reported against two rows. A line number in a string comparison is generalised the same way.

What the summary will and will not claim:

Three kinds of difference carry no location. A containment failure names missing or extra, a set entry names a member, and a scalar failure is the whole value. Those group on the failure's own diagnostic line when it has one, so five uploads differing from their expected bodies only by a trailing newline read as one cause across five different payloads. They print as 5 of 5 failing tests share one scalar difference, because a family is not a place and a summary pointing at one would be inventing it.
Containment and set differences without a diagnostic stay out of it. Their two fields hold presence rather than values, so a missing item reports None on the actual side, and a heading built from that would tell you your value was None when it was a list of three.
The floor is a count, never a share of the run. Under a share, every additional cause raises the bar for all the others, and a run of forty failures splitting cleanly into five causes of eight would say nothing at all.
The share is measured against every test that went red, errors from a broken fixture included, so 10 of 13 outside any cluster of 3 is the honest reading of a run that was mostly environment. It names the floor because that is what it measures: two tests sharing a difference under a floor of three are related, and the summary declined to print them rather than found nothing.
Tests, not reports. A test that fails its assertion and then errors in teardown is one broken test, and pytest counts it in both of its own totals.

A collection that failed is red and is not a test, so it is named on its own line rather than folded into a count of tests: 1 collection error, not counted below. That only happens under --continue-on-collection-errors, since otherwise the run stops before anything is summarised.

Under pytest-xdist, a worker killed mid-run never ships what it recorded, and a worker running a different version of this library can ship something unreadable. Either way the summary says so rather than presenting a share of what it happened to receive:

assertpy2 failure clusters:
  1 worker died, so these counts cover only what was reported
  6 of 6 failing tests differ at user.role

Everything printed is bounded, because a diagnostic that grows with the run hurts the worst runs most:

values longer than 200 characters are cut
a cluster reports a floor rather than a count past 64 distinct values per side (and 64+ other values)
at most five clusters print, followed by a count of the rest

What a cluster is keyed on is not cut, since two payloads agreeing for their first 200 characters are still two failures. The failure message and the structured diff keep their values whole.

Nothing here can fail a run that would otherwise have passed or reported. If the summary cannot be built, it says so in a warning and the run's own results are untouched.

A run that passes pays nothing for it, since there is no failure to record. A run that fails walks each failed report carrying a diff once, measured at 2.3 µs over one differing entry and 0.11 ms over fifty, and builds the summary once for the whole run, 15 µs over forty failing tests. Turned off, the recorder returns on the configuration alone and nothing is read at all.

Configuration¶

Two of the settings below guard against a test that passes without checking anything: the dangling detector and the vacuous-quantifier warning. Both are off unless you ask, because either can turn a green run red, and a suite that inherited this library did not ask for that. A new suite gets both in one line:

[tool.pytest.ini_options]
assertpy2_profile = "safe"   # dangling on, vacuous on (default "compatible": both off)

Naming a setting yourself wins over the profile, so safe plus assertpy2_dangling = "off" is a suite that wants the other and has said so where the next reader will look.

Failure clustering is not one of them and needs no profile. It reads a run that has already gone red, so there is no green run for it to change. Write assertpy2_failure_clusters = "off" to turn it off.

Seeing them, and being stopped by them¶

safe warns. A run full of findings still exits zero, which is a fair first look and a poor gate. The exception is a suite that escalates warnings: under filterwarnings = ["error"] each finding turns its test red, as strict would.

strict turns on the same two guards and fails the tests they find:

[tool.pytest.ini_options]
assertpy2_profile = "strict"

The two read differently in the report, because they are found at different moments:

ERROR test_orders.py::test_the_total    DanglingAssertionWarning
FAILED test_orders.py::test_every_item  VacuousAssertionWarning

A dangling chain is found while the module is collected and reported when its test is set up, so the test never runs and pytest calls it an error. A vacuous pass is found while the assertion itself runs, so it is a failure. Both stop the run.

The escalation is scoped to this library's own two warnings. A DeprecationWarning from somewhere else in your suite is left exactly as it was, which a blanket filterwarnings = "error" would not do.

Failure clustering is not escalated and cannot be: it reads a run that already went red.

[tool.pytest.ini_options]
assertpy2_diff = "off"              # disable structured diff sections entirely
assertpy2_diff_max_entries = "100"  # max entries to show (default 50, 0 = unlimited)
assertpy2_comparison = "off"        # do not hand a failed equality to comparison listeners
assertpy2_poll_report = "off"       # silence the near-timeout poll report (default 0.7)
assertpy2_failure_clusters = "off"  # stop grouping failures that share one difference (default 3)
assertpy2_dangling = "on"           # warn about assert_that() statements that assert nothing
assertpy2_dangling_entries = "check"  # your own assert_that wrappers, for the check above
assertpy2_vacuous = "on"            # warn when a universal assertion passes over an empty value

In a terminal that takes color, or with --color=yes, diffs are colored on screen: red removals, green additions, cyan headers. Every other reader of the failure gets the same diff as plain text. Entries beyond the limit are hidden behind a ... and N more entries summary.

The plugin also adds assertpy2_allure (see Allure) and these command-line flags, each documented where it is used:

Flag	What it does
`--assertpy2-vacuous`	Warn when a universal assertion passes over an empty value, or `is_equal_to` compares nothing
`--assertpy2-dangling`	Warn when `assert_that()` is written as a statement that asserts nothing
`--assertpy2-snapshot-update`	Overwrite failing snapshots with the current values
`--assertpy2-snapshot-ci`	Fail instead of creating a missing snapshot (auto-enabled on CI)
`--assertpy2-snapshot-no-ci`	Disable CI mode and its autodetection

What a failure exposes¶

An assertion library's job is to put the values it compared in front of you, so a failing assertion is a deliberate disclosure.

That output does not stay in your terminal. It reaches CI logs, report attachments and, for snapshots, files committed to the repository. Below is what travels where, so you can decide what to hand the library in the first place.

assertpy2 caps size. It never redacts. Nothing here inspects a value to decide whether it looks like a credential, and nothing ever will: a masker that guesses wrong in the safe direction hides the difference you were debugging, and one that guesses wrong in the other direction is worse than no masker at all.

The channels¶

Channel	Carries	Reaches
the message, `str(failure)`	the differing values, capped	pytest output, CI logs, anything catching `AssertionError`
`failure.actual` / `failure.expected`	the whole untouched values	the pytest report section, Allure attachments, your own `except` block
`failure.diff`	per-path actual and expected, capped when rendered	the same
a snapshot file	the serialised value	`__snapshots/*.json`, normally committed

The caps are on rendering only. The value under test, a value it is held against, a pattern and a list of items are printed whole up to 4000 characters in every assertion. Past that the message says how much it left out, and what it keeps depends on what the assertion is about:

What a message prints	What it keeps past the cap
the two sides of `is_equal_to`	the first 4000 characters, with the rest left to its diff
two texts held against each other: `starts_with`, `ends_with`, the comparisons that ignore case or whitespace	a cut around the place they part
anything else	its first 3000 and last 1000
a name or a count you pass: a key, a class, a file, a length	all of it, as given

A diff has caps of its own:

Part of a diff	Cap
a row	400 characters
a row a `soft_assertions()` block keeps	4000 a side
the whole block	20 KB
matching parts of a structure	collapse to `..`

None of that shrinks what failure.actual holds, and failure.actual is what a reporting integration serialises.

Under pytest -vv a message and the values of the report section are printed whole, as pytest prints its own output there. Under ignore= or include= alone those are the two views compared, whole. The rows of a diff stay cut.

Keeping a value out¶

Excluding a key from the comparison also excludes it from the message and the diff:

from assertpy2 import AssertionFailure, assert_that

payload = {"user": "alice", "api_key": "sk-live-9f3b2a7c", "n": 1}
expected = {"user": "alice", "api_key": "sk-live-different", "n": 2}

try:
    assert_that(payload).is_equal_to(expected, ignore="api_key")
except AssertionFailure as failure:
    print("sk-live" in str(failure))   # False: the key was never compared, so it is not reported
    print("sk-live" in str(failure.actual))  # True: the object you passed is unchanged

ignore and include govern the report as well as the verdict, so the value stays out of the log. The exception still holds the original objects, because they are yours and the library does not copy them.

For anything stronger, redact before asserting rather than after. Compare a filtered copy, or wrap the value in a type whose __repr__ prints a placeholder: both put the decision where the meaning is known, which is your code and not ours.

Turning channels off¶

assertpy2_diff = "off" drops the structured diff section from pytest reports, and assertpy2_allure = "off" stops attaching values to Allure. Both leave the message itself intact, so neither is a substitute for not passing the value in.

Failure and expected exceptions¶

fail()¶

Force a test failure explicitly:

from assertpy2 import fail

fail("forced failure")

Expected exceptions¶

For a called function, assert it raises and chain assertions on the message:

assert_that(some_func).raises(RuntimeError).when_called_with("foo")
assert_that(some_func).raises(RuntimeError).when_called_with("foo").is_equal_to(
    "some err"
)

Or assert it does not raise a given exception:

assert_that(safe_func).does_not_raise(ValueError).when_called_with("foo")

Tip

For the common "did it raise?" case without inspecting the message, prefer pytest's pytest.raises context manager.

Beyond the message, the caught exception can be inspected in three more ways.

The exception object - raised() pivots to the caught exception itself, so you can assert on its type, args, or custom attributes, not only its message string:

err = assert_that(load).raises(ConfigError).when_called_with("bad.toml").raised().value
assert_that(err.code).is_equal_to(42)

A type checker reads err as the ConfigError that raises() named. After caused_by(), has_root_cause() or error_of() it reads the type that step asked for, since that is the exception the chain moved to.

The cause chain. caused_by() asserts the exception was chained from a given cause, either an explicit raise ... from or one raised during handling.

has_root_cause() walks to the root of the chain. Both pivot to that cause's message, so the chain continues:

# def save(row): ... raise ServiceError("save failed") from TimeoutError("db timeout")
assert_that(save).raises(ServiceError).when_called_with(row).caused_by(TimeoutError)
assert_that(save).raises(ServiceError).when_called_with(row).has_root_cause(
    TimeoutError
).is_equal_to("db timeout")

Either pivot also takes a union or a tuple of classes, as isinstance does, and holds when the cause is any one of them: caused_by(TimeoutError | ConnectionError). With no single class named, a type checker reads the cause as BaseException (Pyrefly keeps a union as the union).

Exception groups (ExceptionGroup, Python 3.11+, e.g. from an asyncio.TaskGroup). contains_error() asserts the caught group holds an exception of each given type, and does_not_contain_error() asserts it holds none of them. Both search the whole tree, so a group nested inside a group is reached.

To ask about the failures rather than their types, pivot. errors() hands back the leaves as a list, so every collection assertion applies to them, and error_of() picks the first exception of a type and continues on its message, with the object itself still one raised() away:

def run_tasks():
    raise ExceptionGroup("2 tasks failed", [ValueError("bad id"), KeyError("missing")])


caught = assert_that(run_tasks).raises(ExceptionGroup).when_called_with()
caught.contains_error(ValueError, KeyError)
caught.does_not_contain_error(TimeoutError)
caught.errors().is_length(2)
caught.error_of(ValueError).contains("bad id")

errors() flattens nesting, so a count is over the failures themselves and not over the shape the group happened to take. contains_error() and error_of() search the wider set, groups included, so the two always agree about whether a type is in there.

To assert the shape itself, matches_error_tree() takes one entry per member and a list for a nested group. Nothing may be left over on either side, and a plain list of types is the only spelling of a subgroup, since a tuple already means "any of these" elsewhere in the library.

Only a nested list looks inside a subgroup. A type entry matches a subgroup node the same way it matches a leaf, by isinstance, so matches_error_tree(Exception) holds for a group of one member whatever that member turns out to hold.

def run_tasks():
    raise ExceptionGroup("failed", [ValueError("bad id"), ExceptionGroup("retries", [KeyError("missing")])])


caught = assert_that(run_tasks).raises(ExceptionGroup).when_called_with()
caught.contains_error(ValueError, KeyError)
caught.matches_error_tree(ValueError, [KeyError])

contains_error() passes there whatever the nesting is. matches_error_tree() is the one that would notice KeyError moving out of retries.

Entries pair with members rather than being read in order, so the same claim written either way round gets the same verdict. matches_error_tree(Exception, ValueError) and matches_error_tree(ValueError, Exception) both hold for a group of one ValueError and one KeyError.

Expected warnings¶

For a called function, assert it emits a warning and chain assertions on the warning message:

assert_that(deprecated_func).warns(DeprecationWarning).when_called_with("foo")
assert_that(deprecated_func).warns(DeprecationWarning).when_called_with("foo").matches(
    "since 2.6"
)

The category defaults to Warning (matches any warning) and matches subclasses. Or assert it does not emit a given category:

assert_that(safe_func).does_not_warn(DeprecationWarning).when_called_with("foo")

To also assert on the value the call returned (alongside the warning, or after does_not_warn / does_not_raise), pivot with returned():

(
    assert_that(make_client).warns(DeprecationWarning).when_called_with()
    .returned().is_instance_of(Client)
)
(
    assert_that(adder).does_not_raise(TypeError).when_called_with(1, 2)
    .returned().is_equal_to(3)
)

returned() hands back a builder over the value the call returned, so that value's own assertions apply: contains() on a returned list, starts_with() on a returned string. A type checker reads its .value as the callable's declared return type. The builder is the whole one rather than the view assert_that() would pick for that type, so a checker accepts any assertion on it, and one the value cannot answer raises at run time. returned() raises TypeError if the call raised (there is no return value to inspect).

Not thread-safe

warns() / does_not_warn() rely on warnings.catch_warnings(), which mutates process-global state.

They are safe within a single thread, including multiple asyncio tasks on one event loop. Concurrent use across OS threads can interfere, the same limitation pytest.warns and unittest.assertWarns carry.

Custom error messages¶

described_as() prepends a custom label to the failure message:

assert_that(1 + 2).described_as("adding stuff").is_equal_to(2)
# [adding stuff] Expected <3> to be equal to <2>, but was not.

What a soft block hands back¶

A soft_assertions() block renders its collected failures into one numbered message. That message is a rendering, not the only copy: the raised AssertionFailure also carries failures, one AssertionOutcome per collected failure, in the order they were collected.

from assertpy2 import AssertionFailure, assert_that, soft_assertions

try:
    with soft_assertions():
        assert_that({"role": "guest"}).is_equal_to({"role": "admin"})
        assert_that("foo").is_length(4)
except AssertionFailure as e:
    print(len(e.failures))                       # 2
    first_diff = e.failures[0].diff
    assert first_diff is not None
    print(first_diff.entries[0].path)            # role
    print(e.failures[1].message)                 # Expected <foo> to be of length <4>, but was <3>.

Each record keeps what the text had flattened: the values, the diff, the group label it was collected under, and the (file, line) the message renders in brackets. A polling assertion that times out inside a soft block keeps its trace there too.

Off pytest the message is the only place a reader finds the diffs, so each entry carries its diff in the block form a single failure prints. For the block above, str(e) reads:

soft assertion failures:
1. Expected <{'role': 'guest'}> to be equal to <{'role': 'admin'}>, but was not.  [test_app.py:6]
   diff (dict):
     role:
       - 'guest'
       + 'admin'
2. Expected <foo> to be of length <4>, but was <3>.  [test_app.py:7]

One message has room for the blocks of twenty failures and 60 000 characters. A failure past that keeps the compact form, one line per differing path, and the last line of the message names it. A block too wide for what is left keeps the compact form too and takes one of the twenty, but none of the characters, so a smaller block after it still fits. Every block is a print away: print(e.failures[20].diff) renders it. diff is None on a failure that has none, such as is_true() on False. An exception of yours that escapes the block gets the same text as a note.

Under pytest the message keeps the compact form, and the plugin hangs the blocks on the report. Each collected failure gets the sections a raised failure gets, numbered as the message numbers it, Structured Diff (failure 2 of 3), and each section opens on that failure's line of the message, with its group in brackets. Under ignore= or include= a section shows the two copies the assertion compared, as it does for a raised failure. The room is the same, twenty failures and 60 000 characters, shared by every soft block that fails in one report. Each block with failures left out gets one more section naming them, and a failure with nothing to show takes no place among the twenty. An exception of yours that escapes the block brings along what the block had collected before it, and those failures get their sections too.

In the compact form, what each kind of value adds:

a scalar, or a short line of text, adds no line at all, since the entry above already carries both values in full
a long line adds one, cut to a window around the first difference, which is the part nobody finds by reading a 200-character payload twice

failures is empty on every other failure, which is about one value rather than a collection of them.

Asking instead of asserting¶

check() runs the next assertion for its verdict and hands back an AssertionOutcome instead of raising. It is truthy when the assertion held, and carries the message, values and diff when it did not.

outcome = assert_that({"role": "guest"}).check().is_equal_to({"role": "admin"})

print(bool(outcome))          # False
print(outcome.message)        # Expected <{'role': 'guest'}> to be equal to <{'role': 'admin'}>, but was not.
assert outcome.diff is not None
print(outcome.diff.entries[0].path)   # role

Use it where a test is not what you are writing: branching on a precondition, or reporting a check into something that is not pytest. An assertion states a requirement, and stopping at the first unmet one is the point of the other modes.

Negation is proxied, so assert_that(5).check().not_.is_positive() answers too. A bad argument still raises: TypeError and ValueError mean the call itself is wrong, which is not a verdict about the value.

Unlike a failure collected by soft_assertions() or logged by assert_warn(), a failed check does not mark the value as unverified: .value keeps working, because a question was asked, not an assertion made.

What was asked, as data¶

message says what failed in English, and actual, expected and diff say what the values were. None of them says what was asked of the value, and expected cannot, since what it holds changes with the assertion:

Assertion	`expected` holds
`is_equal_to(2)`	the operand
`contains(2)`	a tuple
`satisfies(...)`	a rendered description
`is_instance_of(str)`	a type
`is_empty()`	nothing at all

requirement answers that in one shape, on every failure, whichever mode delivered it:

outcome = assert_that(1).check().is_close_to(9, 0.1)
assert outcome.requirement is not None

print(outcome.requirement.operation)    # is_close_to
print(outcome.requirement.parameters)   # {'other': 9, 'tolerance': 0.1}
print(outcome.requirement.negated)      # False

The same three fields are on the raised AssertionFailure, on each entry of a soft block's failures, and in the Allure attachment. That is what lets a report group failures without parsing messages: every is_close_to failure in a run carries the same operation, and a negated one is marked rather than worded differently.

Two details worth knowing before you build on it:

Parameters are the values the assertion ran with, keyed by its own parameter names. One left out appears with its default, and is_close_to(9, 0.1) reads the same as is_close_to(other=9, tolerance=0.1).
An assertion that delegates answers its own name: is_positive() asks is_greater_than(0) underneath, and reports is_positive.

requirement is None where the failure is not one assertion's, and that is a limit rather than a gap:

Failure	Its `requirement`
`fail()`, `soft_fail()`, a bare `error()` carrying a message of your own	`None`, since they name no operation
a precondition of one of the few members that assert nothing on their own	`None`, since it is about the shape of the value rather than about a requirement you stated
what a soft block or `assert_all` raises for everything it collected	`None`, with the answer on each entry in its `failures`
a polling timeout	that of whatever its last attempt raised: usually the assertion that kept failing, and `None` when the probe raised something that is not one of this library's failures, such as an exception it was told to ignore or a bare `AssertionError`

Reading it costs a built-in assertion nothing when it passes, since nothing runs until a failure is composed. A registered extension is the exception: it says what it is at the call, so it pays a wrapper whether it passes or fails.

Warnings instead of failures¶

For defensive assertions outside tests, replace assert_that with assert_warn: failures log a warning instead of raising:

assert_warn("foo").is_length(4)   # logs a warning, does not raise

assert_warn() vs warns()

These are unrelated despite the similar names.

assert_warn(...) is a soft entry point. The assertion still checks your value, but logs a warning instead of raising on failure.

assert_that(func).warns(...) goes the opposite way. It asserts that calling func emits a Python warning.

Warning output and custom logger

2019-10-27 20:00:35 WARNING [app.py:42]: Expected <foo> to be of length <4>, but was <3>.

Pass your own logger for custom formatting:

assert_warn("foo", logger=my_logger).is_length(4)

The assertpy2 logger carries its own handler and also propagates, so if your suite calls logging.basicConfig() each warning is printed twice. Turning propagation off silences the duplicate:

logging.getLogger("assertpy2").propagate = False

That trade is yours to make: propagation is also how pytest's caplog fixture receives these records, so turning it off stops you asserting on them.

Handing each failure to your own code¶

on_failure takes a callable and hands it each failure as its AssertionOutcome instead of logging it: the message, the values, the diff, the requirement and the (file, line) the assertion was written at. A logger that should carry the record, diff included, takes it from there:

import logging

from assertpy2 import assert_warn

log = logging.getLogger("checks")


def report(outcome):
    text = outcome.message if outcome.diff is None else f"{outcome.message}\n{outcome.diff}"
    log.warning(text, extra={"assertpy2_outcome": outcome})


assert_warn({"status": 500}, on_failure=report).is_equal_to({"status": 200})

Every link of a chain is handed over, a pivot and extracting() included, and a polling timeout comes with its trace. The value is marked as unverified before your callable runs, as it is when a failure is logged. An exception your callable raises comes out of the assertion that failed, as one a logger raises does, an AssertionError included. on_failure takes the place of logger, and passing both is a TypeError. Through a LoggerAdapter the record keeps your extra only with merge_extra=True, which arrived in Python 3.13.