`zarr_metadata.typed_json`

zarr_metadata.typed_json ¶

JSON checked against a TypedDict: the public door.

Every Zarr document and configuration this package describes is a TypedDict -- ZarrV3ArrayMetadataJSON, GzipCodecConfiguration -- and check makes any of them executable. It type-checks a JSON value against the TypedDict and gives back a value of it, or None, and every problem, each located in the value:

import json

from zarr_metadata import ZarrV3ArrayMetadataJSON
from zarr_metadata.typed_json import check

document, problems = check(json.loads(raw), ZarrV3ArrayMetadataJSON)
for problem in problems:
    print(problem.loc, problem.kind, problem.message)

A TypedDict reads as the typing spec defines it, however its module writes annotations:

A key is required when its Required or NotRequired qualifier says so, and otherwise when the total of the class that declared it does. The runtime's __required_keys__ cannot see a qualifier written as a string, as from __future__ import annotations writes every one; typeddict_keys reads the annotations evaluated, and can.
A key the TypedDict does not declare is what closed, extra_items or, when the class says neither, its bases make it: in a closed TypedDict it is reported, as unknown_key, and left out, and the value still comes back; with extra_items= it is checked as that type; in an open one it is kept.
Each annotation is evaluated in the module of the class that wrote it, as the spec has it, where get_type_hints would read what a subclass inherited in the subclass's module.
ReadOnly and Annotated are peeled wherever they are written; a NewType reads as the type it names, a type alias as the type it stands for, and a TypedDict or alias that holds itself as deep as the value goes.
A union of TypedDicts that each require a key as a Literal of values of their own is read by the branch that key names, and its problems are that branch's. Otherwise a value is read by the first branch it fits with no problem -- among TypedDicts, the one declaring the most of its keys -- and failing that, reported by the branch with the fewest problems.

What comes back is a value of the TypedDict: arrays as tuples, and each object a new dict of the keys its type admits. Problems are values, not exceptions: ValidationProblem(loc, message, kind), with kind one of missing_key, invalid_type, invalid_value, unknown_key and invalid_json, so a caller that tolerates a key the TypedDict does not declare can tell it from a wrong value.

check reads the shapes JSON takes and no others -- int, float for any number, bool, str, None, JSONValue, a Literal, tuple[T, ...] and tuple[T1, T2], a union, a TypedDict, Mapping[str, V], a NewType and a type alias -- and a TypedDict holding anything else is a TypeError naming the member, down to the TypedDict that holds it. So is one the spec itself refuses, such as a key both Required and NotRequired, and a generic TypedDict, since no runtime records what its type arguments bind in its bases' keys. A string annotation resolves only in its module, so a TypedDict written inside a function cannot name a class of that function. typing.TypedDict on Python 3.11 does not record a class's bases, so there a subclass evaluates what it inherits in its own module; typing_extensions.TypedDict records them on every version.

JSONValue `module-attribute` ¶

JSONValue = TypeAliasType(
    "JSONValue",
    int
    | float
    | bool
    | str
    | list["JSONValue"]
    | tuple["JSONValue", ...]
    | Mapping[str, "JSONValue"]
    | None,
)

A recursive type alias for JSON-encodable values.

Defined via TypeAliasType (rather than a plain TypeAlias) so the self-reference is a named recursion point that pydantic can resolve when building a TypeAdapter; a bare recursive TypeAlias raises PydanticUserError/RecursionError at validation time.

Loc `module-attribute` ¶

Loc: TypeAlias = tuple[str | int, ...]

Where in a document a value sits: the keys and indices down to it.

ProblemKind `module-attribute` ¶

ProblemKind = Literal[
    "missing_key",
    "invalid_type",
    "invalid_value",
    "invalid_json",
    "unknown_key",
]

Machine-readable classification of a ValidationProblem.

missing_key: a required key (document key or store key) is absent.
invalid_type: a value has the wrong structural type (e.g. a string where a mapping is required, a non-JSON-serializable object).
invalid_value: a value has an acceptable type but an invalid content (e.g. zarr_format: 2 in a v3 document, order: "Q").
invalid_json: bytes that do not decode as JSON.
unknown_key: a key an object's type does not declare, where the type says it is closed, as a closed TypedDict does. Whether a Zarr configuration is closed is rarely said (zarr-developers/zarr-specs#270 has been open since 2023), and many readers refuse such a key. It gets a kind of its own so that a caller who tolerates it can tell it from a wrong value, and so that it never masks the other findings about the same object.

all `module-attribute` ¶

__all__ = [
    "JSONValue",
    "Loc",
    "ProblemKind",
    "TypedDictKeys",
    "ValidationProblem",
    "check",
    "typeddict_keys",
]

TypedDictKeys `dataclass` ¶

What a TypedDict says of an object's keys, read as the typing spec defines it.

members holds every key it declares, its bases' included, with the type of the key's value -- qualifiers and Annotated metadata peeled -- and whether the key is required. extra_items is what any other key may hold: Never when the TypedDict is closed, object when it is open, and the extra_items type otherwise. declared is whether that was said, by the TypedDict or a base, rather than defaulted: a TypedDict that says nothing is open.

Source code in src/zarr_metadata/_typed_json.py

@dataclass(frozen=True, slots=True)
class TypedDictKeys:
    """What a TypedDict says of an object's keys, read as the typing spec defines it.

    `members` holds every key it declares, its bases' included, with the
    type of the key's value -- qualifiers and `Annotated` metadata peeled
    -- and whether the key is required. `extra_items` is what any other
    key may hold: `Never` when the TypedDict is closed, `object` when it
    is open, and the `extra_items` type otherwise. `declared` is whether
    that was said, by the TypedDict or a base, rather than defaulted: a
    TypedDict that says nothing is open.
    """

    members: Mapping[str, tuple[object, bool]]
    extra_items: object
    declared: bool

    @property
    def required(self) -> frozenset[str]:
        """The keys an object of this TypedDict must have."""
        return frozenset(key for key, (_, required) in self.members.items() if required)

    @property
    def closed(self) -> bool:
        """Whether a key it does not declare is not a key of the type: `closed=True`, or `extra_items=Never`."""
        return self.extra_items is Never or self.extra_items is NoReturn

    @property
    def open(self) -> bool:
        """Whether a key it does not declare may hold anything: the default, or `closed=False`."""
        return self.extra_items is object

closed `property` ¶

closed: bool

Whether a key it does not declare is not a key of the type: closed=True, or extra_items=Never.

declared `instance-attribute` ¶

declared: bool

extra_items `instance-attribute` ¶

extra_items: object

members `instance-attribute` ¶

members: Mapping[str, tuple[object, bool]]

open `property` ¶

open: bool

Whether a key it does not declare may hold anything: the default, or closed=False.

required `property` ¶

required: frozenset[str]

The keys an object of this TypedDict must have.

init ¶

__init__(
    members: Mapping[str, tuple[object, bool]],
    extra_items: object,
    declared: bool,
) -> None

ValidationProblem `dataclass` ¶

A single problem found in a value: where it is, what is wrong, and what kind of wrong.

loc is the path from the root of what was judged to the offending value, e.g. ("codecs", 0, "name") in a document, and an empty loc refers to that root. kind classifies the failure mode for programmatic dispatch; message is the human-readable description.

Source code in src/zarr_metadata/_json.py

@dataclass(frozen=True, slots=True)
class ValidationProblem:
    """A single problem found in a value: where it is, what is wrong, and what kind of wrong.

    `loc` is the path from the root of what was judged to the offending
    value, e.g. `("codecs", 0, "name")` in a document, and an empty `loc`
    refers to that root.
    `kind` classifies the failure mode for programmatic dispatch; `message`
    is the human-readable description.
    """

    loc: tuple[str | int, ...]
    message: str
    kind: ProblemKind

    def __post_init__(self) -> None:
        # The runtime half of the annotations: a rule written without a type
        # checker, as an extension's may be, fails where it builds a problem
        # rather than reporting one at a location that is not one.
        loc = cast("object", self.loc)
        if not isinstance(loc, tuple) or not all(
            isinstance(part, str) or (isinstance(part, int) and not isinstance(part, bool))
            for part in cast("tuple[object, ...]", loc)
        ):
            msg = f"a ValidationProblem's loc is a tuple of keys and indices, got {loc!r}"
            raise TypeError(msg)
        message = cast("object", self.message)
        if not isinstance(message, str):
            msg = f"a ValidationProblem's message is a string, got {message!r}"
            raise TypeError(msg)
        kind = cast("object", self.kind)
        if not isinstance(kind, str) or kind not in get_args(ProblemKind):
            msg = f"a ValidationProblem's kind is one of {get_args(ProblemKind)!r}, got {kind!r}"
            raise TypeError(msg)

    def __str__(self) -> str:
        location = ".".join(str(part) for part in self.loc) if self.loc else "<root>"
        return f"{location}: {self.message}"

kind `instance-attribute` ¶

kind: ProblemKind

loc `instance-attribute` ¶

loc: tuple[str | int, ...]

message `instance-attribute` ¶

message: str

init ¶

__init__(
    loc: tuple[str | int, ...],
    message: str,
    kind: ProblemKind,
) -> None

__post_init__ ¶

__post_init__() -> None

Source code in src/zarr_metadata/_json.py

def __post_init__(self) -> None:
    # The runtime half of the annotations: a rule written without a type
    # checker, as an extension's may be, fails where it builds a problem
    # rather than reporting one at a location that is not one.
    loc = cast("object", self.loc)
    if not isinstance(loc, tuple) or not all(
        isinstance(part, str) or (isinstance(part, int) and not isinstance(part, bool))
        for part in cast("tuple[object, ...]", loc)
    ):
        msg = f"a ValidationProblem's loc is a tuple of keys and indices, got {loc!r}"
        raise TypeError(msg)
    message = cast("object", self.message)
    if not isinstance(message, str):
        msg = f"a ValidationProblem's message is a string, got {message!r}"
        raise TypeError(msg)
    kind = cast("object", self.kind)
    if not isinstance(kind, str) or kind not in get_args(ProblemKind):
        msg = f"a ValidationProblem's kind is one of {get_args(ProblemKind)!r}, got {kind!r}"
        raise TypeError(msg)

str ¶

__str__() -> str

Source code in src/zarr_metadata/_json.py

def __str__(self) -> str:
    location = ".".join(str(part) for part in self.loc) if self.loc else "<root>"
    return f"{location}: {self.message}"

check ¶

check(
    value: object, shape: type[T], loc: Loc = ()
) -> tuple[T | None, tuple[ValidationProblem, ...]]

value type-checked as shape, a TypedDict: a value of it or None, and every problem.

value is refined to JSON first -- arrays as tuples, string keys, finite floats -- and then checked member by member, each problem located under loc. What comes back holds what shape admits and nothing else: a key a closed TypedDict does not declare is reported, as unknown_key, and left out, and the value still comes back. Anything else wrong and it does not. TypeError for a shape that is not a TypedDict, or holds something no parser reads.

Source code in src/zarr_metadata/_typed_json.py

def check(
    value: object, shape: type[T], loc: Loc = ()
) -> tuple[T | None, tuple[ValidationProblem, ...]]:
    """`value` type-checked as `shape`, a TypedDict: a value of it or None, and every problem.

    `value` is refined to JSON first -- arrays as tuples, string keys,
    finite floats -- and then checked member by member, each problem
    located under `loc`. What comes back holds what `shape` admits and
    nothing else: a key a closed TypedDict does not declare is reported,
    as `unknown_key`, and left out, and the value still comes back.
    Anything else wrong and it does not. `TypeError` for a `shape` that is
    not a TypedDict, or holds something no parser reads.
    """
    if not is_typeddict(shape):
        msg = f"{shape!r} is not a TypedDict"
        raise TypeError(msg)
    refined, problems = refine_json(value, loc)
    if len(problems) != 0:
        return None, problems
    typed, found = _checker(shape)(refined, loc)
    readable = all(problem.kind == "unknown_key" for problem in found)
    return (cast("T", typed) if readable else None), found

typeddict_keys `cached` ¶

typeddict_keys(typeddict: type) -> TypedDictKeys

What typeddict says of an object's keys; TypeError if its annotations do not resolve.

Whether a key is required follows the spec: a Required or NotRequired qualifier says, and otherwise total of the class that declared the key does. The qualifiers are read off the annotations as evaluated, each where its class was defined, because the runtime's __required_keys__ is worked out before they are: an annotation written as a string -- as from __future__ import annotations writes every one -- hides its qualifier. For a key with no qualifier, __required_keys__ is right, and carries the total that applied.

Openness follows the spec too: closed=True closes the TypedDict, extra_items= types every other key, closed=False opens it, and one that says none of these is as its bases are -- which the runtime does not record -- or open, when no base says either.

Source code in src/zarr_metadata/_typed_json.py

@functools.cache
def typeddict_keys(typeddict: type) -> TypedDictKeys:
    """What `typeddict` says of an object's keys; `TypeError` if its annotations do not resolve.

    Whether a key is required follows the spec: a `Required` or
    `NotRequired` qualifier says, and otherwise `total` of the class that
    declared the key does. The qualifiers are read off the annotations as
    evaluated, each where its class was defined, because the runtime's
    `__required_keys__` is worked out before they are: an annotation
    written as a string -- as `from __future__ import annotations` writes
    every one -- hides its qualifier. For a key with no qualifier,
    `__required_keys__` is right, and carries the `total` that applied.

    Openness follows the spec too: `closed=True` closes the TypedDict,
    `extra_items=` types every other key, `closed=False` opens it, and one
    that says none of these is as its bases are -- which the runtime does
    not record -- or open, when no base says either.
    """
    hints = dict(_hints(typeddict))
    required_at_runtime = cast("frozenset[str]", getattr(typeddict, "__required_keys__", ()))
    optional_at_runtime = cast("frozenset[str]", getattr(typeddict, "__optional_keys__", ()))
    if frozenset(hints) != required_at_runtime | optional_at_runtime:
        msg = (
            f"{typeddict.__name__}: its annotations declare {sorted(hints)!r}, and the runtime "
            f"{sorted(required_at_runtime | optional_at_runtime)!r}"
        )
        raise TypeError(msg)
    members: dict[str, tuple[object, bool]] = {}
    for key, hint in hints.items():
        said = qualifiers(hint)
        if "Required" in said and "NotRequired" in said:
            msg = f"{typeddict.__name__}.{key}: Required and NotRequired both; the spec allows one"
            raise TypeError(msg)
        if "Required" in said:
            required = True
        elif "NotRequired" in said:
            required = False
        else:
            required = key in required_at_runtime
        members[key] = (strip_annotation(hint)[0], required)
    extra_items, declared = _openness(typeddict)
    return TypedDictKeys(types.MappingProxyType(members), extra_items, declared)

zarr_metadata.typed_json