From 885b4fe5ddc06a808618d1f5cfec053e0f96d527 Mon Sep 17 00:00:00 2001 From: James Prior Date: Sat, 6 Sep 2025 08:58:47 +0100 Subject: [PATCH 1/2] Update docs and rename `STRICT_ENV` --- CHANGELOG.md | 30 ++++++++++++++++++------------ docs/syntax.md | 31 +++++++++++++++++++++++++++++-- jsonpath/__init__.py | 16 ++++++++-------- 3 files changed, 55 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3bc8488..67ecf36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,27 +4,33 @@ **JSONPath syntax changes** -These breaking changes apply to Python JSONPath in its default configuration. We've also introduced a _strict mode_ where we follow the RFC 9535 specification exactly. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies) and the [syntax guide](https://jg-rp.github.io/python-jsonpath/syntax/) for more information. - -- Using bracket notation, unquoted property names are no longer interpreted as quoted property names. These paths used to be equivalent, `$[foo]`, `$['foo']` and `$["foo"]`. Now, names without quotes start a _singular query selector_. With an implicit _root identifier_, `$.a[b]` is equivalent to `$.a[$.b]`. See [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector) in the syntax guide. -- In filter selector expressions, float literals now follow the specification. Previously `.1` and `1.` where allowed, now it must be `0.1` and `1.0`, with at least one digit either side of the decimal point. -- Slice selector indexes and step now follow the specification. Previously leading zeros and negative zero were allowed, now they raise a `JSONPathSyntaxError`. -- Whitespace is no longer allowed between a dot (`.` or `..`) and a name when using shorthand notation for the name selector. Whitespace before the dot oor double dot is OK. +These breaking changes affect the **default configuration** of Python JSONPath. +Version 2 also introduces a new _strict mode_, which enforces full compliance with [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535). See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies) and the [syntax guide](https://jg-rp.github.io/python-jsonpath/syntax/) for details. + +- **Bracket notation** - unquoted property names are no longer treated as quoted names. + - Before: `$[foo]`, `$['foo']`, and `$["foo"]` were equivalent. + - Now: `$[foo]` is a _singular query selector_. With an implicit root identifier, `$.a[b]` is equivalent to `$.a[$.b]`. See [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector). +- **Filter expressions** - float literals must follow the RFC. + - `.1` is now invalid (use `0.1`) + - `1.` is now invalid (use `1.0`) +- **Slice selectors** - indexes and steps must follow the RFC. + - Leading zeros and negative zero are no longer valid and raise `JSONPathSyntaxError`. +- **Dot notation** - no whitespace is allowed between `.` or `..` and the following name. Whitespace before the dot is still permitted. **JSONPath function extension changes** -- Added the `startswith(value, prefix)` function extension. `startswith` returns `True` if both arguments are strings and the second argument is a prefix of the first argument. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#startswith) documentation. -- The non-standard `keys()` function extension has been reimplemented. It used to be a simple Python function, `jsonpath.function_extensions.keys`. Now it is a "well-typed" class, `jsonpath.function_extensions.Keys`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#keys) documentation. +- Added the `startswith(value, prefix)` function extension. Returns `True` if both arguments are strings and `prefix` is a prefix of `value`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#startswith) documentation. +- Reimplemented the non-standard `keys()` function extension. It used to be a simple Python function, `jsonpath.function_extensions.keys`. Now it is a "well-typed" class, `jsonpath.function_extensions.Keys`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#keys) documentation. - Added `cache_capacity`, `debug` and `thread_safe` arguments to `jsonpath.function_extensions.Match` and `jsonpath.function_extensions.Search` constructors. **JSONPath features** - Added the [Keys filter selector](https://jg-rp.github.io/python-jsonpath/syntax/#keys-filter-selector). - Added the [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector). -- We now use the [regex] package, if available, instead of `re` for match and search function extensions. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies). -- Added the `strict` argument to all [convenience functions](https://jg-rp.github.io/python-jsonpath/convenience/), the CLI and the `JSONPathEnvironment` constructor. When `strict=True`, all extensions to RFC 9535, any non-standard function extensions and any lax parsing rules will be disabled. +- Match and search function extensions now use the [`regex`](https://pypi.org/project/regex/) package (if installed) instead of `re`. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies). +- Added the `strict` argument to all [convenience functions](https://jg-rp.github.io/python-jsonpath/convenience/), the CLI and the `JSONPathEnvironment` constructor. When `strict=True`, all non-standard extensions and relaxed parsing rules are disabled. - Added class variable `JSONPathEnvironment.max_recursion_depth` to control the maximum recursion depth of descendant segments. -- Added pretty exception messages. +- Improved exception messages (prettier, more informative). **Python API changes** @@ -32,7 +38,7 @@ These breaking changes apply to Python JSONPath in its default configuration. We **Low level API changes** -These breaking changes will only affect you if you're customizing the JSONPath lexer or parser. +These only affect projects customizing the JSONPath lexer or parser. - The tokens produced by the JSONPath lexer have changed. Previously we broadly skipped some punctuation and whitespace. Now the parser can make better choices about when to accept whitespace and do a better job of enforcing dots. - We've change the internal representation of compiled JSONPath queries. We now model segments and selectors explicitly and use terminology that matches RFC 9535. diff --git a/docs/syntax.md b/docs/syntax.md index f8381a3..50ba773 100644 --- a/docs/syntax.md +++ b/docs/syntax.md @@ -1,8 +1,35 @@ # JSONPath Syntax -Python JSONPath extends the [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) specification with additional features and relaxed rules. If you need strict compliance with RFC 9535, set `strict=True` when calling [`findall()`](convenience.md#jsonpath.findall), [`finditer()`](convenience.md#jsonpath.finditer), etc., which enforces the standard without these extensions. +Python JSONPath extends the [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) specification with extra selectors and relaxed rules for convenience. If you need strict compliance with RFC 9535, pass `strict=True` when calling [`findall()`](convenience.md#jsonpath.findall), [`finditer()`](convenience.md#jsonpath.finditer), and similar functions. In strict mode, the syntax and behavior conform to the specification, and no non-standard extensions are registered by default. You can still add them manually if needed. -In this guide, we first outline the standard syntax (see the specification for the formal definition), and then describe the non-standard extensions and their semantics in detail. +This guide first introduces the standard JSONPath syntax (see the RFC for the formal definition), then explains the non-standard extensions and their semantics. + +??? info "Preconfigured JSONPath Environments" + + Python JSONPath provides two ready-to-use environments: + + - **Default environment** – includes relaxed syntax, non-standard selectors, and additional function extensions. + - **Strict environment** – starts with only the RFC 9535 selectors and functions registered. Non-standard extensions can still be enabled explicitly. + + For custom setups, subclass [`JSONPathEnvironment`](./api.md#jsonpath.JSONPathEnvironment) and override `setup_function_extensions()`: + + ```python + from jsonpath import JSONPathEnvironment + from jsonpath.function_extensions import StartsWith + + + class MyJSONPathEnvironment(JSONPathEnvironment): + def __init__(self) -> None: + super().__init__(strict=True) + + def setup_function_extensions(self) -> None: + super().setup_function_extensions() + self.function_extensions["startswith"] = StartsWith() + + + jsonpath = MyJSONPathEnvironment() + query = jsonpath.compile("...") + ``` ## JSONPath Terminology diff --git a/jsonpath/__init__.py b/jsonpath/__init__.py index 31fd4a9..03a7ef4 100644 --- a/jsonpath/__init__.py +++ b/jsonpath/__init__.py @@ -92,7 +92,7 @@ # For convenience and to delegate to strict or non-strict environments. DEFAULT_ENV = JSONPathEnvironment() -STRICT_ENV = JSONPathEnvironment(strict=True) +_STRICT_ENV = JSONPathEnvironment(strict=True) def compile(path: str, *, strict: bool = False) -> Union[JSONPath, CompoundJSONPath]: # noqa: A001 @@ -112,7 +112,7 @@ def compile(path: str, *, strict: bool = False) -> Union[JSONPath, CompoundJSONP JSONPathTypeError: If filter functions are given arguments of an unacceptable type. """ - return STRICT_ENV.compile(path) if strict else DEFAULT_ENV.compile(path) + return _STRICT_ENV.compile(path) if strict else DEFAULT_ENV.compile(path) def findall( @@ -146,7 +146,7 @@ def findall( an incompatible way. """ return ( - STRICT_ENV.findall(path, data, filter_context=filter_context) + _STRICT_ENV.findall(path, data, filter_context=filter_context) if strict else DEFAULT_ENV.findall(path, data, filter_context=filter_context) ) @@ -183,7 +183,7 @@ async def findall_async( an incompatible way. """ return ( - await STRICT_ENV.findall_async(path, data, filter_context=filter_context) + await _STRICT_ENV.findall_async(path, data, filter_context=filter_context) if strict else await DEFAULT_ENV.findall_async(path, data, filter_context=filter_context) ) @@ -219,7 +219,7 @@ def finditer( an incompatible way. """ return ( - STRICT_ENV.finditer(path, data, filter_context=filter_context) + _STRICT_ENV.finditer(path, data, filter_context=filter_context) if strict else DEFAULT_ENV.finditer(path, data, filter_context=filter_context) ) @@ -256,7 +256,7 @@ async def finditer_async( an incompatible way. """ return ( - await STRICT_ENV.finditer_async(path, data, filter_context=filter_context) + await _STRICT_ENV.finditer_async(path, data, filter_context=filter_context) if strict else await DEFAULT_ENV.finditer_async(path, data, filter_context=filter_context) ) @@ -292,7 +292,7 @@ def match( an incompatible way. """ return ( - STRICT_ENV.match(path, data, filter_context=filter_context) + _STRICT_ENV.match(path, data, filter_context=filter_context) if strict else DEFAULT_ENV.match(path, data, filter_context=filter_context) ) @@ -359,7 +359,7 @@ def query( an incompatible way. """ return ( - STRICT_ENV.query(path, data, filter_context=filter_context) + _STRICT_ENV.query(path, data, filter_context=filter_context) if strict else DEFAULT_ENV.query(path, data, filter_context=filter_context) ) From e419de2354996e9145f9d2722e140b1bd9fc508a Mon Sep 17 00:00:00 2001 From: James Prior Date: Tue, 9 Sep 2025 13:55:53 +0100 Subject: [PATCH 2/2] Update optional dependencies docs --- docs/index.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/docs/index.md b/docs/index.md index 60c5421..abf4179 100644 --- a/docs/index.md +++ b/docs/index.md @@ -34,11 +34,20 @@ conda install -c conda-forge python-jsonpath ### Optional dependencies -By default, and without any additional dependencies, the syntax supported by Python JSONPath is **very close** to RFC 9535. For strict compatibility with the specification, install [regex](https://pypi.org/project/regex/) and [iregexp-check](https://pypi.org/project/iregexp-check/) packages too. +Python JSONPath works out of the box with **no extra dependencies**, and its syntax is already **very close** to [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535). -With these two packages installed, the [`match()`](functions.md#match) and [`search()`](functions.md#search) filter functions will use [regex](https://pypi.org/project/regex/) instead of `re` from the standard library, and will validate regular expression patterns against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485). +For strict compliance with the specification, [strict mode](syntax.md) and the `strict` extra were added in **version 2.0.0**. -See the [syntax guide](syntax.md) for more information about strict compatibility with RFC 9535 and extensions to the specification. +```console +pip install python-jsonpath[strict] +``` + +This installs [`regex`](https://pypi.org/project/regex/) and [`iregexp-check`](https://pypi.org/project/iregexp-check/), enabling: + +- [`match()`](functions.md#match) and [`search()`](functions.md#search) to use `regex` instead of Python's built-in `re` module. +- Validation of regular expressions against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485). + +See the [syntax guide](syntax.md) for strict mode details and specification extensions. ## Example