Skip to content

Conversation

@ivanthewebber
Copy link

@ivanthewebber ivanthewebber commented Dec 13, 2025

Please take the work from here, but these edits should add async support and unlock the fixture scope which will make adoption easier for companies like mine that are deeply entrenched in class scopes.

Chore that needs to be done:

  • Add newsfragment pipenv run towncrier create [issue_number].[type].rst

Types are defined in the pyproject.toml, issue_number either from issue tracker or the Pull request number

Summary by CodeRabbit

  • New Features

    • Async PostgreSQL fixtures and async database utilities added to support asyncio-based tests.
    • Configurable fixture scopes (session, package, module, class, function) for greater test flexibility.
    • Optional dependency group added for async testing extras.
  • Tests

    • Extensive async test coverage added across unit and Docker integration tests.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 13, 2025

Warning

Rate limit exceeded

@ivanthewebber has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 16 minutes and 18 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between f3546d7 and d436969.

📒 Files selected for processing (3)
  • pytest_postgresql/factories/__init__.py (1 hunks)
  • pytest_postgresql/janitor.py (5 hunks)
  • pytest_postgresql/types.py (1 hunks)

Walkthrough

Adds comprehensive asynchronous support: async fixtures, AsyncDatabaseJanitor, async loader and retry utilities, fixture-scope configurability, optional async dependencies, and corresponding async test coverage across the test suite.

Changes

Cohort / File(s) Summary
Dependency configuration
pyproject.toml
Adds [project.optional-dependencies] group async = ["pytest-asyncio", "aiofiles"].
Factory entrypoint
pytest_postgresql/factories/__init__.py
Imports postgresql_async from pytest_postgresql.factories.client (not added to __all__).
Factory implementations
pytest_postgresql/factories/client.py
Adds postgresql_async async fixture factory; adds scope: FixtureScopeT parameter to postgresql factory (propagated to pytest fixture decorator); updates imports for async types.
Process / No-process factories
pytest_postgresql/factories/process.py, pytest_postgresql/factories/noprocess.py
Add scope: FixtureScopeT = "session" parameter and pass scope into @pytest.fixture(scope=...); import FixtureScopeT.
Async janitor
pytest_postgresql/janitor.py
Adds AsyncDatabaseJanitor with async lifecycle (init, drop, load), async cursor context manager and helpers; replaces an assertion with ValueError for missing dbname/template_dbname; exposes async loader/retry helpers.
Loader (sync + async)
pytest_postgresql/loader.py
Replaces __import__ with importlib.import_module; adds build_loader_async and sql_async for async SQL loading; maintains sync loader APIs.
Retry utility
pytest_postgresql/retry.py
Adds retry_async to retry awaitable callables with timeout and asyncio.sleep semantics.
Types
pytest_postgresql/types.py
Adds FixtureScopeT = Literal["session","package","module","class","function"].
Test fixtures
tests/conftest.py
Adds async fixtures postgresql2_async and postgresql_load_1_async using postgresql_async.
Async tests — janitor & loader
tests/test_janitor.py, tests/test_loader.py
Adds async test variants exercising AsyncDatabaseJanitor, build_loader_async and sql_async.
Async tests — postgres & template
tests/test_postgresql.py, tests/test_template_database.py, tests/docker/test_noproc_docker.py
Adds multiple async tests and fixtures using AsyncConnection, retry_async, and postgresql_async counterparts mirroring existing sync tests.

Sequence Diagram(s)

sequenceDiagram
    actor Test
    participant Fixture as postgresql_async
    participant Janitor as AsyncDatabaseJanitor
    participant DB as PostgreSQL (AsyncConnection)

    Test->>Fixture: request fixture (scope)
    activate Fixture
    Fixture->>Janitor: instantiate Janitor
    activate Janitor
    Janitor->>DB: connect (AsyncConnection)
    activate DB
    Janitor->>Janitor: init() / create DB
    Janitor->>Janitor: load() via build_loader_async / sql_async
    Fixture->>Test: yield AsyncConnection
    deactivate DB
    deactivate Janitor

    Test->>Fixture: test completes
    activate Fixture
    Fixture->>Janitor: trigger drop()/cleanup
    activate Janitor
    Janitor->>DB: close connection
    deactivate Janitor
    deactivate Fixture
Loading
sequenceDiagram
    participant Caller as test/code
    participant Loader as build_loader_async
    participant SQL as sql_async
    participant Retry as retry_async
    participant DB as AsyncConnection

    Caller->>Loader: build_loader_async(path | "module.attr" | callable)
    alt Path input
        Loader-->>Caller: partial(sql_async, path)
    else Str import path
        Loader->>Loader: importlib.import_module(...)
        Loader-->>Caller: resolved callable
    else Callable
        Loader-->>Caller: passthrough callable
    end

    Caller->>Retry: retry_async(loader_func, timeout)
    activate Retry
    loop until success or timeout
        Retry->>Loader: await loader_func()
        alt sql_async used
            Loader->>SQL: execute async SQL file
            SQL->>DB: run statements
            DB-->>SQL: results
        else callable
            Loader-->>Retry: result / raise
        end
        alt exception
            Retry->>Retry: await asyncio.sleep(1)
        else success
            Retry-->>Caller: return result
        end
    end
    deactivate Retry
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • Areas needing careful review:
    • pytest_postgresql/janitor.py — async context managers, awaiting points, parity with sync janitor and exception handling.
    • pytest_postgresql/factories/client.py — fixture lifecycle for async fixtures and cleanup sequencing.
    • pytest_postgresql/loader.py & pytest_postgresql/retry.py — dynamic import resolution and async retry behaviour.
    • Tests — correct use of pytest-asyncio fixtures/markers and async mocks/patches.

"I hop and I code beneath moonlight's glow,
Async seeds I plant where tests may grow.
Connections hum, then sleep, then wake anew,
Tables dance, then vanish — tidy through and through.
A rabbit cheers, soft paws on keys, hooray! 🐇"

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Async parity and unlock scope' accurately reflects the main objectives of the PR: adding async support and making fixture scope configurable.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 14

🧹 Nitpick comments (8)
pyproject.toml (1)

39-42: Consider adding a compatible lower bound for pytest-asyncio, and ensure CI installs the extra.

Right now async = ["pytest-asyncio"] leaves version selection unconstrained; it’s safer to set a minimum known to work with your supported pytest/Python versions, and to ensure the test environment installs .[async] (or equivalent dev deps).

pytest_postgresql/factories/__init__.py (1)

20-24: Decide whether postgresql_async is public; if yes, add it to __all__.

If users are meant to consume pytest_postgresql.factories.postgresql_async, exporting it in __all__ keeps API parity with postgresql, postgresql_proc, etc.

pytest_postgresql/retry.py (1)

40-66: retry_async looks correct; consider monotonic time for timeout robustness.

Using datetime for timeouts can misbehave if system time changes; time.monotonic() (or asyncio.get_running_loop().time()) avoids that.

pytest_postgresql/factories/noprocess.py (1)

25-26: Avoid _pytest.scope._ScopeName (private pytest API) for public typing.

Consider replacing the import with a local typing.Literal["function", "class", "module", "package", "session"] alias to reduce coupling to pytest internals.

Also applies to: 49-50, 60-65

tests/docker/test_noproc_docker.py (1)

61-73: Add type annotation for consistency.

The async_postgres_with_template parameter lacks a type annotation, unlike other async test functions in this file.

 @pytest.mark.asyncio
 @pytest.mark.parametrize("_", range(5))
-async def test_template_database_async(async_postgres_with_template, _: int) -> None:
+async def test_template_database_async(async_postgres_with_template: AsyncConnection, _: int) -> None:
     """Async check that the database structure gets recreated out of a template."""
pytest_postgresql/janitor.py (3)

214-226: Async init/drop parity looks good; consider using identifiers instead of f-strings. The logic mirrors the sync janitor well, but interpolating database names via f-strings is brittle (quoting edge-cases) and can become injection-prone if names ever come from user config. Consider psycopg’s SQL composition (psycopg.sql.Identifier) for database identifiers.

Also applies to: 231-241


300-305: Add return types for __aenter__/__aexit__ (API polish). This improves editor support and keeps parity with the typed sync context manager.

@@
-    async def __aenter__(self):
+    async def __aenter__(self) -> "AsyncDatabaseJanitor":
@@
-    async def __aexit__(self, exc_type, exc_val, exc_tb):
+    async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:

255-276: Rename cor to result for clarity. The variable name cor suggests a coroutine, but build_loader_async() can return callables that execute synchronously and return None (when a sync function is provided) or return an awaitable (when an async function or partial(sql_async, ...) is provided). The current await logic correctly handles both cases with inspect.isawaitable(), but renaming to result better reflects that the value could be any return type, not necessarily a coroutine.

@@
-        cor = _loader(
+        result = _loader(
@@
-        if inspect.isawaitable(cor):
-            await cor
+        if inspect.isawaitable(result):
+            await result
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 24049e0 and de78d6c.

📒 Files selected for processing (14)
  • pyproject.toml (1 hunks)
  • pytest_postgresql/factories/__init__.py (1 hunks)
  • pytest_postgresql/factories/client.py (3 hunks)
  • pytest_postgresql/factories/noprocess.py (3 hunks)
  • pytest_postgresql/factories/process.py (3 hunks)
  • pytest_postgresql/janitor.py (3 hunks)
  • pytest_postgresql/loader.py (3 hunks)
  • pytest_postgresql/retry.py (2 hunks)
  • tests/conftest.py (1 hunks)
  • tests/docker/test_noproc_docker.py (4 hunks)
  • tests/test_janitor.py (5 hunks)
  • tests/test_loader.py (2 hunks)
  • tests/test_postgresql.py (2 hunks)
  • tests/test_template_database.py (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (7)
tests/conftest.py (1)
pytest_postgresql/factories/client.py (1)
  • postgresql_async (93-159)
tests/test_postgresql.py (3)
pytest_postgresql/factories/client.py (1)
  • postgresql_async (93-159)
pytest_postgresql/janitor.py (2)
  • cursor (131-152)
  • cursor (278-298)
pytest_postgresql/retry.py (1)
  • retry (12-37)
tests/docker/test_noproc_docker.py (1)
pytest_postgresql/factories/client.py (2)
  • postgresql_async (93-159)
  • postgresql (34-90)
pytest_postgresql/factories/client.py (2)
pytest_postgresql/janitor.py (8)
  • AsyncDatabaseJanitor (169-305)
  • DatabaseJanitor (22-166)
  • drop (84-95)
  • drop (231-240)
  • connect (134-141)
  • connect (281-288)
  • load (110-128)
  • load (255-275)
pytest_postgresql/config.py (1)
  • get_config (28-52)
pytest_postgresql/loader.py (1)
pytest_postgresql/janitor.py (2)
  • load (110-128)
  • load (255-275)
tests/test_loader.py (2)
pytest_postgresql/loader.py (4)
  • build_loader (12-24)
  • build_loader_async (36-48)
  • sql (27-33)
  • sql_async (51-57)
tests/loader.py (1)
  • load_database (9-18)
tests/test_template_database.py (2)
tests/docker/test_noproc_docker.py (1)
  • test_template_database (49-58)
pytest_postgresql/janitor.py (2)
  • cursor (131-152)
  • cursor (278-298)
🪛 Ruff (0.14.8)
tests/test_janitor.py

74-74: Possible hardcoded password assigned to argument: "password"

(S106)


78-78: Possible hardcoded password assigned to argument: "password"

(S106)

pytest_postgresql/retry.py

61-61: Consider moving this statement to an else block

(TRY300)


64-64: Avoid specifying long messages outside the exception class

(TRY003)

pytest_postgresql/factories/client.py

153-153: Undefined name pg_load

(F821)

tests/test_loader.py

23-23: Unused function argument: args

(ARG001)


23-23: Unused function argument: kwargs

(ARG001)

tests/test_template_database.py

36-36: Redefinition of unused test_template_database from line 23

(F811)


36-36: Unused function argument: async_postgresql_template

(ARG001)

🔇 Additional comments (15)
tests/test_template_database.py (1)

4-4: AsyncConnection import is fine for psycopg v3 async coverage.

tests/test_loader.py (2)

5-8: Async loader test imports are straightforward.


37-43: Async SQL loader mapping coverage is good.

tests/docker/test_noproc_docker.py (3)

17-17: LGTM!

The async fixture is correctly defined using postgresql_async factory, mirroring the synchronous counterpart.


25-27: LGTM!

Async fixture for template database correctly configured with dbname parameter.


40-46: LGTM!

Async test correctly uses async with for cursor context management and awaits all async operations.

tests/test_janitor.py (4)

22-27: LGTM!

Async version cast test correctly mirrors the synchronous test and validates AsyncDatabaseJanitor version handling.


38-44: LGTM!

Async cursor test properly mocks AsyncConnection.connect and validates the expected database connection parameters.


64-79: LGTM!

Async password connection test correctly validates that password is passed through to the connection. The hardcoded password is appropriate for test fixtures.


103-119: LGTM!

Async populate test correctly validates the loader mechanism with both dot and colon separator formats for import paths.

tests/test_postgresql.py (4)

77-82: LGTM!

Async main postgres test correctly uses async context manager and awaits cursor operations.


85-94: LGTM!

Async two postgres test correctly validates multiple async connections in a single test.


97-103: LGTM!

Async load test correctly verifies fixture loading with async cursor operations.


106-109: LGTM!

Async port test correctly validates connection status.

pytest_postgresql/factories/client.py (1)

34-50: LGTM!

The scope parameter addition to the synchronous fixture factory is clean and correctly applies the scope to the pytest fixture decorator.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (4)
tests/test_template_database.py (1)

40-51: Missing @pytest.mark.xdist_group decorator for parallel test coordination.

The synchronous test at line 26 has @pytest.mark.xdist_group(name="template_database") to prevent parallel interference when running with pytest-xdist. The async test should have the same marker to ensure both tests don't run simultaneously and interfere with each other on the shared template database.

+@pytest.mark.xdist_group(name="template_database")
 @pytest.mark.asyncio
 @pytest.mark.parametrize("_", range(5))
 async def test_template_database_async(async_postgresql_template: AsyncConnection, _: int) -> None:
pytest_postgresql/loader.py (1)

44-44: Same incorrect importlib.import_module usage in async variant.

Apply the same fix as the synchronous version.

-        _temp_import = importlib.import_module(import_path, loader_name)
+        _temp_import = importlib.import_module(import_path)
pytest_postgresql/janitor.py (2)

40-51: Docstring parameter name for template_dbname is still incorrect in both janitors

The constructor docstrings list :param dbname: twice; the second entry in each class should describe template_dbname, not dbname. This was raised before for the async janitor and is still outstanding.

You can align both docstrings like this:

@@ class DatabaseJanitor:
-        :param dbname: database name
-        :param dbname: template database name
+        :param dbname: database name
+        :param template_dbname: template database name
@@ class AsyncDatabaseJanitor:
-        :param dbname: database name
-        :param dbname: template database name
+        :param dbname: database name
+        :param template_dbname: template database name

Also applies to: 187-197


231-241: Fix potential connection leak in AsyncDatabaseJanitor.cursor if conn.cursor() fails

As written, await conn.close() is inside the async with conn.cursor() block. If conn.cursor() or entering the async context raises, the finally block is never reached and the connection is left open.

Restructuring to always close the connection, regardless of where an error occurs, would make this safer:

-        conn = await retry_async(connect, timeout=self._connection_timeout, possible_exception=psycopg.OperationalError)
-        await conn.set_isolation_level(self.isolation_level)
-        await conn.set_autocommit(True)
-        # We must not run a transaction since we create a database.
-        async with conn.cursor() as cur:
-            try:
-                yield cur
-            finally:
-                await conn.close()
+        conn = await retry_async(
+            connect,
+            timeout=self._connection_timeout,
+            possible_exception=psycopg.OperationalError,
+        )
+        try:
+            await conn.set_isolation_level(self.isolation_level)
+            await conn.set_autocommit(True)
+            # We must not run a transaction since we create a database.
+            async with conn.cursor() as cur:
+                yield cur
+        finally:
+            await conn.close()

This follows the same intent but guarantees that the underlying async connection is closed even if cursor acquisition or context entry fails.

Also applies to: 277-305

🧹 Nitpick comments (1)
pyproject.toml (1)

39-43: Consider adding version constraints to optional dependencies.

The async optional dependencies lack version constraints, which could lead to compatibility issues with future breaking changes. Consider specifying minimum versions, similar to the core dependencies.

 [project.optional-dependencies]
 async = [
-  "pytest-asyncio",
-  "aiofiles"
+  "pytest-asyncio >= 0.21",
+  "aiofiles >= 23.0"
 ]
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between de78d6c and fb3ce5c.

📒 Files selected for processing (9)
  • pyproject.toml (1 hunks)
  • pytest_postgresql/factories/client.py (2 hunks)
  • pytest_postgresql/factories/noprocess.py (3 hunks)
  • pytest_postgresql/factories/process.py (3 hunks)
  • pytest_postgresql/janitor.py (4 hunks)
  • pytest_postgresql/loader.py (3 hunks)
  • pytest_postgresql/types.py (1 hunks)
  • tests/test_postgresql.py (2 hunks)
  • tests/test_template_database.py (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pytest_postgresql/factories/process.py
🧰 Additional context used
🧬 Code graph analysis (3)
pytest_postgresql/loader.py (1)
pytest_postgresql/janitor.py (2)
  • load (110-128)
  • load (255-275)
tests/test_postgresql.py (3)
pytest_postgresql/retry.py (2)
  • retry (12-37)
  • retry_async (40-65)
pytest_postgresql/factories/client.py (1)
  • postgresql_async (93-150)
pytest_postgresql/janitor.py (2)
  • cursor (131-152)
  • cursor (278-298)
pytest_postgresql/factories/client.py (1)
pytest_postgresql/janitor.py (6)
  • AsyncDatabaseJanitor (169-305)
  • DatabaseJanitor (22-166)
  • drop (84-95)
  • drop (231-240)
  • connect (134-141)
  • connect (281-288)
🪛 Ruff (0.14.8)
pytest_postgresql/janitor.py

57-57: Avoid specifying long messages outside the exception class

(TRY003)


204-204: Avoid specifying long messages outside the exception class

(TRY003)

🔇 Additional comments (8)
pytest_postgresql/types.py (1)

1-3: LGTM!

Clean type alias definition covering all valid pytest fixture scopes. This provides good type safety for the scope parameter across factory modules.

pytest_postgresql/factories/noprocess.py (1)

30-30: LGTM!

The scope parameter implementation is clean and maintains backward compatibility with the "session" default. The type annotation using FixtureScopeT provides proper type safety.

Also applies to: 49-49, 64-64

pytest_postgresql/loader.py (1)

51-59: Async SQL loader implementation looks correct.

The aiofiles import is appropriately placed inside the function to avoid import errors when the async optional dependency isn't installed. The cursor handling and awaits are correct for psycopg 3's AsyncConnection.

tests/test_postgresql.py (1)

77-109: Async test implementations look correct.

All async tests properly use @pytest.mark.asyncio, correctly await async operations, and maintain parity with their synchronous counterparts.

pytest_postgresql/factories/client.py (2)

34-51: Configurable scope for the synchronous PostgreSQL fixture looks sound

Threading the scope argument through to @pytest.fixture while keeping the default "function" preserves existing behaviour and lets users opt into class/session-level fixtures without extra boilerplate. No functional issues spotted in the way the scope type and default are wired here.


93-150: Async PostgreSQL fixture factory is correctly structured and mirrors the sync path

The async factory correctly uses pytest_asyncio.fixture, AsyncDatabaseJanitor as an async context manager, and AsyncConnection.connect(...) with await, and ensures the connection is closed with await db_connection.close(). This gives good parity with the synchronous postgresql factory while keeping pytest_asyncio an optional import.

pytest_postgresql/janitor.py (2)

172-213: AsyncDatabaseJanitor constructor validation and version handling look correct

The async constructor mirrors the synchronous one, validates that at least one of dbname or template_dbname is provided via an explicit ValueError, and normalises version to a Version instance. This is clear and robust and avoids the earlier assert pitfall.


255-276: Async loader integration looks reasonable

Using build_loader_async and inspect.isawaitable to handle both synchronous and asynchronous loader callables is a pragmatic way to support mixed ecosystems and keeps the async surface area simple. No behavioural issues spotted here.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pytest_postgresql/janitor.py (1)

38-51: Fix duplicate parameter name in docstring.

Line 44 has :param dbname: template database name but this should be :param template_dbname: to match the actual parameter.

         :param dbname: database name
-        :param dbname: template database name
+        :param template_dbname: template database name
         :param version: postgresql version number
♻️ Duplicate comments (1)
pytest_postgresql/janitor.py (1)

185-198: Fix duplicate parameter name in docstring.

Line 191 has the same issue as the sync version - :param dbname: appears twice instead of :param template_dbname:.

         :param dbname: database name
-        :param dbname: template database name
+        :param template_dbname: template database name
         :param version: postgresql version number
🧹 Nitpick comments (4)
pytest_postgresql/janitor.py (3)

84-95: Consider replacing assert with explicit ValueError for consistency.

Line 89 uses assert db_to_drop which can be stripped with python -O. For consistency with the __init__ validation fix, consider using an explicit check.

     def drop(self) -> None:
         """Drop database in postgresql."""
         db_to_drop = self.template_dbname if self.is_template() else self.dbname
-        assert db_to_drop
+        if not db_to_drop:
+            raise ValueError("No database to drop; either dbname or template_dbname must be set.")
         with self.cursor() as cur:

231-240: Consider replacing assert with explicit ValueError for consistency.

Line 234 uses assert db_to_drop which has the same issue as the sync version.

     async def drop(self) -> None:
         """Drop database in postgresql (async)."""
         db_to_drop = self.template_dbname if self.is_template() else self.dbname
-        assert db_to_drop
+        if not db_to_drop:
+            raise ValueError("No database to drop; either dbname or template_dbname must be set.")
         async with self.cursor() as cur:

300-305: Consider adding type hints to async context manager methods.

For consistency with the sync DatabaseJanitor and better IDE support, consider adding type hints:

-    async def __aenter__(self):
+    async def __aenter__(self) -> "AsyncDatabaseJanitor":
         await self.init()
         return self

-    async def __aexit__(self, exc_type, exc_val, exc_tb):
+    async def __aexit__(
+        self,
+        exc_type: Type[BaseException] | None,
+        exc_val: BaseException | None,
+        exc_tb: TracebackType | None,
+    ) -> None:
         await self.drop()
pytest_postgresql/loader.py (1)

51-59: Consider documenting the optional aiofiles dependency in the function docstring or adding error handling.

Although aiofiles is declared as an optional dependency in pyproject.toml, the sql_async function lacks documentation about this requirement. If users call this function without installing the optional dependency, they will encounter a bare ImportError rather than a clear message explaining that the async extra is required. Consider either adding a note to the docstring or wrapping the import with a try/except block that provides a more informative error message.

The implementation is otherwise correct: AsyncConnection.cursor() is correctly not awaited (synchronous factory in psycopg3), whilst I/O methods are properly awaited.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fb3ce5c and f3546d7.

📒 Files selected for processing (5)
  • pytest_postgresql/janitor.py (4 hunks)
  • pytest_postgresql/loader.py (3 hunks)
  • tests/test_loader.py (2 hunks)
  • tests/test_postgresql.py (2 hunks)
  • tests/test_template_database.py (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
tests/test_postgresql.py (3)
pytest_postgresql/retry.py (2)
  • retry (12-37)
  • retry_async (40-65)
pytest_postgresql/factories/client.py (1)
  • postgresql_async (93-150)
pytest_postgresql/janitor.py (2)
  • cursor (131-152)
  • cursor (278-298)
pytest_postgresql/janitor.py (3)
pytest_postgresql/loader.py (2)
  • build_loader (12-24)
  • build_loader_async (36-48)
pytest_postgresql/retry.py (2)
  • retry (12-37)
  • retry_async (40-65)
pytest_postgresql/executor.py (2)
  • template_dbname (135-137)
  • version (201-213)
pytest_postgresql/loader.py (1)
pytest_postgresql/janitor.py (6)
  • load (110-128)
  • load (255-275)
  • connect (134-141)
  • connect (281-288)
  • cursor (131-152)
  • cursor (278-298)
🪛 Ruff (0.14.8)
pytest_postgresql/janitor.py

57-57: Avoid specifying long messages outside the exception class

(TRY003)


204-204: Avoid specifying long messages outside the exception class

(TRY003)

🔇 Additional comments (9)
tests/test_loader.py (2)

17-27: LGTM!

The async test correctly verifies build_loader_async handles callable references, string import paths, and inline async functions. The unused parameter fix (*_args, **_kwargs) addresses the previous review comment.


37-43: LGTM!

The async SQL loader test correctly verifies that build_loader_async returns a partial wrapping sql_async for Path inputs, mirroring the synchronous test structure.

tests/test_template_database.py (2)

20-23: LGTM!

The async fixture is correctly defined using postgresql_async factory, pointing to the same process fixture with the templated database name.


40-52: LGTM!

The async test addresses all previously identified issues: unique function name, proper async def, correct @pytest.mark.asyncio decorator, correct fixture parameter, and all necessary await calls. The separate xdist_group name prevents parallel interference with the sync test.

pytest_postgresql/loader.py (1)

36-48: LGTM!

The async loader builder correctly mirrors the synchronous implementation, properly using importlib.import_module and returning partials for Path inputs or pass-through for callables.

tests/test_postgresql.py (2)

77-94: LGTM!

The async tests correctly mirror their synchronous counterparts, properly using async with for cursor context management and await for all I/O operations.


112-130: LGTM!

The async connection termination test now includes the @pytest.mark.skipif decorator for PostgreSQL version checking (addressing the previous review comment), correctly uses retry_async instead of retry, and properly defines the inner check function as async def.

pytest_postgresql/janitor.py (2)

56-57: LGTM!

The validation now correctly uses ValueError instead of assert, ensuring it cannot be stripped by Python's optimisation flags (-O).


277-298: LGTM!

The async cursor context manager correctly:

  • Uses retry_async for connection retry logic
  • Uses async setters (await conn.set_isolation_level, await conn.set_autocommit) per psycopg3 API
  • Wraps cursor usage in try/finally to ensure connection closure even if cursor creation fails
  • Does not await conn.cursor() (correct - it's a synchronous factory in psycopg3)

@ivanthewebber
Copy link
Author

@fizyk could you update fizyk/actions-reuse/.github/actions/pipenv-setup@v4.1.1 if needed to install the optional async dependencies for testing and give this final review?

Copy link
Member

@fizyk fizyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivanthewebber try this branch for setup-pipenv pipenv-categories

I'll release it once we'll get it running here. Haven't needed it anywhere else so far.

unixsocketdir: str | None = None,
postgres_options: str | None = None,
load: list[Callable | str | Path] | None = None,
scope: FixtureScopeT = "session",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you want to add the scope parameter to all the factories there are? What's the use case?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For codebases that already have thousands of tests switching everything to a different scope to test out this library to see if it speeds up tests might be rather prohibitory. We have 6000 tests and over a hundred fixtures e.g. and I used this to get things running and next I can work on side effects of changing scope. Could have a warning or something if it's somehow critical

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see now; yes it's probably not necessary for proc and noproc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have 6000 tests
😍 love it!

So, the original intention behind those scopes would be that:

process sets up the database instance (with later additon of template database)
And client fixture creates a one-off database and drops it after the test (with later addition of doing that based on the template). This makes sure the state does not leak between tests, and they do not influence each other or make the order of tests dependant.

I've tested it manually, and the speed is rather similar if I had created a single database and used transactions. However, then all your tests are inside transactions, and you have to use that single connection that set up the transaction (that's why warehouse is using just DatabaseJanitor in their tests).

With the template database approach, you can create your own client connection to read the state as you prepare it, and not worry about hanging transactions with data fixtures.
Even run external processes in tests, if you wish to do so.

To address somethig like you described, I planned #890, although got distracted along the way if thinking about it. Maybe it's time to return to that idea....

pg_password = proc_fixture.password
pg_options = proc_fixture.options
pg_db = dbname or proc_fixture.dbname
janitor = AsyncDatabaseJanitor(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need AsyncDatabaseJanitor? Is there something in Janitor that would require tests to block on I/O?

This might come in handy in case test would have more async fixtures to prepare, not in a straightforward scenario with a single fixture?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your assessment is accurate; you can get around without it for sure unless you're using it directly in an async test or fixture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants