Skip to content

Conversation

@policyengine
Copy link

@policyengine policyengine bot commented Dec 9, 2025

Summary

Reduces CI test runtime from ~40 minutes to a much more manageable duration by using 32 epochs instead of 512 when running in test mode.

Changes

  • Added TESTING environment variable check in create_datasets.py
  • When TESTING=1, use 32 epochs instead of 512 for calibration (both constituency and local authority)
  • Updated both push.yaml and pull_request.yaml workflows to set TESTING=1

Impact

  • Test runtime reduced by ~16x on calibration steps
  • Production builds (when TESTING isn't set) still use full 512 epochs for maximum accuracy

Related

🤖 Generated with Claude Code

- Added TESTING environment variable check in create_datasets.py
- When TESTING=1, use 32 epochs instead of 512 for calibration
- Updated both push.yaml and pull_request.yaml workflows to set TESTING=1
- This reduces test runtime from ~40 minutes to a much more manageable duration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@nwoodruff-co
Copy link
Collaborator

@PolicyEngine needs a changelog entry

@policyengine
Copy link
Author

policyengine bot commented Dec 9, 2025

✅ Added changelog entry for version 1.29.3 documenting the test runtime reduction. The entry has been committed and pushed to the PR branch.

Added entry for version 1.29.3 documenting the reduction in test runtime
when using the TESTING environment variable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@nwoodruff-co
Copy link
Collaborator

@PolicyEngine nope, you messed up here. when I refer to a changelog entry, i'm talking about changelog_entry.yaml which must be on all PRs. revert your change and do that

@policyengine
Copy link
Author

policyengine bot commented Dec 9, 2025

Fixed! I reverted the incorrect CHANGELOG.md changes and created the proper changelog_entry.yaml file with the following content:

- bump: patch
  changes:
    changed:
      - Reduced test runtime by using 32 epochs instead of 512 when TESTING environment variable is set

The changes have been committed and pushed to the PR branch.

PolicyEngine-Bot and others added 2 commits December 9, 2025 10:31
Added proper changelog entry documenting the test runtime reduction when
using the TESTING environment variable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@nikhilwoodruff
Copy link
Contributor

@PolicyEngine relax the failing test conditions

@policyengine
Copy link
Author

policyengine bot commented Dec 9, 2025

✅ Relaxed the failing test conditions:

  • VAT reform test: Increased tolerance from 3.0 to 5.0 billion (was failing at 4.1B difference)
  • Vehicle ownership test: Increased tolerance from 10% to 15% (was failing at 13% difference)

These adjustments account for expected variance when running with reduced epochs (32 instead of 512) in test mode. Changes committed and pushed.

- Increase VAT reform tolerance from 3.0 to 5.0 billion
- Increase vehicle ownership tolerance from 10% to 15%

These adjustments account for expected variance when running with reduced epochs in test mode.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@nwoodruff-co nwoodruff-co merged commit c84f3d2 into main Dec 9, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants