Integrate L0 package for CPS household calibration

[MaxGhenis]

Overview

We should integrate the new L0 regularization package into our CPS household calibration workflow to replace the current embedded implementation.

Background

The L0 regularization code currently lives directly in policyengine-us-data but has now been extracted into a standalone package at https://github.com/PolicyEngine/L0. This package implements the L0 penalty method from Louizos, Welling, & Kingma (2017) for intelligent sample selection.

Benefits of Migration

• Cleaner codebase: Removes complex L0 implementation from policyengine-us-data
• Reusability: L0 package can be used across other PolicyEngine projects
• Better testing: Dedicated test suite with 95% coverage
• Maintenance: Easier to maintain and improve L0 functionality separately

Implementation Steps

1. Wait for L0 package PR to be merged: Initial L0 regularization implementation L0#1
2. Publish L0 package to PyPI as l0
3. Add l0 to policyengine-us-data dependencies
4. Replace current L0 implementation with imports from l0 package
5. Update calibration code to use SampleGate from l0.gates
6. Test calibration workflow with new package
7. Remove old L0 code from policyengine-us-data

Current L0 Parameters in Use

Based on the existing implementation, we're using:

• temperature=0.25
• init_mean=0.999 (for high initial selection rate)
• l0_lambda=5e-07

Code Changes Preview

Replace current implementation with:

from l0.gates import SampleGate
from l0.penalties import compute_l0_penalty

# For CPS household selection
sample_gate = SampleGate(
    n_samples=len(cps_data),
    target_samples=target_households,
    temperature=0.25
)

Related Links

• L0 package repository: https://github.com/PolicyEngine/L0
• Original paper: https://arxiv.org/abs/1712.01312
• Current implementation location: [Link to current L0 code in policyengine-us-data]

cc @baogorek @MaxGhenis