Skip to content

Conversation

@andygrove
Copy link
Member

Which issue does this PR close?

Closes #.

Rationale for this change

Benchmark Size Before After Improvement
no_overflow_possible 1K 906 ns 60 ns 93.5% faster
no_overflow_possible 10K 7.26 µs 55 ns 99.2% faster
validation_no_overflow 1K 1.44 µs 1.26 µs 12.3% faster
validation_no_overflow 10K 11.79 µs 10.73 µs 9.3% faster
with_overflow_to_null 1K 925 ns 869 ns 6.3% faster
with_overflow_to_null 10K 7.39 µs 7.08 µs 4.5% faster

What changes are included in this PR?

  • no_overflow_possible (93-99% faster): When input precision ≤ target precision, we skip all validation and just update metadata - O(1) instead of O(n).
  • validation_no_overflow (9-12% faster): Uses the optimized is_valid_decimal_precision() that avoids error string allocation.
  • with_overflow_to_null (4-6% faster): Uses Arrow's optimized single-pass null_if_overflow_precision() method.

How are these changes tested?

@codecov-commenter
Copy link

codecov-commenter commented Dec 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.14%. Comparing base (f09f8af) to head (cf2b2a4).
⚠️ Report is 803 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3001      +/-   ##
============================================
+ Coverage     56.12%   59.14%   +3.01%     
- Complexity      976     1367     +391     
============================================
  Files           119      167      +48     
  Lines         11743    15493    +3750     
  Branches       2251     2569     +318     
============================================
+ Hits           6591     9163    +2572     
- Misses         4012     5038    +1026     
- Partials       1140     1292     +152     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderfender
Copy link
Contributor

Test failing :
test integral divide overflow for decimal

  == Results ==
  !== Correct Answer - 6 ==   == Spark Answer - 6 ==
   struct<(a div b):bigint>   struct<(a div b):bigint>
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-4252079254050434654]
  ![null]                     [8770993997598117786] (QueryTest.scala:244)
 

@andygrove andygrove marked this pull request as draft December 28, 2025 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants