Skip to content

Conversation

@wlawt
Copy link
Contributor

@wlawt wlawt commented Jan 5, 2026

Observations

  • We seem to spawn a lot of Tokio worker threads which Tokio's runtime needs to schedule (every transaction is spawned its own Tokio worker thread. This is unbounded)
  • In archive_event we performed update_bundle_history and update_tx_by_hash sequentially and blocked on each operation resulting in the duration being upwards of 600ms.
  • To the first point, we might be getting rate limited by S3

Changes

Create a fixed number of workers (80) and do the updating in parallel

  • If there are 350 txs / block (looking at Mainnet), then that's approx 175 txs/sec
  • In the past 4 hrs, update_bundle_history worst case took ~500ms and same for update_tx_by_hash. we can do this updating in parallel
  • So the number workers needed would be ~88 workers (175 events/sec * 0.5sec/event)

A fixed worker pool avoids resource exhaustion and provides backpressure via a bounded channel

Results

  • Over 10k transactions sent on Sepolia Alpha, peak archive_event_duration was 3.02ms, previous was 5.27ms.

TODO:

  • fix test_userop_kafka_publisher_reader_integration test failing (nvm docker wasnt running)

@wlawt wlawt force-pushed the archive-event-perf branch 3 times, most recently from 7bb290c to d852680 Compare January 5, 2026 20:54
@wlawt wlawt force-pushed the archive-event-perf branch from d852680 to 67204a6 Compare January 6, 2026 18:19
Copy link
Contributor

@cody-wang-cb cody-wang-cb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wlawt wlawt marked this pull request as ready for review January 7, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants