Skip to content

Conversation

@yunlishao
Copy link
Contributor

@yunlishao yunlishao commented Nov 20, 2025

Summary

Adds opt-in performance timing instrumentation to TrafficLayer and VirEnvHelper to diagnose why FIXS co-simulation runs slower than real-time as traffic scales up.

This PR does not fix the performance issue - it adds measurement capabilities. Investigation found that SUMO's traffic_step degrades from ~3.5ms to ~20ms over ~2 minutes of simulation, which is a SUMO limitation.

Related Issues / Tasks

Closes #88

See investigation findings: #88 (comment)

Type of Change

  • Bug fix
  • New feature
  • Maintenance / Refactor
  • Documentation
  • Test case / scenario update

Affected Modules / Components

  • CommonLib/PerformanceTimer.h - new opt-in performance timing utility with PERF_LOG macro
  • CommonLib/VirEnvHelper.cpp - socket timing at lines 423-428 (recv_socket), 405-452 (receive_realsim)
  • TrafficLayer/TrafficLayer/mainTrafficLayer.cpp - timing at lines 911-1384 (main_loop), 917-943 (traffic_step)
  • tests/UserScenarios/ - folder for proprietary user test cases

Code Snippets

recv_socket timing (VirEnvHelper.cpp:423-428)

PERF_TIC("recv_socket");
if (Sock_c.recvData(Sock_c.serverSock[iS], &simStateRecv, &simTimeRecv, Msg_c) < 0) {
    *errorMsg = "RealSim: Receive from traffic simulator failed";
    return ERROR_STEP_RECV_REALSIM;
}
PERF_TOC("recv_socket");

traffic_step timing (mainTrafficLayer.cpp:917-943)

PERF_TIC("traffic_step");
if (ENABLE_REALSIM && !g_shutdown.trafficSimulatorClosed) {
    try {
        Traffic_c.runOneStepSimulation();
    }
    catch (const std::exception& e) {
        // error handling...
    }
}
PERF_TOC("traffic_step");

Performance Logs

https://gist.github.com/yunlishao/f4d9dc8e9a0af653242db83ed909caaf

or download here:

Test Cases

Performance logs generated from tests/UserScenarios/issue_88_runningSlowIssue/

Environment

  • SUMO version: 1.21.0
  • IPG CarMaker version: 13.1.2

Checklist

  • Code compiles/runs as expected
  • Tests pass locally
  • Documentation is updated (if applicable)
  • Relevant issues are linked
  • Version-specific changes are noted (if any)

Additional Notes

Performance timing is opt-in via PERF_LOG macro to avoid overhead in normal builds.

Adds logic to build_sumo_executables.ps1 that:
- Patches tools/build_config/templates.py before building to fix escape issues
- Restores templates.py to upstream state after build completes
- Handles restoration in error cases

Also updates cav_casestudy submodule and adds CLAUDE.md project instructions.
Creates tests/UserScenarios/ with workflow for handling user-provided scenarios:
- Folder structure tracked via .gitkeep (exists in all branches)
- README.md documents workflow from OneDrive portal to debugging
- All scenario contents gitignored (maintains privacy)
- Enables full IDE/debugging access without committing proprietary files

This establishes a standardized process for receiving user scenarios via OneDrive file request, creating issue-specific branches, debugging locally, and cleaning up after resolution.

Addresses Task 1 of #80
Adds intelligent config file discovery to streamline debugging without manual
VS project settings changes:

**Auto-discovery logic:**
1. Check TrafficLayer/.active_config for explicit path
2. Auto-discover single YAML in tests/UserScenarios/
3. Error with guidance if multiple YAMLs or none found
4. Always allows -f override

**Implementation:**
- Cross-platform file discovery (WIN32 and POSIX)
- C++11 compatible (no std::filesystem dependency for discovery)
- Works when working directory is repo root (VS debugging scenario)
- Deployed executables still require explicit -f flag

**Changes:**
- Add findYamlFiles() with platform-specific implementations
- Remove hardcoded LocalDebuggerCommandArguments from .vcxproj.user
- Update UserScenarios/README.md with new workflow
- Gitignore TrafficLayer/.active_config (per-worktree)

**Usage:**
- Single scenario: Just F5 in VS
- Multiple scenarios: echo path/to/config.yaml > TrafficLayer/.active_config
- Deployment: TrafficLayer.exe -f path/to/config.yaml

Part of #80 (Task 2)
Update SENDBUFFERSIZE in RealSimSocket.cpp and buffer allocations in
RealSimPack.m to support larger message payloads. The previous 200-byte
limit was insufficient for messages with all fields enabled and
maximum-length string values (calculated max: 444 bytes).
Added PerformanceTimer.h integration to measure execution time of key operations in runStep():
- receive_realsim: Data reception from traffic simulator
- map_ids: Vehicle ID mapping
- cleanup_vehicles: Vehicle removal
- update_traffic_state: Traffic state updates
- sync_signals: Traffic signal synchronization
- send_ego: Ego vehicle data transmission
- refresh_visualization: Visualization updates

Timing disabled by default, enable with ENABLE_PERF_TIMING define.
- Remove unconditional ENABLE_PERF_TIMING define from header
- Each .cpp file must now define flag before including header
- Add PERF_LOG for printf-style diagnostic messages
- Truncate log file on init instead of append
- Add PERF_LOG calls for receive operations
- Log socket buffer status and message counts
- Wrap diagnostics under ENABLE_PERF_TIMING flag
- Add timing for main loop, traffic step, send operations
- Log vehicle counts and send buffer status
- Wrap all timing under ENABLE_PERF_TIMING flag
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] FIXS running slower than real time in co-simulation btw IPG and SUMO

3 participants