Puffer Scenario v0 #107

vcharraut · 2025-10-30T20:08:51Z

Data Format & Architecture Specification

I. Format Structure

1. Header

Size: 12 bytes
Fields:
- num_agents (int32)
- num_road_elements (int32)
- num_traffic_controls (int32)

2. Agent Structure

Core Fields

id (int32)
type (int32)
trajectory_length (int32)

Trajectory Arrays (Log - Ground Truth)

Size: trajectory_length
Position: log_trajectory_x, log_trajectory_y, log_trajectory_z (float32[])
Orientation: log_heading (float32[])
Dynamics: log_velocity_x, log_velocity_y (float32[])
Dimensions: log_length, log_width, log_height (float32[])
Validity: log_valid (int32[])

Route Information

routes_length (int32)
routes (int32[]) - List of lane IDs (Note: Currently supports only the first route)
goal_position (float32, float32, float32) - x, y, z
mark_as_expert (int32)
- 1: No routes (Expert/Uncontrollable)
- 0: Routes exist (Controllable)

3. RoadMapElement Structure

Core Fields

id (int32)
type (int32)
segment_length (int32)

Geometry Arrays

Size: segment_length
Coordinates: x, y, z (float32[])

Connectivity & Limits

num_entry (int32) & entry_lanes (int32[])
num_exit (int32) & exit_lanes (int32)
speed_limit (float32) - Unit: m/s

4. TrafficControlElement Structure

Core Fields

id (int32)
type (int32)
Position: x, y, z (float32, float32, float32)

State & Control

state_length (int32) and states (int32[])
num_controlled_lanes (int32) and controlled_lanes (int32[])

5. Metadata (Variable Length)

scenario_id (char[128])
map_id (int32)
dataset_name (char[64])
length (int32) - Number of timesteps
sdc_index (int32) - Ego vehicle ID
num_objects_of_interest + corresponding ID array
num_tracks_to_predict + corresponding ID array

II. Major Architectural Changes

1. New Data Structure Separation (`datatypes.h`)

Replaced a single Entity struct with three specialized structures:

Agent: Vehicles, pedestrians, cyclists.
RoadMapElement: Lanes, lines, edges.
TrafficControlElement: Traffic lights, stop signs, etc.

2. Renamed Fields: "Log" vs. "Sim"

Agent fields now clearly distinguish between:

Log trajectory: Ground truth from the dataset (e.g., log_trajectory_x).
Sim state: Current simulation state (e.g., sim_x, sim_y, sim_z).

3. Metadata Tracking

Added key fields to the Drive struct to improve scenario management:

scenario_id, map_index, dataset_name, log_length, sdc_index.
Tracking for "objects of interest" and "tracks to predict".

4. Updated Road Type System

Lane types: FREEWAY, SURFACE_STREET, BIKE_LANE
Road line types: 8 variations (e.g., broken/solid, white/yellow).
Road edge types: BOUNDARY, MEDIAN, SIDEWALK.
Removed types (from encoder): CROSSWALK, SPEED_BUMP, DRIVEWAY.

5. Traffic Control Updates

Defined types: TRAFFIC_LIGHT, STOP_SIGN, YIELD_SIGN.
Note: Maps containing traffic lights are currently skipped (not yet supported).

6. GridMap Entity Type System

GridMapEntity now uses an explicit entity_type field:

DynamicAgent
RoadMapElement
TrafficControl

7. Agent Dimensions

DynamicAgent tracks time-varying dimensions:

log_length/width/height arrays (per timestep).
sim_length/width/height (current simulation values).

8. Route Information

Added route_length and route array to DynamicAgent to enable path planning and lane following.

Current Limit: Only one route is supported per agent.

III. Routes Logic Details

Description
A list of lane IDs representing the path for an agent to follow. The path is calculated to be the closest possible match to the ground truth trajectory.

Conditions

Must be a Vehicle.
Valid at timestep decided in config (default: 0).
Valid for a minimum duration of timesteps (default: 0).
Not Offroad: The bounding box must not cross a road edge OR be >5m from a lane.

Current Constraints

Only one route per agent is supported at this time.

III. Note on nuPlan Data

The format functions correctly with nuPlan data, but the source data is noted to be "messy" (requires careful handling).

…ith memory management functions

… dimensions

…_observations functions

… bounds checks and updating trajectory handling in visualization functions

…on handling

…y and consistency

…ation

eugenevinitsky · 2025-11-11T13:49:34Z

pufferlib/ocean/drive/datatypes.h

+int is_road_lane(int type){
+    return (type >= 0 && type <= 9);
+}
+
+int is_drivable_road_lane(int type){
+    return (type == LANE_FREEWAY || type == LANE_SURFACE_STREET);
+}
+
+int is_road_line(int type){
+    return (type >= 10 && type <= 19);
+}
+
+int is_road_edge(int type){
+    return (type >= 20 && type <= 29);
+}
+


don't have a solution for this yet but this seems very error prone since it relies on someone remembering to change this if they add a new road type

yes I agree on that

and I don't know if we want to merge certain types

eugenevinitsky · 2025-11-11T13:50:10Z

pufferlib/ocean/drive/datatypes.h

+int is_road_lane(int type){
+    return (type >= 0 && type <= 9);
+}
+
+int is_drivable_road_lane(int type){
+    return (type == LANE_FREEWAY || type == LANE_SURFACE_STREET);
+}
+
+int is_road_line(int type){


a second thing is that lane and line being separated by a single char also seems error prone

l1onh3art88

Overall, I think the addition does improve readability as types improve but there is some excess bloat on the loading the binaries function and arguably unnecessary variable creations throughout. ~100 lines or so could be cut.

One potential issue in load binaries if drive.py is not adjusted to write traffic objects as is. Please check on this if traffic logic is not in yet.

l1onh3art88 · 2025-11-12T17:00:33Z

pufferlib/ocean/drive/binding.c

IDK if daphne's PR was already merged on the no active agent condition. But it would be good to also address that here because there is now additional freeing logic.

pufferlib/ocean/drive/drive.h

l1onh3art88 · 2025-11-12T17:35:35Z

pufferlib/ocean/drive/drive.h

+            return -1;
+        }
+    }
+


It looks like a good portion of the code bloat is from error handling here. Are all of these checks really necessary? I know they can be helpful b/c debugging here is tough, but it is a large addition with effectively no function for the sim.

l1onh3art88 · 2025-11-12T17:37:15Z

pufferlib/ocean/drive/drive.h

-            e->vy = 0;
-            e->vz = 0;
-            e->collided_before_goal = 0;
+            agent->sim_vx = 0.0f;


did vz get removed or forgotten here

I removed it, we don't use it and I don't think it is gonna be relevant for the next steps

l1onh3art88 · 2025-11-12T17:51:00Z

pufferlib/ocean/drive/drive.h

    }

+    // Subtract mean from road elements


Set means should be refactorable to cut lines by half by combining loops.

l1onh3art88 · 2025-11-12T17:53:49Z

pufferlib/ocean/drive/drive.h

+    agent->sim_heading = agent->log_heading[t];
+    agent->sim_vx = agent->log_velocity_x[t];
+    agent->sim_vy = agent->log_velocity_y[t];
+    agent->sim_length = agent->log_length[t];


Why are we setting length width height and valid here and resetting to 0 above?

I can be debateble, but for the data-driven approach, it might be better to have dynamic bounding boxes but it represents the perception noise better. Now because the noise comes the ego perception, it makes more sense to apply this in ghost mode (=control only sdc) rather than multi-agent

pufferlib/ocean/drive/drive.h

l1onh3art88 · 2025-11-12T19:28:15Z

pufferlib/ocean/drive/drive.h

    // Rotate to ego vehicle's frame
    float rel_goal_x = goal_x*cos_heading + goal_y*sin_heading;
    float rel_goal_y = -goal_x*sin_heading + goal_y*cos_heading;
    float distance_to_goal = relative_distance_2d(0, 0, rel_goal_x, rel_goal_y);
-    // Shrink agent size
-    env->entities[agent_idx].width *= 0.7f;


I know this was in the original GPU drive, so is this no longer relevant?

That's true, I added it back in set_start_position

l1onh3art88 · 2025-11-12T19:38:25Z

pufferlib/ocean/drive/drive.h

+    int num_dynamic_agents, num_roads, num_traffic;
+    if (fread(&num_dynamic_agents, sizeof(int), 1, file) != 1 ||
+        fread(&num_roads, sizeof(int), 1, file) != 1 ||
+        fread(&num_traffic, sizeof(int), 1, file) != 1) {


this will break the loading of any map until traffic is in the .py file no? Should this not be an env variable passed through config to check and then read the header properly. I think if fread is used it jumps to the next mem slot.

greptile-apps

Greptile Summary

Refactors unified Entity struct into three specialized types: DynamicAgent, RoadMapElement, and TrafficControlElement
Updates binary format to include route information, lane connectivity, traffic control states, and scenario metadata

Confidence Score: 1/5

This PR has critical memory management bugs that will cause leaks and potential crashes
Multiple critical memory leaks found: double allocation of static_agent_indices and expert_static_agent_indices arrays, incorrect malloc usage for entry_lanes/exit_lanes when counts are 0, and duplicate field declarations in structs
pufferlib/ocean/drive/drive.h requires immediate attention to fix memory leaks and duplicate declarations

Important Files Changed

Filename	Overview
pufferlib/ocean/drive/drive.h	Refactored entity handling with three typed arrays, but contains critical memory leaks (double allocation, incorrect malloc patterns) and duplicate field declarations

Sequence Diagram

sequenceDiagram
    participant Binary as "Binary File"
    participant LoadFunc as "load_map_binary()"
    participant Drive as "Drive struct"
    participant Memory as "Memory"

    LoadFunc->>Binary: "Open and read header"
    Binary-->>LoadFunc: "num_dynamic_agents, num_roads, num_traffic"
    LoadFunc->>Memory: "Allocate dynamic_agents array"
    LoadFunc->>Memory: "Allocate road_elements array"
    LoadFunc->>Memory: "Allocate traffic_elements array"
    
    loop For each dynamic agent
        LoadFunc->>Binary: "Read agent metadata"
        LoadFunc->>Memory: "Allocate trajectory arrays"
        LoadFunc->>Binary: "Read trajectory data"
        LoadFunc->>Binary: "Read route data"
    end
    
    loop For each road element
        LoadFunc->>Binary: "Read road metadata"
        LoadFunc->>Memory: "Allocate coordinate arrays"
        LoadFunc->>Binary: "Read coordinates"
        alt is_road_lane
            LoadFunc->>Memory: "Allocate entry/exit lane arrays"
            LoadFunc->>Binary: "Read lane connectivity"
        end
    end
    
    loop For each traffic element
        LoadFunc->>Binary: "Read traffic metadata"
        LoadFunc->>Memory: "Allocate states array"
        LoadFunc->>Binary: "Read traffic states"
    end
    
    LoadFunc->>Binary: "Read scenario metadata"
    LoadFunc->>Drive: "Populate metadata fields"
    LoadFunc->>Binary: "Close file"

Additional Comments (1)

pufferlib/ocean/drive/drive.h, line 241-258 (link)

logic: scenario_length field is declared twice in the Drive struct

_{4 files reviewed, 5 comments}

_{Edit Code Review Agent Settings | Greptile}
_{React with 👍 or 👎 to share your feedback on this new summary format}

pufferlib/ocean/drive/drive.h

…WAY)

…stency

vcharraut added 19 commits October 30, 2025 14:44

Add comments to Entity struct

e442fe3

Add DynamicAgent, RoadMapElement, and TrafficControlElement structs w…

41d5f80

…ith memory management functions

Refactor code structure for improved readability and maintainability

168138f

Fix memory management for dynamic agents and update loading function

7c14bd7

Add road and traffic control type definitions and utility functions

600f6c3

Refactor agent initialization and metrics handling in Drive struct

12c6196

Add test data

8e4c085

Enhance DynamicAgent structure with additional logging and simulation…

6433bea

… dimensions

Refactor drawing logic for dynamic agents and improve road edge checks

6577a9f

Add unnormalize_road_type function to map normalized types to road types

8f44bf4

Add simulation dimensions to agent state in set_start_position function

e6295dd

Fix condition checks for agent type in set_start_position and compute…

9434b83

…_observations functions

Improve grid map initialization and observation computation by adding…

d38827b

… bounds checks and updating trajectory handling in visualization functions

Remove goal_radius from DynamicAgent struct to streamline goal positi…

d257ffa

…on handling

Update RoadMapElement and TrafficControlElement structures for clarit…

91710e7

…y and consistency

Update traffic control type definitions for consistency

b678599

Add bounds check to skip out-of-bounds entities in grid map initializ…

96d5521

…ation

Merge branch 'main' into vcha/new_puffer_data

d770388

Delete test_data

8f9f0b4

eugenevinitsky reviewed Nov 11, 2025

View reviewed changes

l1onh3art88 reviewed Nov 12, 2025

View reviewed changes

vcharraut added 4 commits November 13, 2025 09:08

Refactor DynamicAgent structure and enhance Drive metadata handling

f5dcb8f

Add changes from main

4d290df

Merge main

c91e910

Merge remote-tracking branch 'emergelab/main' into vcha/new_puffer_data

53daa76

vcharraut marked this pull request as ready for review November 17, 2025 22:10

greptile-apps bot reviewed Nov 17, 2025

View reviewed changes

pufferlib/ocean/drive/drive.h Outdated Show resolved Hide resolved

pufferlib/ocean/drive/drive.h Outdated Show resolved Hide resolved

pufferlib/ocean/drive/drive.h Outdated Show resolved Hide resolved

pufferlib/ocean/drive/drive.h Outdated Show resolved Hide resolved

vcharraut added 2 commits November 18, 2025 11:05

Refactor road type definitions and update Drive struct for log length

ce2682f

Add temporary skip for traffic_lights

1516af3

vcharraut added 10 commits November 18, 2025 17:04

Add misc changes

dfca469

Add freeing in c_reset

ad386ea

Comment out agent dimension assignments in move_expert function

4e7a60d

Add map_index to Drive struct and update load_map_binary function

b4255c6

Merge branch 'main' into vcha/new_puffer_data

5c9e178

Format commit

fe425c1

Fix DRIVEWAY definition and add is_road function for road type checks

d43ae55

Remove unused classes from road encoder (CROSSWALK, SPEED_BUMP, DRIVE…

aebb422

…WAY)

Refactor DynamicAgent to Agent and update related functions for consi…

50cb68c

…stency

Fix one-hot encoding for road categories in Drive class

e9e83a4

daphne-cornelisse added the enhancement New feature or request label Nov 24, 2025

Puffer Scenario v0 #107

Are you sure you want to change the base?

Puffer Scenario v0 #107

Uh oh!

Conversation

vcharraut commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Data Format & Architecture Specification

I. Format Structure

1. Header

2. Agent Structure

3. RoadMapElement Structure

4. TrafficControlElement Structure

5. Metadata (Variable Length)

II. Major Architectural Changes

1. New Data Structure Separation (datatypes.h)

2. Renamed Fields: "Log" vs. "Sim"

3. Metadata Tracking

4. Updated Road Type System

5. Traffic Control Updates

6. GridMap Entity Type System

7. Agent Dimensions

8. Route Information

III. Routes Logic Details

III. Note on nuPlan Data

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vcharraut Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l1onh3art88 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Greptile Summary

Confidence Score: 1/5

Important Files Changed

Sequence Diagram

Additional Comments (1)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

vcharraut commented Oct 30, 2025 •

edited

Loading

1. New Data Structure Separation (`datatypes.h`)

vcharraut Nov 12, 2025 •

edited

Loading

greptile-apps bot left a comment •

edited

Loading