Model analysis report for

▶
root: {} 12 keys
- report_name: 'analyse_lat_mlsim_no_noise'
- route_list: '/code.nfs/branches/caches/tinytower-haraschax_2026-09-14_10-41-51/xx/comma_data/lists/train_10m_20260826_test_2k.txt'
- output_dir: null
- save_tmp: false
- read_only: false
- shuffle: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 10 keys
  - priority: 3
  - job_name: 'analyse_lat_mlsim_no_noise'
  - queue_name: 'remote_v3'
  - redis_host: 'redis.comma.internal'
  - codedir: '/code.nfs/branches/caches/tinytower-haraschax_2026-09-14_10-41-51/xx'
  - use_local_codedir: false
  - ▶
    limits: {} 7 keys
    - cpu_threads: true
    - memory: true
    - small_gpu_memory: false
    - big_gpu_memory: false
    - timeout_seconds: 60
    - triton: true
    - node_whitelist: null
  - ▶
    env: {} 0 keys
  - queue_timeout: 259200
  - job_group: 'rldriving_validation_f78ed37d-afad-4dbc-8050-40ea885eedde'
- ▶
  filter: {} 3 keys
  - min_hist: 100
  - max_hist: -60
  - max_fidx: 1200
- ▶
  rollout: {} 9 keys
  - type: 'TinySimRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 35 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 5 keys
      - segment: ''
      - depth_type: 'EASYDEPTH'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2026_02'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - ▶
      physics: {} 3 keys
      - type: 'BICYCLE'
      - ▶
        bicycle_model_config: {} 1 keys
        
        vehicle_platform: 'HONDA_CIVIC'
      - ▶
        ml_model_config: {} 11 keys
        
        platform: 'TOYOTA_COROLLA_TSS2'
        
        eid: 'effd03d9-472d-44a1-910a-6d49a41eb63f/50'
        
        y_range: [-3.0, 3.0]
        
        vocab_size: 1024
        
        temperature: 0.5
        
        use_gt_features: false
        
        platform_vec_size: 256
        
        platform_map_path: '/code.nfs/branches/caches/tinytower-haraschax_2026-09-14_10-41-51/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
        
        lateral_tuning_type: 'AUTO'
        
        ▶
        lateral_tuning_params: {} 0 keys
        
        ▶
        lateral_initial_state_params: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.tinysim_env.TinySimEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: false
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_decoder_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - noise_model: 'NONE'
    - photo_noise_model: 'NONE'
    - augment_mode: 'DEPTH'
    - inpaint_mode: 'NONE'
    - initial_speed: null
    - allow_any_frame: false
    - lat_lag_override: null
    - lat_action_bias_override: null
    - max_x_error: 4.0
    - max_y_error: 3.0
    - max_yaw_error: 0.2
    - reset_every: null
    - avoid_hood: false
    - cache_granularity: true
  - ▶
    agent: {} 8 keys
    - supercombo: 'f78ed37d-afad-4dbc-8050-40ea885eedde/12864'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - use_allow_throttle: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false

Plan analysis

All segments

Metrics

passed :1537 / 1772 aka 86.73814898419865%

better :1222 / 1772 aka 68.96162528216703%

best (0.5m):650 / 1772 aka 36.681715575620764%

global bias y: +0.127

global mae y: 0.259

global mae y of lanecenter (when detected) in GT: 0.211

global mae y of lanecenter (when detected) in Sim: 0.230

avg resets: 0.264

Mean abs curvature: 0.133 deg/m

Mean abs curvature rate: 0.135 deg/ms

Mean abs lateral accel: 0.266 m/(ss)

Mean abs lateral accel GT: 0.291 m/(ss)

Mean abs lateral jerk: 0.173 m/(sss)

Mean abs lateral jerk GT: 0.249 m/(sss)

Easy straight

Metrics

passed :625 / 644 aka 97.04968944099379%

better :606 / 644 aka 94.09937888198758%

best (0.5m):421 / 644 aka 65.37267080745342%

global bias y: +0.129

global mae y: 0.197

global mae y of lanecenter (when detected) in GT: 0.173

global mae y of lanecenter (when detected) in Sim: 0.179

avg resets: 0.036

Mean abs curvature: 0.033 deg/m

Mean abs curvature rate: 0.045 deg/ms

Mean abs lateral accel: 0.088 m/(ss)

Mean abs lateral accel GT: 0.108 m/(ss)

Mean abs lateral jerk: 0.084 m/(sss)

Mean abs lateral jerk GT: 0.169 m/(sss)

Low speed

Metrics

passed :216 / 267 aka 80.89887640449437%

better :141 / 267 aka 52.80898876404494%

best (0.5m):61 / 267 aka 22.846441947565545%

global bias y: +0.119

global mae y: 0.325

global mae y of lanecenter (when detected) in GT: 0.231

global mae y of lanecenter (when detected) in Sim: 0.262

avg resets: 0.551

Mean abs curvature: 0.297 deg/m

Mean abs curvature rate: 0.339 deg/ms

Mean abs lateral accel: 0.210 m/(ss)

Mean abs lateral accel GT: 0.236 m/(ss)

Mean abs lateral jerk: 0.189 m/(sss)

Mean abs lateral jerk GT: 0.181 m/(sss)

No desire segments

Metrics

passed :976 / 1044 aka 93.48659003831418%

better :911 / 1044 aka 87.26053639846744%

best (0.5m):567 / 1044 aka 54.310344827586206%

global bias y: +0.127

global mae y: 0.216

global mae y of lanecenter (when detected) in GT: 0.187

global mae y of lanecenter (when detected) in Sim: 0.199

avg resets: 0.093

Mean abs curvature: 0.065 deg/m

Mean abs curvature rate: 0.061 deg/ms

Mean abs lateral accel: 0.216 m/(ss)

Mean abs lateral accel GT: 0.238 m/(ss)

Mean abs lateral jerk: 0.128 m/(sss)

Mean abs lateral jerk GT: 0.219 m/(sss)

No desire turns (max lat accel > 2.5 m/ss)

Metrics

passed :48 / 77 aka 62.33766233766234%

better :28 / 77 aka 36.36363636363637%

best (0.5m):3 / 77 aka 3.896103896103896%

global bias y: +0.152

global mae y: 0.351

global mae y of lanecenter (when detected) in GT: 0.264

global mae y of lanecenter (when detected) in Sim: 0.338

avg resets: 0.571

Mean abs curvature: 0.260 deg/m

Mean abs curvature rate: 0.225 deg/ms

Mean abs lateral accel: 0.574 m/(ss)

Mean abs lateral accel GT: 0.612 m/(ss)

Mean abs lateral jerk: 0.325 m/(sss)

Mean abs lateral jerk GT: 0.374 m/(sss)

Left lane change segments

127/138 aka 92.03%

better: 92/138 aka 66.67%

best (0.5m): 39/138 aka 28.26%

global mae: 0.240920

avg resets: 0.130435

Right lane change segments

145/164 aka 88.41%

better: 92/164 aka 56.10%

best (0.5m): 20/164 aka 12.20%

global mae: 0.282227

avg resets: 0.134146

Left turn segments

51/65 aka 78.46%

better: 17/65 aka 26.15%

best (0.5m): 4/65 aka 6.15%

global mae: 0.314764

avg resets: 1.000000

Right turn segments

57/80 aka 71.25%

better: 23/80 aka 28.75%

best (0.5m): 3/80 aka 3.75%

global mae: 0.369255

avg resets: 0.700000

Left keep segments

27/37 aka 72.97%

better: 19/37 aka 51.35%

best (0.5m): 9/37 aka 24.32%

global mae: 0.311797

avg resets: 0.324324

Right keep segments

30/40 aka 75.00%

better: 23/40 aka 57.50%

best (0.5m): 6/40 aka 15.00%

global mae: 0.312382

avg resets: 0.300000

Plan y error in turns (> 6°/s)

Mean path error : 0.134 meters

Mean absolute path error : 0.552 meters

Turn bias test (y_pos at 50m > 3m)

got 28461 left samples

got 31783 right samples

mean left gt: 0.16554153

mean right gt: 0.046444252

mean turn lanecenter error (GT pos, GT ll): left = 0.11865702271461487, right = -0.2751586437225342

mean turn lanecenter error (sim pos, GT ll): left = 0.2884436249732971, right = -0.23511014878749847

mean turn lanecenter error (GT pos, model ll): left = 0.12490925192832947, right = -0.2477678507566452

mean turn lanecenter error (sim pos, model ll): left = 0.255417138338089, right = -0.21663609147071838

Mean abs curvature: 0.184838 deg/m

Mean abs curvature rate: 0.084679 deg/ms

Mean gt pos in left and right turns

Lane position frequency-domain analysis

Mean frequency: 0.166585 Hz, period 6.002955 s

Mean amplitude: 0.097223 m

Mean ping-pong significance (frequency x amplitude): 1.596382 %

Desire analysis

Desire pulses by type

Command	Count
turn_left	112
turn_right	163
lane_change_left	283
lane_change_right	302
keep_left	67
keep_right	77

Mean difference in path y-coord after desire pulse

Command	Diff
turn_left	-3.975m
turn_right	+3.219m
lane_change_left	-1.224m
lane_change_right	+1.398m
keep_left	-1.098m
keep_right	+1.307m

Mean desire duration - ground truth

Command	Duration
turn_left	6.405s
turn_right	6.198s
lane_change_left	3.345s
lane_change_right	3.320s
keep_left	3.497s
keep_right	3.821s

Elapsed simulator displacement in command direction after desire pulse

Command	20cm	50cm	100cm	200cm
turn_left	2.60s (80/112)	3.07s (75/112)	3.28s (71/112)	3.67s (61/112)
turn_right	1.97s (141/163)	2.46s (137/163)	2.88s (132/163)	3.36s (117/163)
lane_change_left	1.14s (232/283)	1.60s (220/283)	2.13s (196/283)	3.00s (129/283)
lane_change_right	1.19s (244/302)	1.67s (234/302)	2.25s (213/302)	2.99s (118/302)

Meta analysis

Group	Gas prob	Brake prob	Steer prob
All segments	0.004	0.014	0.261
Easy straight	0.003	0.010	0.150
Low speed	0.007	0.023	0.386
No desire segments	0.003	0.010	0.194
No desire turns (max lat accel > 2.5 m/ss)	0.006	0.024	0.413