Model analysis report for

• ▶
  root: {} 10 keys
  • report_name: 'rldriving_driving.analyse_lat_cheat_rl'
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'rldriving_driving.analyse_lat_cheat_rl'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-yassine_2025-06-09_15-43-38'
    • force_local: false
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 30 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: true
      • fps: 20
      • allow_lat: true
      • allow_long: false
      • min_hist: 40
      • compressor_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'REALISTIC'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 12 keys
      • supercombo: '6a8a3da8-c264-4f91-b0a6-d04722cccfce/700'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: true
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: false
      • use_raw_plan_long: false
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments

Metrics

passed :1285 / 1658 aka 77.503%

better :935 / 1658 aka 56.393%

best (0.5m):438 / 1658 aka 26.417%

global mae x: 0.168

global mae y: 0.262

global mae y of lanecenter (when detected) in GT: 0.209

global mae y of lanecenter (when detected) in Sim: 0.252

avg resets: 0.358

Mean abs curvature: 0.218 deg/m

Mean abs curvature rate: 0.304 deg/ms

Mean abs lateral accel: 0.369 m/(ss)

Mean abs lateral accel GT: 0.336 m/(ss)

Mean abs lateral jerk: 0.495 m/(sss)

Mean abs lateral jerk GT: 0.272 m/(sss)

Easy straight

Metrics

passed :507 / 510 aka 99.412%

better :490 / 510 aka 96.078%

best (0.5m):298 / 510 aka 58.431%

global mae x: 0.106

global mae y: 0.175

global mae y of lanecenter (when detected) in GT: 0.161

global mae y of lanecenter (when detected) in Sim: 0.182

avg resets: 0.018

Mean abs curvature: 0.049 deg/m

Mean abs curvature rate: 0.089 deg/ms

Mean abs lateral accel: 0.173 m/(ss)

Mean abs lateral accel GT: 0.130 m/(ss)

Mean abs lateral jerk: 0.303 m/(sss)

Mean abs lateral jerk GT: 0.171 m/(sss)

Low speed

Metrics

passed :294 / 347 aka 84.726%

better :190 / 347 aka 54.755%

best (0.5m):71 / 347 aka 20.461%

global mae x: 0.229

global mae y: 0.279

global mae y of lanecenter (when detected) in GT: 0.266

global mae y of lanecenter (when detected) in Sim: 0.267

avg resets: 0.478

Mean abs curvature: 0.423 deg/m

Mean abs curvature rate: 0.610 deg/ms

Mean abs lateral accel: 0.313 m/(ss)

Mean abs lateral accel GT: 0.305 m/(ss)

Mean abs lateral jerk: 0.434 m/(sss)

Mean abs lateral jerk GT: 0.228 m/(sss)

No desire segments

Metrics

passed :920 / 938 aka 98.081%

better :826 / 938 aka 88.060%

best (0.5m):426 / 938 aka 45.416%

global mae x: 0.125

global mae y: 0.196

global mae y of lanecenter (when detected) in GT: 0.180

global mae y of lanecenter (when detected) in Sim: 0.192

avg resets: 0.064

Mean abs curvature: 0.120 deg/m

Mean abs curvature rate: 0.161 deg/ms

Mean abs lateral accel: 0.318 m/(ss)

Mean abs lateral accel GT: 0.284 m/(ss)

Mean abs lateral jerk: 0.385 m/(sss)

Mean abs lateral jerk GT: 0.236 m/(sss)

No desire turns (max lat accel > 2.5 m/ss)

Metrics

passed :124 / 132 aka 93.939%

better :79 / 132 aka 59.848%

best (0.5m):15 / 132 aka 11.364%

global mae x: 0.214

global mae y: 0.256

global mae y of lanecenter (when detected) in GT: 0.260

global mae y of lanecenter (when detected) in Sim: 0.231

avg resets: 0.174

Mean abs curvature: 0.290 deg/m

Mean abs curvature rate: 0.329 deg/ms

Mean abs lateral accel: 0.666 m/(ss)

Mean abs lateral accel GT: 0.649 m/(ss)

Mean abs lateral jerk: 0.620 m/(sss)

Mean abs lateral jerk GT: 0.398 m/(sss)

Left lane change segments

35/120 aka 29.17%

better: 12/120 aka 10.00%

best (0.5m): 1/120 aka 0.83%

mean max error C3: 2.137862

mean max error C2: 2.411382

global mae: 0.348708

avg resets: 0.375000

Right lane change segments

44/150 aka 29.33%

better: 11/150 aka 7.33%

best (0.5m): 4/150 aka 2.67%

mean max error C3: 2.223350

mean max error C2: 2.427628

global mae: 0.342371

avg resets: 0.593333

Left turn segments

69/79 aka 87.34%

better: 22/79 aka 27.85%

best (0.5m): 0/79 aka 0.00%

mean max error C3: 1.333027

mean max error C2: 1.390971

global mae: 0.289714

avg resets: 0.810127

Right turn segments

79/91 aka 86.81%

better: 32/91 aka 35.16%

best (0.5m): 3/91 aka 3.30%

mean max error C3: 1.258187

mean max error C2: 1.371061

global mae: 0.289876

avg resets: 0.769231

Left keep segments

18/33 aka 54.55%

better: 10/33 aka 30.30%

best (0.5m): 2/33 aka 6.06%

mean max error C3: 1.570003

mean max error C2: 1.889257

global mae: 0.296570

avg resets: 0.363636

Right keep segments

19/26 aka 73.08%

better: 5/26 aka 19.23%

best (0.5m): 2/26 aka 7.69%

mean max error C3: 2.091650

mean max error C2: 1.301632

global mae: 0.324842

avg resets: 0.192308

Plan y error in turns (> 6°/s)

Mean path error : 0.005 meters

Mean absolute path error : 0.431 meters

Turn bias test (y_pos at 50m > 3m)

got 29069 left samples

got 30815 right samples

mean left gt: -0.017986309

mean right gt: 0.018264374

mean turn lanecenter error (GT pos, GT ll): left = 0.21726641058921814, right = -0.24762772023677826

mean turn lanecenter error (sim pos, GT ll): left = 0.22301208972930908, right = -0.2354280799627304

mean turn lanecenter error (GT pos, model ll): left = 0.20013463497161865, right = -0.25039684772491455

mean turn lanecenter error (sim pos, model ll): left = 0.17624036967754364, right = -0.21204376220703125

Mean abs curvature: 0.214210 deg/m

Mean abs curvature rate: 0.203573 deg/ms

Mean gt pos in left and right turns

Lane position frequency-domain analysis

Mean frequency: 0.175560 Hz, period 5.696047 s

Mean amplitude: 0.137568 m

Mean ping-pong significance (frequency x amplitude): 2.342404 %