Model analysis report for

▶
root: {} 10 keys
- report_name: 'rldriving_driving.analyse_driving_rl'
- route_list: '/home/batman/xx/routes/default_test_2k'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'rldriving_driving.analyse_driving_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-06-07_19-42-25'
  - force_local: false
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 20 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 12 keys
    - supercombo: 'b92dd772-6ae6-4329-880d-7e1cc60dd9da/700'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: true
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - use_raw_plan_long: false
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.394

Plan error when stationary

Plan velocity MAE when stationary: 0.085 meters/sec

Plan y error

Plan y MAE: 0.239 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.824 meters

Plan y MAE in turns (C2): 0.811 meters

Plan y MAE in turns (C3): 0.844 meters

Plan y error when path changing

Plan y MAE when path changing: 0.532 meters

Velocity prediction

Velocity MAE: 0.461 meters/sec

Plan x vel error

Plan x vel MAE: 0.461 meters/sec

Plan x vel mean err: 0.130 meters/sec

Pose x error

Pose x MAE: 0.451 meters/sec

Pose x mean err: 0.036 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.004 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.006 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.426 meters

Road edges

Road edge overall MAE: 1.373 meters

Lead pred

Leads (t=0) MAE: 1.791

Leads (t=0) MedAE: 0.785

Lead prob

Lead prob overall MAE: 0.060

Lead prob overall accuracy: 0.987

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	23165	0.992661	0.394169	0.10963	0.0781956	0.0978745
10 < lead x < 20	41124	0.987136	0.555655	0.148138	0.375289	0.240849
20 < lead x < 40	96012	0.991855	0.911875	0.175198	0.539999	0.197639
40 < lead x < 80	112011	0.988653	2.11961	0.248929	0.988417	0.157073
80 < lead x	70548	0.869918	8.20327	0.491185	2.00018	0.143025

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	253670	0.975575	1.21658	0.197495	0.597978	0.158197
2 < \|v_ego - v_lead\| < 5	69139	0.945921	2.46992	0.310803	1.43268	0.202985
5 < \|v_ego - v_lead\| < 10	16573	0.894588	4.42883	0.412755	2.74554	0.352464
10 < \|v_ego - v_lead\| < 15	3130	0.944409	4.98611	0.428625	2.42941	0.443643
15 < \|v_ego - v_lead\|	453	0.838852	18.3044	0.794462	4.95052	0.642773

Group	Gas prob	Brake prob	Steer prob
All segments	0.009	0.023	0.253
Easy straight	0.006	0.009	0.126
Low speed	0.016	0.035	0.364
No desire	0.007	0.014	0.192
No desire turns (max lat accel > 2.5 m/ss)	0.011	0.029	0.366