Model analysis report for

▶
root: {} 11 keys
- report_name: 'rldriving_analysedriving'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- shuffle: false
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 1000
  - job_name: 'rldriving_analysedriving'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-05-13_10-13-43'
  - force_local: false
- route_list: '/home/batman/xx/routes/default_test_2k'
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 20 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 10 keys
    - supercombo: 'e30305e9-5fb8-48bd-a278-f714e880aef1/700'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: true
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - use_raw_plan_long: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.393

Plan error when stationary

Plan velocity MAE when stationary: 0.083 meters/sec

Plan y error

Plan y MAE: 0.234 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.852 meters

Plan y MAE in turns (C2): 0.843 meters

Plan y MAE in turns (C3): 0.866 meters

Plan y error when path changing

Plan y MAE when path changing: 0.528 meters

Velocity prediction

Velocity MAE: 0.450 meters/sec

Plan x vel error

Plan x vel MAE: 0.450 meters/sec

Plan x vel mean err: 0.095 meters/sec

Pose x error

Pose x MAE: 0.457 meters/sec

Pose x mean err: 0.043 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.004 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.006 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.426 meters

Road edges

Road edge overall MAE: 1.361 meters

Lead pred

Leads (t=0) MAE: 1.818

Leads (t=0) MedAE: 0.782

Lead prob

Lead prob overall MAE: 0.062

Lead prob overall accuracy: 0.987

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	23165	0.991323	0.420223	0.124498	0.0799533	0.0980771
10 < lead x < 20	41124	0.983829	0.564428	0.142505	0.376231	0.261377
20 < lead x < 40	96012	0.989741	0.923755	0.175379	0.525486	0.193797
40 < lead x < 80	112011	0.98751	2.00708	0.251486	0.992917	0.156315
80 < lead x	70548	0.864036	8.75745	0.486124	1.89619	0.143309

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	253670	0.974171	1.1787	0.19691	0.588814	0.15858
2 < \|v_ego - v_lead\| < 5	69139	0.941466	2.50835	0.313637	1.35963	0.202444
5 < \|v_ego - v_lead\| < 10	16573	0.881856	4.37772	0.435419	2.75539	0.352229
10 < \|v_ego - v_lead\| < 15	3130	0.934505	5.36789	0.518961	2.32281	0.4417
15 < \|v_ego - v_lead\|	453	0.8234	17.2942	0.735047	3.40201	0.620083

Group	Gas prob	Brake prob	Steer prob
All segments	0.009	0.023	0.254
Easy straight	0.006	0.009	0.127
Low speed	0.016	0.035	0.365
No desire	0.007	0.014	0.193
No desire turns (max lat accel > 2.5 m/ss)	0.011	0.029	0.368