Model analysis report for

▶
root: {} 10 keys
- report_name: 'rldriving_driving.analyse_driving_rl'
- route_list: '/home/batman/xx/routes/default_test_2k'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'rldriving_driving.analyse_driving_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-06-20_19-46-11'
  - force_local: false
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 20 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 12 keys
    - supercombo: 'b2714021-a7bc-41d6-8e1c-bfd20e59cc75/700'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: true
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - use_raw_plan_long: false
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.393

Plan error when stationary

Plan velocity MAE when stationary: 0.083 meters/sec

Plan y error

Plan y MAE: 0.233 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.829 meters

Plan y MAE in turns (C2): 0.812 meters

Plan y MAE in turns (C3): 0.855 meters

Plan y error when path changing

Plan y MAE when path changing: 0.523 meters

Velocity prediction

Velocity MAE: 0.455 meters/sec

Plan x vel error

Plan x vel MAE: 0.455 meters/sec

Plan x vel mean err: 0.118 meters/sec

Pose x error

Pose x MAE: 0.458 meters/sec

Pose x mean err: 0.071 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.005 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.006 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.428 meters

Road edges

Road edge overall MAE: 1.355 meters

Lead pred

Leads (t=0) MAE: 1.871

Leads (t=0) MedAE: 0.806

Lead prob

Lead prob overall MAE: 0.067

Lead prob overall accuracy: 0.986

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	23165	0.992877	0.454101	0.112415	0.079993	0.101782
10 < lead x < 20	41124	0.977361	0.613509	0.147577	0.371049	0.241406
20 < lead x < 40	96012	0.985491	0.955402	0.170673	0.555562	0.19309
40 < lead x < 80	112011	0.98693	2.14568	0.253191	1.01148	0.156458
80 < lead x	70548	0.853887	8.96614	0.484332	2.02524	0.147813

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	253670	0.973706	1.24619	0.198825	0.608669	0.157089
2 < \|v_ego - v_lead\| < 5	69139	0.930777	2.63672	0.300802	1.46794	0.207838
5 < \|v_ego - v_lead\| < 10	16573	0.856212	4.77448	0.444651	2.68758	0.350102
10 < \|v_ego - v_lead\| < 15	3130	0.904473	5.60477	0.468145	3.43562	0.47127
15 < \|v_ego - v_lead\|	453	0.728477	14.8919	0.895419	5.30015	0.650942

Group	Gas prob	Brake prob	Steer prob
All segments	0.010	0.022	0.258
Easy straight	0.006	0.008	0.127
Low speed	0.017	0.035	0.369
No desire	0.008	0.013	0.197
No desire turns (max lat accel > 2.5 m/ss)	0.013	0.029	0.382