Model analysis report for

▶
root: {} 9 keys
- report_name: 'rldriving_analysedriving'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 1000
  - job_name: 'rldriving_analysedriving'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-01-25_18-29-03'
  - force_local: false
- route_list: '/home/batman/xx/routes/default_test_2k'
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 13 keys
    - ▶
      ep: {} 6 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - lpips_model: null
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 15 keys
    - supercombo: 'b25c27a8-263b-4f3f-b1f8-72ac49b4f9c7/700'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - act_on_lat: true
    - act_on_long: false
    - min_hist_lat: true
    - min_hist_long: 10
    - calibrate_speed: false
    - run_planners: false
    - use_long_mpc: true
    - use_allow_throttle: false
    - use_prev_desired_curv: true
    - model_trained_fps: 20
    - use_worldmodel_as_agent: false
    - use_raw_plan: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.401

Plan error when stationary

Plan velocity MAE when stationary: 0.080 meters/sec

Plan y error

Plan y MAE: 0.241 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.840 meters

Plan y MAE in turns (C2): 0.816 meters

Plan y MAE in turns (C3): 0.875 meters

Plan y error when path changing

Plan y MAE when path changing: 0.533 meters

Velocity prediction

Velocity MAE: 0.485 meters/sec

Plan x vel error

Plan x vel MAE: 0.485 meters/sec

Plan x vel mean err: 0.072 meters/sec

Pose x error

Pose x MAE: 0.495 meters/sec

Pose x mean err: 0.023 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.004 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.005 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.431 meters

Road edges

Road edge overall MAE: 1.376 meters

Lead pred

Leads (t=0) MAE: 1.863

Leads (t=0) MedAE: 0.823

Lead prob

Lead prob overall MAE: 0.059

Lead prob overall accuracy: 0.988

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	23165	0.994215	0.402514	0.104837	0.0756864	0.0971169
10 < lead x < 20	41124	0.988523	0.62409	0.144869	0.382487	0.247902
20 < lead x < 40	96012	0.990585	0.967455	0.174328	0.607097	0.197005
40 < lead x < 80	112011	0.990787	2.07387	0.252639	1.09514	0.157258
80 < lead x	70548	0.872229	8.62587	0.489786	2.15326	0.146157

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	253670	0.977132	1.22797	0.196526	0.64877	0.158412
2 < \|v_ego - v_lead\| < 5	69139	0.943549	2.63819	0.303324	1.53973	0.209255
5 < \|v_ego - v_lead\| < 10	16573	0.904664	4.3416	0.448383	3.14824	0.350887
10 < \|v_ego - v_lead\| < 15	3130	0.937061	5.77509	0.518664	3.32816	0.439039
15 < \|v_ego - v_lead\|	453	0.834437	13.6262	0.991628	4.79036	0.731974

Group	Gas prob	Brake prob	Steer prob
All segments	0.009	0.022	0.260
Easy straight	0.006	0.008	0.133
Low speed	0.016	0.034	0.366
No desire	0.007	0.013	0.200
No desire turns (max lat accel > 2.5 m/ss)	0.012	0.029	0.379