Model analysis report for

▶
root: {} 11 keys
- report_name: 'mlsimdriving_driving.analyse_driving_rl'
- route_list: '/home/batman/xx/routes/default_test_2k'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'mlsimdriving_driving.analyse_driving_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-07-06_10-53-22'
  - force_local: false
- plan_only: false
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 22 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_decoder_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - ▶
      physics: {} 4 keys
      - type: 'BICYCLE'
      - ▶
        bicycle_model_config: {} 1 keys
        
        vehicle_platform: 'HONDA_CIVIC'
      - ▶
        classic_model_config: {} 1 keys
        
        platform: 'COROLLA'
      - ▶
        ml_model_config: {} 6 keys
        
        platform: 'COROLLA'
        
        eid: 'd3e5ef06-14ec-4e35-b708-d8db73d29f1e/51'
        
        y_range: [-5.0, 5.0]
        
        vocab_size: 2048
        
        temperature: 0.5
        
        use_gt_features: false
    - with_gt: false
  - ▶
    agent: {} 12 keys
    - supercombo: 'fef4d896-597e-4c6f-90cf-8eb7102c89d2/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: true
    - use_raw_plan_long: true
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.383

Plan error when stationary

Plan velocity MAE when stationary: 0.087 meters/sec

Plan y error

Plan y MAE: 0.209 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.769 meters

Plan y MAE in turns (C2): 0.762 meters

Plan y MAE in turns (C3): 0.780 meters

Plan y error when path changing

Plan y MAE when path changing: 0.489 meters

Velocity prediction

Velocity MAE: 0.420 meters/sec

Plan x vel error

Plan x vel MAE: 0.420 meters/sec

Plan x vel mean err: 0.070 meters/sec

Pose x error

Pose x MAE: 0.458 meters/sec

Pose x mean err: 0.072 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.005 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.006 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.426 meters

Road edges

Road edge overall MAE: 1.325 meters

Lead pred

Leads (t=0) MAE: 1.347

Leads (t=0) MedAE: 0.699

Lead prob

Lead prob overall MAE: 0.068

Lead prob overall accuracy: 0.984

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	26212	0.986876	0.320021	0.101616	0.0486968	0.0672363
10 < lead x < 20	40561	0.983679	0.501459	0.134152	0.366	0.251387
20 < lead x < 40	95999	0.985448	0.831936	0.158426	0.621254	0.20028
40 < lead x < 80	111485	0.984913	1.74953	0.233262	1.00435	0.156103
80 < lead x	49000	0.828102	5.99371	0.375671	1.77834	0.136657

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	247114	0.974971	0.987072	0.176768	0.588876	0.157381
2 < \|v_ego - v_lead\| < 5	60237	0.929014	1.79332	0.251301	1.49632	0.213384
5 < \|v_ego - v_lead\| < 10	13300	0.871203	2.94608	0.317965	2.52441	0.338229
10 < \|v_ego - v_lead\| < 15	2401	0.897959	3.11847	0.40747	2.03296	0.420058
15 < \|v_ego - v_lead\|	308	0.707792	7.2586	0.722804	3.63782	0.472273

Group	Gas prob	Brake prob	Steer prob
All segments	0.010	0.022	0.258
Easy straight	0.006	0.008	0.128
Low speed	0.017	0.035	0.369
No desire	0.008	0.013	0.198
No desire turns (max lat accel > 2.5 m/ss)	0.013	0.029	0.382