Model analysis report for

▶
root: {} 12 keys
- report_name: 'mlsimdriving_driving.analyse_driving_rl'
- route_list: '/home/batman/xx/routes/default_test_2k'
- output_dir: null
- save_tmp: false
- read_only: false
- shuffle: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'mlsimdriving_driving.analyse_driving_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-harald_2025-07-25_08-15-05'
  - force_local: false
- plan_only: false
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 22 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_decoder_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - ▶
      physics: {} 4 keys
      - type: 'BICYCLE'
      - ▶
        bicycle_model_config: {} 1 keys
        
        vehicle_platform: 'HONDA_CIVIC'
      - ▶
        classic_model_config: {} 1 keys
        
        platform: 'TOYOTA_COROLLA_TSS2'
      - ▶
        ml_model_config: {} 8 keys
        
        platform: 'TOYOTA_COROLLA_TSS2'
        
        eid: '3bec45a4-9512-4c7f-bdbc-9415e5890fb6/76'
        
        y_range: [-5.0, 5.0]
        
        vocab_size: 2048
        
        temperature: 0.5
        
        use_gt_features: false
        
        platform_vec_size: 256
        
        platform_map_path: '/code.nfs/branches/caches/workstation-harald_2025-07-25_08-15-05/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
    - with_gt: false
  - ▶
    agent: {} 11 keys
    - supercombo: 'f35eaf21-ef51-413d-9857-398bcdabb7da/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: true
    - use_raw_plan_long: true
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.372

Plan error when stationary

Plan velocity MAE when stationary: 0.086 meters/sec

Plan y error

Plan y MAE: 0.220 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.803 meters

Plan y MAE in turns (C2): 0.797 meters

Plan y MAE in turns (C3): 0.813 meters

Plan y error when path changing

Plan y MAE when path changing: 0.494 meters

Velocity prediction

Velocity MAE: 0.414 meters/sec

Plan x vel error

Plan x vel MAE: 0.414 meters/sec

Plan x vel mean err: 0.064 meters/sec

Pose x error

Pose x MAE: 0.462 meters/sec

Pose x mean err: 0.071 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.004 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.006 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.428 meters

Road edges

Road edge overall MAE: 1.336 meters

Lead pred

Leads (t=0) MAE: 1.344

Leads (t=0) MedAE: 0.705

Lead prob

Lead prob overall MAE: 0.068

Lead prob overall accuracy: 0.985

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	26212	0.99237	0.288581	0.106408	0.0459651	0.0677456
10 < lead x < 20	40561	0.987574	0.47395	0.131767	0.367307	0.252804
20 < lead x < 40	95999	0.989146	0.826347	0.161527	0.613314	0.198653
40 < lead x < 80	111485	0.985397	1.83051	0.237542	1.0145	0.154627
80 < lead x	49000	0.834653	5.61933	0.375658	1.7264	0.137265

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	247114	0.975732	0.984755	0.178789	0.59151	0.156152
2 < \|v_ego - v_lead\| < 5	60237	0.937148	1.82705	0.250437	1.47921	0.214014
5 < \|v_ego - v_lead\| < 10	13300	0.893985	3.11435	0.344495	2.84471	0.340884
10 < \|v_ego - v_lead\| < 15	2401	0.916285	3.48642	0.37912	2.22444	0.432955
15 < \|v_ego - v_lead\|	308	0.730519	9.14364	0.76636	3.16369	0.452201

Group	Gas prob	Brake prob	Steer prob
All segments	0.010	0.022	0.261
Easy straight	0.006	0.009	0.131
Low speed	0.017	0.036	0.371
No desire	0.007	0.014	0.200
No desire turns (max lat accel > 2.5 m/ss)	0.012	0.029	0.383