Model analysis report for

▶
root: {} 13 keys
- report_name: 'mlsimdriving_driving.analyse_driving_rl'
- route_list: '/code.nfs/branches/caches/workstation-harald_2026-01-07_09-23-07/xx/datasets/lists/train_500k_20240617_test_2k.txt'
- output_dir: null
- save_tmp: false
- read_only: false
- shuffle: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'mlsimdriving_driving.analyse_driving_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - unique_id: null
  - codedir: '/code.nfs/branches/caches/workstation-harald_2026-01-07_09-23-07/xx'
  - use_local_codedir: false
- plan_only: false
- ▶
  filter: {} 3 keys
  - min_hist: 100
  - max_hist: -60
  - max_fidx: 1000
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 22 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2025_07'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - ▶
      physics: {} 3 keys
      - type: 'BICYCLE'
      - ▶
        bicycle_model_config: {} 1 keys
        
        vehicle_platform: 'HONDA_CIVIC'
      - ▶
        ml_model_config: {} 11 keys
        
        platform: 'TOYOTA_COROLLA_TSS2'
        
        eid: 'effd03d9-472d-44a1-910a-6d49a41eb63f/50'
        
        y_range: [-3.0, 3.0]
        
        vocab_size: 1024
        
        temperature: 0.5
        
        use_gt_features: false
        
        platform_vec_size: 256
        
        platform_map_path: '/code.nfs/branches/caches/workstation-harald_2026-01-07_09-23-07/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
        
        lateral_tuning_type: 'AUTO'
        
        ▶
        lateral_tuning_params: {} 0 keys
        
        ▶
        lateral_initial_state_params: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 20
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_decoder_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 8 keys
    - supercombo: 'f1d30a23-4122-400a-80a6-557502284c36/200'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - use_allow_throttle: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

Plan overall error

Plan overall MAE: 0.378

Plan POSITION MAE: 2.923, 0.855, 0.165

Plan VELOCITY MAE: 0.962, 0.058, 0.042

Plan ACCELERATION MAE: 0.285, 0.182, 0.125

Plan T_FROM_CURRENT_EULER MAE: 0.007, 0.004, 0.031

Plan ORIENTATION_RATE MAE: 0.008, 0.007, 0.013

Plan error when stationary

Plan velocity MAE when stationary: 0.069 meters/sec

Plan y error

Plan y MAE: 0.226 meters

Plan y error in turns (> 6°/s)

Plan y MAE in turns: 0.757 meters

Plan y MAE in turns (C2): 0.746 meters

Plan y MAE in turns (C3): 0.772 meters

Plan y error when path changing

Plan y MAE when path changing: 0.509 meters

Velocity prediction

Velocity MAE: 0.422 meters/sec

Plan x vel error

Plan x vel MAE: 0.422 meters/sec

Plan x vel mean err: 0.102 meters/sec

Pose x error

Pose x MAE: 0.447 meters/sec

Pose x mean err: 0.039 meters/sec

Wide from device

C3 Wide from device roll MAE: 0.000 radians

C2 Wide from device roll MAE: 0.000 radians

C3 Wide from device pitch MAE: 0.003 radians

C2 Wide from device pitch MAE: 0.001 radians

C3 Wide from device yaw MAE: 0.005 radians

C2 Wide from device yaw MAE: 0.001 radians

Lanelines

Lane line overall MAE: 0.449 meters

Road edges

Road edge overall MAE: 1.398 meters

Lead pred

Leads (t=0) MAE: 1.594

Leads (t=0) MedAE: 0.801

Lead prob

Lead prob overall MAE: 0.074

Lead prob overall accuracy: 0.983

First lead metrics for various lead x buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
lead x < 10	27459	0.984158	0.348622	0.120644	0.0557803	0.0658942
10 < lead x < 20	42322	0.978096	0.655852	0.180242	0.374232	0.234552
20 < lead x < 40	98674	0.984211	1.12809	0.220567	0.574748	0.187255
40 < lead x < 80	113788	0.978442	2.35632	0.286284	0.903402	0.150191
80 < lead x	49743	0.765535	6.78053	0.419052	1.61876	0.131447

First lead metrics for various |v_ego - v_lead| buckets

	count	prob accuracy	x MedAE	y MedAE	v MedAE	a MedAE
\|v_ego - v_lead\| < 2	251816	0.962878	1.30555	0.227602	0.529638	0.148359
2 < \|v_ego - v_lead\| < 5	62746	0.918098	2.36143	0.309901	1.50225	0.203743
5 < \|v_ego - v_lead\| < 10	14520	0.855579	3.74684	0.393966	2.84535	0.321344
10 < \|v_ego - v_lead\| < 15	2629	0.881324	3.38993	0.41458	3.0683	0.411114
15 < \|v_ego - v_lead\|	377	0.625995	8.7203	1.05602	5.80964	0.818943

Group	Gas prob	Brake prob	Steer prob
All segments	0.008	0.023	0.261
Easy straight	0.006	0.010	0.131
Low speed	0.013	0.035	0.362
No desire	0.006	0.014	0.199
No desire turns (max lat accel > 2.5 m/ss)	0.010	0.026	0.378