Model analysis report for

• ▶
  root: {} 11 keys
  • report_name: 'mlsimdriving_driving.analyse_long_rl'
  • route_list: '/code.nfs/branches/caches/workstation-yassine_2025-10-06_15-35-15/xx/datasets/lists/train_500k_20240617_test_2k.txt'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • shuffle: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'mlsimdriving_driving.analyse_long_rl'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: '/code.nfs/branches/caches/workstation-yassine_2025-10-06_15-35-15/xx'
    • use_local_codedir: false
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 32 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2025_07'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • ▶
        physics: {} 3 keys
        • type: 'BICYCLE'
        • ▶
          bicycle_model_config: {} 1 keys
          • vehicle_platform: 'HONDA_CIVIC'
        • ▶
          ml_model_config: {} 11 keys
          • platform: 'TOYOTA_COROLLA_TSS2'
          • eid: 'effd03d9-472d-44a1-910a-6d49a41eb63f/50'
          • y_range: [-3.0, 3.0]
          • vocab_size: 1024
          • temperature: 0.5
          • use_gt_features: false
          • platform_vec_size: 256
          • platform_map_path: '/code.nfs/branches/caches/workstation-yassine_2025-10-06_15-35-15/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
          • lateral_tuning_type: 'AUTO'
          • ▶
            lateral_tuning_params: {} 0 keys
          • ▶
            lateral_initial_state_params: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_decoder_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 9 keys
      • supercombo: 'e2d9c622-25a8-4ccd-8c8e-c62537b7aa0c/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • use_allow_throttle: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: true
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1690 segments || 1464.6508333333334 minutes

Metrics

1544/1690 aka 91.36094674556213%

better: 0/1690 aka 0.0%

best (0.5m): 0/1690 aka 0.0%

global mae x: 1.143

global mae y: 0.066

total # resets: 6382.000

reset per minute : 4.357

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.325 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 0.946 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

665 segments || 563.6333333333333 minutes

Metrics

660/665 aka 99.24812030075188%

better: 0/665 aka 0.0%

best (0.5m): 0/665 aka 0.0%

global mae x: 1.163

global mae y: 0.024

total # resets: 2219.000

reset per minute : 3.937

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.265 m/(ss)

Mean abs long acceleration GT: 0.288 m/(ss)

Mean abs long jerk: 0.794 m/(sss)

Mean abs long jerk GT: 0.449 m/(sss)

Hard acceleration

306 segments || 16.12 minutes

Metrics

300/306 aka 98.03921568627452%

better: 72/306 aka 23.529411764705884%

best (0.5m): 31/306 aka 10.130718954248366%

global mae x: 1.070

global mae y: 0.176

total # resets: 147.000

reset per minute : 9.119

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.244 m/(ss)

Mean abs long acceleration GT: 2.020 m/(ss)

Mean abs long jerk: 1.766 m/(sss)

Mean abs long jerk GT: 1.394 m/(sss)

Hard deceleration

490 segments || 29.346666666666668 minutes

Metrics

485/490 aka 98.9795918367347%

better: 97/490 aka 19.79591836734694%

best (0.5m): 50/490 aka 10.204081632653061%

global mae x: 1.112

global mae y: 0.137

total # resets: 190.000

reset per minute : 6.474

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.476 m/(ss)

Mean abs long acceleration GT: 2.106 m/(ss)

Mean abs long jerk: 1.588 m/(sss)

Mean abs long jerk GT: 1.325 m/(sss)

No lead

514 segments || 214.9575 minutes

Metrics

484/514 aka 94.16342412451363%

better: 29/514 aka 5.642023346303502%

best (0.5m): 17/514 aka 3.307392996108949%

global mae x: 1.152

global mae y: 0.072

total # resets: 974.000

reset per minute : 4.531

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.289 m/(ss)

Mean abs long acceleration GT: 0.319 m/(ss)

Mean abs long jerk: 0.964 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

551 segments || 110.93166666666666 minutes

Metrics

540/551 aka 98.00362976406534%

better: 79/551 aka 14.337568058076226%

best (0.5m): 20/551 aka 3.629764065335753%

global mae x: 1.036

global mae y: 0.122

total # resets: 299.000

reset per minute : 2.695

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.420 m/(ss)

Mean abs long acceleration GT: 0.506 m/(ss)

Mean abs long jerk: 0.655 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

452 segments || 17.278333333333332 minutes

Metrics

449/452 aka 99.33628318584071%

better: 125/452 aka 27.654867256637168%

best (0.5m): 63/452 aka 13.938053097345133%

global mae x: 1.083

global mae y: 0.203

total # resets: 135.000

reset per minute : 7.813

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.173 m/(ss)

Mean abs long acceleration GT: 1.387 m/(ss)

Mean abs long jerk: 1.445 m/(sss)

Mean abs long jerk GT: 0.831 m/(sss)

Strong turns (>2 m/ss)

566 segments || 42.6775 minutes

Metrics

527/566 aka 93.1095406360424%

better: 124/566 aka 21.908127208480565%

best (0.5m): 62/566 aka 10.954063604240282%

global mae x: 1.090

global mae y: 0.436

total # resets: 191.000

reset per minute : 4.475

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.477 m/(ss)

Mean abs long acceleration GT: 0.521 m/(ss)

Mean abs long jerk: 1.403 m/(sss)

Mean abs long jerk GT: 0.799 m/(sss)