Model analysis report for

• ▶
  root: {} 11 keys
  • report_name: 'mlsimdriving_driving.analyse_long_rl'
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • shuffle: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'mlsimdriving_driving.analyse_long_rl'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-harald_2025-08-15_16-18-00'
    • force_local: false
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 32 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_decoder_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • ▶
        physics: {} 4 keys
        • type: 'BICYCLE'
        • ▶
          bicycle_model_config: {} 1 keys
          • vehicle_platform: 'HONDA_CIVIC'
        • ▶
          classic_model_config: {} 1 keys
          • platform: 'TOYOTA_COROLLA_TSS2'
        • ▶
          ml_model_config: {} 8 keys
          • platform: 'TOYOTA_COROLLA_TSS2'
          • eid: '3bec45a4-9512-4c7f-bdbc-9415e5890fb6/100'
          • y_range: [-5.0, 5.0]
          • vocab_size: 2048
          • temperature: 0.5
          • use_gt_features: false
          • platform_vec_size: 256
          • platform_map_path: '/code.nfs/branches/caches/workstation-harald_2025-08-15_16-18-00/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 11 keys
      • supercombo: 'a8f96b93-bde2-4e28-a732-4df21ebba968/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • use_allow_throttle: false
      • use_prev_desired_curv: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: true
      • use_raw_plan_long: true
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1588 segments || 1376.2508333333333 minutes

Metrics

1505/1588 aka 94.77329974811083%

better: 0/1588 aka 0.0%

best (0.5m): 0/1588 aka 0.0%

global mae x: 1.095

global mae y: 0.064

total # resets: 6812.000

reset per minute : 4.950

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.368 m/(ss)

Mean abs long acceleration GT: 0.369 m/(ss)

Mean abs long jerk: 1.080 m/(sss)

Mean abs long jerk GT: 0.546 m/(sss)

Easy straight (whole segments)

630 segments || 534.4641666666666 minutes

Metrics

625/630 aka 99.2063492063492%

better: 0/630 aka 0.0%

best (0.5m): 0/630 aka 0.0%

global mae x: 1.095

global mae y: 0.023

total # resets: 2438.000

reset per minute : 4.562

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.299 m/(ss)

Mean abs long acceleration GT: 0.282 m/(ss)

Mean abs long jerk: 0.902 m/(sss)

Mean abs long jerk GT: 0.445 m/(sss)

Hard acceleration

292 segments || 15.4 minutes

Metrics

285/292 aka 97.6027397260274%

better: 63/292 aka 21.575342465753426%

best (0.5m): 27/292 aka 9.246575342465754%

global mae x: 1.079

global mae y: 0.164

total # resets: 110.000

reset per minute : 7.143

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.359 m/(ss)

Mean abs long acceleration GT: 2.023 m/(ss)

Mean abs long jerk: 1.892 m/(sss)

Mean abs long jerk GT: 1.388 m/(sss)

Hard deceleration

450 segments || 26.858333333333334 minutes

Metrics

445/450 aka 98.88888888888889%

better: 103/450 aka 22.88888888888889%

best (0.5m): 56/450 aka 12.444444444444445%

global mae x: 1.084

global mae y: 0.145

total # resets: 187.000

reset per minute : 6.962

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.480 m/(ss)

Mean abs long acceleration GT: 2.100 m/(ss)

Mean abs long jerk: 1.702 m/(sss)

Mean abs long jerk GT: 1.328 m/(sss)

No lead

487 segments || 206.64833333333334 minutes

Metrics

473/487 aka 97.12525667351129%

better: 23/487 aka 4.722792607802875%

best (0.5m): 17/487 aka 3.490759753593429%

global mae x: 1.081

global mae y: 0.071

total # resets: 1080.000

reset per minute : 5.226

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.340 m/(ss)

Mean abs long acceleration GT: 0.319 m/(ss)

Mean abs long jerk: 1.118 m/(sss)

Mean abs long jerk GT: 0.532 m/(sss)

Come to stop

518 segments || 105.47583333333334 minutes

Metrics

510/518 aka 98.45559845559846%

better: 78/518 aka 15.057915057915057%

best (0.5m): 25/518 aka 4.826254826254826%

global mae x: 0.963

global mae y: 0.108

total # resets: 328.000

reset per minute : 3.110

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.424 m/(ss)

Mean abs long acceleration GT: 0.498 m/(ss)

Mean abs long jerk: 0.697 m/(sss)

Mean abs long jerk GT: 0.487 m/(sss)

Start from stop

428 segments || 16.268333333333334 minutes

Metrics

426/428 aka 99.53271028037383%

better: 124/428 aka 28.97196261682243%

best (0.5m): 57/428 aka 13.317757009345794%

global mae x: 1.126

global mae y: 0.217

total # resets: 111.000

reset per minute : 6.823

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.256 m/(ss)

Mean abs long acceleration GT: 1.409 m/(ss)

Mean abs long jerk: 1.480 m/(sss)

Mean abs long jerk GT: 0.824 m/(sss)

Strong turns (>2 m/ss)

526 segments || 39.64333333333333 minutes

Metrics

506/526 aka 96.1977186311787%

better: 107/526 aka 20.342205323193916%

best (0.5m): 48/526 aka 9.125475285171103%

global mae x: 1.164

global mae y: 0.435

total # resets: 181.000

reset per minute : 4.566

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.546 m/(ss)

Mean abs long acceleration GT: 0.519 m/(ss)

Mean abs long jerk: 1.497 m/(sss)

Mean abs long jerk GT: 0.797 m/(sss)