Model analysis report for

• ▶
  root: {} 11 keys
  • report_name: 'mlsimdriving_driving.analyse_long_rl'
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • shuffle: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'mlsimdriving_driving.analyse_long_rl'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-harald_2025-07-25_08-15-05'
    • force_local: false
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 32 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_decoder_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • ▶
        physics: {} 4 keys
        • type: 'BICYCLE'
        • ▶
          bicycle_model_config: {} 1 keys
          • vehicle_platform: 'HONDA_CIVIC'
        • ▶
          classic_model_config: {} 1 keys
          • platform: 'TOYOTA_COROLLA_TSS2'
        • ▶
          ml_model_config: {} 8 keys
          • platform: 'TOYOTA_COROLLA_TSS2'
          • eid: '3bec45a4-9512-4c7f-bdbc-9415e5890fb6/76'
          • y_range: [-5.0, 5.0]
          • vocab_size: 2048
          • temperature: 0.5
          • use_gt_features: false
          • platform_vec_size: 256
          • platform_map_path: '/code.nfs/branches/caches/workstation-harald_2025-07-25_08-15-05/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 11 keys
      • supercombo: 'f35eaf21-ef51-413d-9857-398bcdabb7da/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • use_allow_throttle: false
      • use_prev_desired_curv: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: true
      • use_raw_plan_long: true
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1587 segments || 1375.3841666666667 minutes

Metrics

1494/1587 aka 94.1398865784499%

better: 0/1587 aka 0.0%

best (0.5m): 0/1587 aka 0.0%

global mae x: 1.090

global mae y: 0.063

total # resets: 6969.000

reset per minute : 5.067

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.316 m/(ss)

Mean abs long acceleration GT: 0.369 m/(ss)

Mean abs long jerk: 0.966 m/(sss)

Mean abs long jerk GT: 0.546 m/(sss)

Easy straight (whole segments)

630 segments || 534.4641666666666 minutes

Metrics

626/630 aka 99.36507936507937%

better: 0/630 aka 0.0%

best (0.5m): 0/630 aka 0.0%

global mae x: 1.108

global mae y: 0.023

total # resets: 2484.000

reset per minute : 4.648

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.273 m/(ss)

Mean abs long acceleration GT: 0.282 m/(ss)

Mean abs long jerk: 0.774 m/(sss)

Mean abs long jerk GT: 0.445 m/(sss)

Hard acceleration

289 segments || 15.3425 minutes

Metrics

280/289 aka 96.88581314878893%

better: 45/289 aka 15.570934256055363%

best (0.5m): 19/289 aka 6.57439446366782%

global mae x: 1.067

global mae y: 0.166

total # resets: 167.000

reset per minute : 10.885

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.024 m/(ss)

Mean abs long acceleration GT: 2.022 m/(ss)

Mean abs long jerk: 2.049 m/(sss)

Mean abs long jerk GT: 1.390 m/(sss)

Hard deceleration

449 segments || 26.915833333333335 minutes

Metrics

442/449 aka 98.44097995545657%

better: 87/449 aka 19.37639198218263%

best (0.5m): 39/449 aka 8.68596881959911%

global mae x: 1.078

global mae y: 0.143

total # resets: 222.000

reset per minute : 8.248

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.365 m/(ss)

Mean abs long acceleration GT: 2.100 m/(ss)

Mean abs long jerk: 2.110 m/(sss)

Mean abs long jerk GT: 1.325 m/(sss)

No lead

494 segments || 208.74083333333334 minutes

Metrics

486/494 aka 98.38056680161944%

better: 21/494 aka 4.251012145748988%

best (0.5m): 14/494 aka 2.834008097165992%

global mae x: 1.066

global mae y: 0.065

total # resets: 1157.000

reset per minute : 5.543

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.291 m/(ss)

Mean abs long acceleration GT: 0.325 m/(ss)

Mean abs long jerk: 0.950 m/(sss)

Mean abs long jerk GT: 0.529 m/(sss)

Come to stop

517 segments || 105.42166666666667 minutes

Metrics

506/517 aka 97.87234042553192%

better: 66/517 aka 12.76595744680851%

best (0.5m): 18/517 aka 3.481624758220503%

global mae x: 1.043

global mae y: 0.122

total # resets: 283.000

reset per minute : 2.684

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.430 m/(ss)

Mean abs long acceleration GT: 0.498 m/(ss)

Mean abs long jerk: 0.818 m/(sss)

Mean abs long jerk GT: 0.487 m/(sss)

Start from stop

427 segments || 16.235 minutes

Metrics

421/427 aka 98.59484777517564%

better: 109/427 aka 25.526932084309134%

best (0.5m): 54/427 aka 12.646370023419204%

global mae x: 1.079

global mae y: 0.185

total # resets: 173.000

reset per minute : 10.656

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.054 m/(ss)

Mean abs long acceleration GT: 1.400 m/(ss)

Mean abs long jerk: 1.878 m/(sss)

Mean abs long jerk GT: 0.826 m/(sss)

Strong turns (>2 m/ss)

525 segments || 39.62083333333333 minutes

Metrics

494/525 aka 94.0952380952381%

better: 121/525 aka 23.047619047619047%

best (0.5m): 68/525 aka 12.952380952380953%

global mae x: 1.056

global mae y: 0.390

total # resets: 215.000

reset per minute : 5.426

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.462 m/(ss)

Mean abs long acceleration GT: 0.522 m/(ss)

Mean abs long jerk: 1.731 m/(sss)

Mean abs long jerk GT: 0.802 m/(sss)