Model analysis report for

• ▶
  root: {} 10 keys
  • report_name: 'mlsimdriving_driving.analyse_long_rl'
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'mlsimdriving_driving.analyse_long_rl'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-yassine_2025-07-06_10-53-22'
    • force_local: false
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 32 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_decoder_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • ▶
        physics: {} 4 keys
        • type: 'BICYCLE'
        • ▶
          bicycle_model_config: {} 1 keys
          • vehicle_platform: 'HONDA_CIVIC'
        • ▶
          classic_model_config: {} 1 keys
          • platform: 'COROLLA'
        • ▶
          ml_model_config: {} 6 keys
          • platform: 'COROLLA'
          • eid: 'd3e5ef06-14ec-4e35-b708-d8db73d29f1e/51'
          • y_range: [-5.0, 5.0]
          • vocab_size: 2048
          • temperature: 0.5
          • use_gt_features: false
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 12 keys
      • supercombo: 'fef4d896-597e-4c6f-90cf-8eb7102c89d2/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: true
      • use_raw_plan_long: true
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1644 segments || 1424.7841666666668 minutes

Metrics

1523/1644 aka 92.63990267639903%

better: 0/1644 aka 0.0%

best (0.5m): 0/1644 aka 0.0%

global mae x: 1.139

global mae y: 0.064

total # resets: 7788.000

reset per minute : 5.466

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.377 m/(ss)

Mean abs long acceleration GT: 0.371 m/(ss)

Mean abs long jerk: 1.488 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

651 segments || 552.2491666666666 minutes

Metrics

640/651 aka 98.31029185867895%

better: 0/651 aka 0.0%

best (0.5m): 0/651 aka 0.0%

global mae x: 1.173

global mae y: 0.023

total # resets: 2781.000

reset per minute : 5.036

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.319 m/(ss)

Mean abs long acceleration GT: 0.284 m/(ss)

Mean abs long jerk: 1.265 m/(sss)

Mean abs long jerk GT: 0.446 m/(sss)

Hard acceleration

303 segments || 16.093333333333334 minutes

Metrics

296/303 aka 97.6897689768977%

better: 64/303 aka 21.122112211221122%

best (0.5m): 26/303 aka 8.58085808580858%

global mae x: 1.098

global mae y: 0.168

total # resets: 140.000

reset per minute : 8.699

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.204 m/(ss)

Mean abs long acceleration GT: 2.018 m/(ss)

Mean abs long jerk: 2.305 m/(sss)

Mean abs long jerk GT: 1.395 m/(sss)

Hard deceleration

469 segments || 28.146666666666665 minutes

Metrics

461/469 aka 98.29424307036247%

better: 102/469 aka 21.748400852878465%

best (0.5m): 50/469 aka 10.660980810234541%

global mae x: 1.111

global mae y: 0.141

total # resets: 188.000

reset per minute : 6.679

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.432 m/(ss)

Mean abs long acceleration GT: 2.106 m/(ss)

Mean abs long jerk: 2.307 m/(sss)

Mean abs long jerk GT: 1.332 m/(sss)

No lead

513 segments || 215.03166666666667 minutes

Metrics

486/513 aka 94.73684210526316%

better: 21/513 aka 4.093567251461988%

best (0.5m): 14/513 aka 2.729044834307992%

global mae x: 1.095

global mae y: 0.073

total # resets: 1358.000

reset per minute : 6.315

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.358 m/(ss)

Mean abs long acceleration GT: 0.326 m/(ss)

Mean abs long jerk: 1.549 m/(sss)

Mean abs long jerk GT: 0.533 m/(sss)

Come to stop

538 segments || 108.405 minutes

Metrics

522/538 aka 97.02602230483271%

better: 47/538 aka 8.736059479553903%

best (0.5m): 15/538 aka 2.7881040892193307%

global mae x: 1.056

global mae y: 0.098

total # resets: 459.000

reset per minute : 4.234

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.431 m/(ss)

Mean abs long acceleration GT: 0.506 m/(ss)

Mean abs long jerk: 0.948 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

445 segments || 17.00166666666667 minutes

Metrics

440/445 aka 98.87640449438203%

better: 130/445 aka 29.213483146067414%

best (0.5m): 68/445 aka 15.280898876404494%

global mae x: 1.043

global mae y: 0.192

total # resets: 118.000

reset per minute : 6.940

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.149 m/(ss)

Mean abs long acceleration GT: 1.395 m/(ss)

Mean abs long jerk: 1.983 m/(sss)

Mean abs long jerk GT: 0.830 m/(sss)

Strong turns (>2 m/ss)

547 segments || 41.024166666666666 minutes

Metrics

523/547 aka 95.61243144424131%

better: 115/547 aka 21.02376599634369%

best (0.5m): 56/547 aka 10.23765996343693%

global mae x: 1.145

global mae y: 0.443

total # resets: 211.000

reset per minute : 5.143

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.551 m/(ss)

Mean abs long acceleration GT: 0.521 m/(ss)

Mean abs long jerk: 2.249 m/(sss)

Mean abs long jerk GT: 0.799 m/(sss)