Model analysis report for

• ▶
  root: {} 11 keys
  • report_name: 'rldriving_analyselong'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • limit: 10000
  • shuffle: false
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 1000
    • job_name: 'rldriving_analyselong'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-yassine_2025-05-11_17-17-52'
    • force_local: false
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 30 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 10 keys
      • supercombo: '5e4cb3d3-b9cc-45c7-a476-38083e75029c/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: true
      • use_raw_plan_long: true
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1656 segments || 1435.1841666666667 minutes

Metrics

1566/1656 aka 94.56521739130434%

better: 0/1656 aka 0.0%

best (0.5m): 0/1656 aka 0.0%

global mae x: 1.139

global mae y: 0.066

total # resets: 6638.000

reset per minute : 4.625

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.313 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 1.056 m/(sss)

Mean abs long jerk GT: 0.547 m/(sss)

Easy straight (whole segments)

660 segments || 559.7199999999999 minutes

Metrics

656/660 aka 99.39393939393939%

better: 0/660 aka 0.0%

best (0.5m): 0/660 aka 0.0%

global mae x: 1.173

global mae y: 0.024

total # resets: 2308.000

reset per minute : 4.123

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.271 m/(ss)

Mean abs long acceleration GT: 0.286 m/(ss)

Mean abs long jerk: 0.933 m/(sss)

Mean abs long jerk GT: 0.447 m/(sss)

Hard acceleration

307 segments || 16.19 minutes

Metrics

299/307 aka 97.39413680781759%

better: 51/307 aka 16.612377850162865%

best (0.5m): 29/307 aka 9.446254071661238%

global mae x: 1.049

global mae y: 0.178

total # resets: 161.000

reset per minute : 9.944

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.072 m/(ss)

Mean abs long acceleration GT: 2.016 m/(ss)

Mean abs long jerk: 1.800 m/(sss)

Mean abs long jerk GT: 1.391 m/(sss)

Hard deceleration

476 segments || 28.600833333333334 minutes

Metrics

473/476 aka 99.36974789915966%

better: 93/476 aka 19.537815126050422%

best (0.5m): 36/476 aka 7.563025210084033%

global mae x: 1.093

global mae y: 0.138

total # resets: 221.000

reset per minute : 7.727

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.385 m/(ss)

Mean abs long acceleration GT: 2.108 m/(ss)

Mean abs long jerk: 1.926 m/(sss)

Mean abs long jerk GT: 1.323 m/(sss)

No lead

501 segments || 204.3625 minutes

Metrics

486/501 aka 97.0059880239521%

better: 28/501 aka 5.588822355289421%

best (0.5m): 16/501 aka 3.193612774451098%

global mae x: 1.136

global mae y: 0.069

total # resets: 984.000

reset per minute : 4.815

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.258 m/(ss)

Mean abs long acceleration GT: 0.312 m/(ss)

Mean abs long jerk: 1.022 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

545 segments || 110.28 minutes

Metrics

538/545 aka 98.71559633027523%

better: 54/545 aka 9.908256880733944%

best (0.5m): 19/545 aka 3.4862385321100917%

global mae x: 1.020

global mae y: 0.121

total # resets: 302.000

reset per minute : 2.738

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.440 m/(ss)

Mean abs long acceleration GT: 0.504 m/(ss)

Mean abs long jerk: 0.850 m/(sss)

Mean abs long jerk GT: 0.493 m/(sss)

Start from stop

451 segments || 17.215833333333332 minutes

Metrics

445/451 aka 98.66962305986696%

better: 105/451 aka 23.28159645232816%

best (0.5m): 58/451 aka 12.86031042128603%

global mae x: 1.099

global mae y: 0.217

total # resets: 182.000

reset per minute : 10.572

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.159 m/(ss)

Mean abs long acceleration GT: 1.393 m/(ss)

Mean abs long jerk: 1.576 m/(sss)

Mean abs long jerk GT: 0.832 m/(sss)

Strong turns (>2 m/ss)

547 segments || 41.125 minutes

Metrics

523/547 aka 95.61243144424131%

better: 145/547 aka 26.508226691042047%

best (0.5m): 74/547 aka 13.528336380255942%

global mae x: 1.062

global mae y: 0.419

total # resets: 154.000

reset per minute : 3.745

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.441 m/(ss)

Mean abs long acceleration GT: 0.521 m/(ss)

Mean abs long jerk: 1.397 m/(sss)

Mean abs long jerk GT: 0.793 m/(sss)