Model analysis report for

• ▶
  root: {} 11 keys
  • report_name: 'rldriving_analyselong'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • limit: 10000
  • shuffle: false
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 1000
    • job_name: 'rldriving_analyselong'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: 'caches/workstation-harald_2025-04-25_17-31-30'
    • force_local: false
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • ▶
    rollout: {} 10 keys
    • type: 'TinySimRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 29 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.tinysim_env.TinySimEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: false
      • fps: 20
      • allow_lat: false
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • future_observation_range_seconds: null
      • max_x_error: 4.0
      • max_y_error: 3.0
      • max_yaw_error: 0.2
      • reset_every: null
      • noise_model: 'NONE'
      • photo_noise_model: 'NONE'
      • augment_mode: 'DEPTH'
      • inpaint_mode: 'NONE'
      • avoid_hood: false
      • cache_granularity: true
      • worldmodel: null
    • ▶
      agent: {} 10 keys
      • supercombo: '846c53a4-b886-4942-a4d3-7ea3aff5be8d/400'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: false
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: false
      • use_raw_plan_long: true
    • ▶
      shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1655 segments || 1434.3175 minutes

Metrics

1526/1655 aka 92.20543806646526%

better: 0/1655 aka 0.0%

best (0.5m): 0/1655 aka 0.0%

global mae x: 1.085

global mae y: 0.065

total # resets: 6467.000

reset per minute : 4.509

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.301 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 0.819 m/(sss)

Mean abs long jerk GT: 0.547 m/(sss)

Easy straight (whole segments)

658 segments || 558.1550000000001 minutes

Metrics

654/658 aka 99.3920972644377%

better: 0/658 aka 0.0%

best (0.5m): 0/658 aka 0.0%

global mae x: 1.112

global mae y: 0.023

total # resets: 2149.000

reset per minute : 3.850

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.243 m/(ss)

Mean abs long acceleration GT: 0.285 m/(ss)

Mean abs long jerk: 0.651 m/(sss)

Mean abs long jerk GT: 0.446 m/(sss)

Hard acceleration

303 segments || 16.084166666666665 minutes

Metrics

292/303 aka 96.36963696369637%

better: 44/303 aka 14.521452145214521%

best (0.5m): 26/303 aka 8.58085808580858%

global mae x: 1.094

global mae y: 0.185

total # resets: 150.000

reset per minute : 9.326

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.321 m/(ss)

Mean abs long acceleration GT: 2.017 m/(ss)

Mean abs long jerk: 1.788 m/(sss)

Mean abs long jerk GT: 1.396 m/(sss)

Hard deceleration

476 segments || 28.68 minutes

Metrics

468/476 aka 98.31932773109244%

better: 101/476 aka 21.218487394957982%

best (0.5m): 61/476 aka 12.815126050420169%

global mae x: 1.097

global mae y: 0.144

total # resets: 207.000

reset per minute : 7.218

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.482 m/(ss)

Mean abs long acceleration GT: 2.106 m/(ss)

Mean abs long jerk: 1.579 m/(sss)

Mean abs long jerk GT: 1.328 m/(sss)

No lead

501 segments || 204.54333333333335 minutes

Metrics

474/501 aka 94.61077844311377%

better: 27/501 aka 5.389221556886228%

best (0.5m): 17/501 aka 3.3932135728542914%

global mae x: 1.083

global mae y: 0.068

total # resets: 942.000

reset per minute : 4.605

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.252 m/(ss)

Mean abs long acceleration GT: 0.313 m/(ss)

Mean abs long jerk: 0.807 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

544 segments || 109.71499999999999 minutes

Metrics

538/544 aka 98.8970588235294%

better: 104/544 aka 19.11764705882353%

best (0.5m): 40/544 aka 7.352941176470588%

global mae x: 0.971

global mae y: 0.125

total # resets: 245.000

reset per minute : 2.233

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.450 m/(ss)

Mean abs long acceleration GT: 0.505 m/(ss)

Mean abs long jerk: 0.668 m/(sss)

Mean abs long jerk GT: 0.493 m/(sss)

Start from stop

450 segments || 17.1825 minutes

Metrics

446/450 aka 99.11111111111111%

better: 101/450 aka 22.444444444444443%

best (0.5m): 63/450 aka 14.0%

global mae x: 1.245

global mae y: 0.236

total # resets: 153.000

reset per minute : 8.904

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.225 m/(ss)

Mean abs long acceleration GT: 1.389 m/(ss)

Mean abs long jerk: 1.525 m/(sss)

Mean abs long jerk GT: 0.834 m/(sss)

Strong turns (>2 m/ss)

548 segments || 41.15916666666667 minutes

Metrics

510/548 aka 93.06569343065694%

better: 129/548 aka 23.54014598540146%

best (0.5m): 60/548 aka 10.94890510948905%

global mae x: 1.074

global mae y: 0.426

total # resets: 243.000

reset per minute : 5.904

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.365 m/(ss)

Mean abs long acceleration GT: 0.523 m/(ss)

Mean abs long jerk: 1.408 m/(sss)

Mean abs long jerk GT: 0.799 m/(sss)