Model analysis report for

▶
root: {} 11 keys
- report_name: 'rldriving_analyselong'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- shuffle: false
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 1000
  - job_name: 'rldriving_analyselong'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-harald_2025-04-29_11-17-03'
  - force_local: false
- route_list: '/home/batman/xx/routes/default_test_2k'
- ▶
  rollout: {} 10 keys
  - type: 'TinySimRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 30 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.tinysim_env.TinySimEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: false
    - fps: 20
    - allow_lat: false
    - allow_long: true
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: false
    - future_observation_range_seconds: null
    - max_x_error: 4.0
    - max_y_error: 3.0
    - max_yaw_error: 0.2
    - reset_every: null
    - noise_model: 'NONE'
    - photo_noise_model: 'NONE'
    - augment_mode: 'DEPTH'
    - inpaint_mode: 'NONE'
    - avoid_hood: false
    - cache_granularity: true
    - worldmodel: null
  - ▶
    agent: {} 10 keys
    - supercombo: '8d0a1b3b-9972-4e53-b9c5-3e13e5e3e404/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: true
    - use_raw_plan_long: true
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1655 segments || 1434.3175 minutes

Metrics

1528/1655 aka 92.3262839879154%

better: 0/1655 aka 0.0%

best (0.5m): 0/1655 aka 0.0%

global mae x: 1.099

global mae y: 0.065

total # resets: 7356.000

reset per minute : 5.129

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.309 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 0.973 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

658 segments || 558.0433333333333 minutes

Metrics

655/658 aka 99.54407294832826%

better: 0/658 aka 0.0%

best (0.5m): 0/658 aka 0.0%

global mae x: 1.133

global mae y: 0.024

total # resets: 2653.000

reset per minute : 4.754

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.265 m/(ss)

Mean abs long acceleration GT: 0.285 m/(ss)

Mean abs long jerk: 0.812 m/(sss)

Mean abs long jerk GT: 0.446 m/(sss)

Hard acceleration

306 segments || 16.1375 minutes

Metrics

298/306 aka 97.38562091503267%

better: 46/306 aka 15.032679738562091%

best (0.5m): 28/306 aka 9.15032679738562%

global mae x: 1.053

global mae y: 0.179

total # resets: 191.000

reset per minute : 11.836

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.021 m/(ss)

Mean abs long acceleration GT: 2.013 m/(ss)

Mean abs long jerk: 1.899 m/(sss)

Mean abs long jerk GT: 1.403 m/(sss)

Hard deceleration

476 segments || 28.645833333333332 minutes

Metrics

468/476 aka 98.31932773109244%

better: 94/476 aka 19.747899159663866%

best (0.5m): 52/476 aka 10.92436974789916%

global mae x: 1.118

global mae y: 0.141

total # resets: 234.000

reset per minute : 8.169

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.421 m/(ss)

Mean abs long acceleration GT: 2.107 m/(ss)

Mean abs long jerk: 1.897 m/(sss)

Mean abs long jerk GT: 1.328 m/(sss)

No lead

502 segments || 204.61833333333334 minutes

Metrics

475/502 aka 94.62151394422311%

better: 28/502 aka 5.577689243027889%

best (0.5m): 18/502 aka 3.585657370517928%

global mae x: 1.054

global mae y: 0.068

total # resets: 1155.000

reset per minute : 5.645

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.252 m/(ss)

Mean abs long acceleration GT: 0.312 m/(ss)

Mean abs long jerk: 0.919 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

544 segments || 109.60083333333334 minutes

Metrics

532/544 aka 97.79411764705883%

better: 59/544 aka 10.845588235294118%

best (0.5m): 23/544 aka 4.227941176470588%

global mae x: 0.992

global mae y: 0.125

total # resets: 307.000

reset per minute : 2.801

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.418 m/(ss)

Mean abs long acceleration GT: 0.505 m/(ss)

Mean abs long jerk: 0.822 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

450 segments || 17.1825 minutes

Metrics

445/450 aka 98.88888888888889%

better: 114/450 aka 25.333333333333332%

best (0.5m): 75/450 aka 16.666666666666668%

global mae x: 1.029

global mae y: 0.189

total # resets: 209.000

reset per minute : 12.164

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.123 m/(ss)

Mean abs long acceleration GT: 1.389 m/(ss)

Mean abs long jerk: 1.851 m/(sss)

Mean abs long jerk GT: 0.836 m/(sss)

Strong turns (>2 m/ss)

550 segments || 41.17583333333334 minutes

Metrics

522/550 aka 94.9090909090909%

better: 140/550 aka 25.454545454545453%

best (0.5m): 67/550 aka 12.181818181818182%

global mae x: 1.045

global mae y: 0.416

total # resets: 203.000

reset per minute : 4.930

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.420 m/(ss)

Mean abs long acceleration GT: 0.520 m/(ss)

Mean abs long jerk: 1.542 m/(sss)

Mean abs long jerk GT: 0.796 m/(sss)