Model analysis report for

▶
root: {} 11 keys
- report_name: 'rldriving_analyselong'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- shuffle: false
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 1000
  - job_name: 'rldriving_analyselong'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-05-01_09-40-16'
  - force_local: false
- route_list: '/home/batman/xx/routes/default_test_2k'
- ▶
  rollout: {} 10 keys
  - type: 'TinySimRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 30 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.tinysim_env.TinySimEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: false
    - fps: 20
    - allow_lat: false
    - allow_long: true
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: false
    - future_observation_range_seconds: null
    - max_x_error: 4.0
    - max_y_error: 3.0
    - max_yaw_error: 0.2
    - reset_every: null
    - noise_model: 'NONE'
    - photo_noise_model: 'NONE'
    - augment_mode: 'DEPTH'
    - inpaint_mode: 'NONE'
    - avoid_hood: false
    - cache_granularity: true
    - worldmodel: null
  - ▶
    agent: {} 10 keys
    - supercombo: 'e9237324-4b92-48f5-acaa-ebdf7fe46339/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - use_raw_plan_long: true
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1657 segments || 1436.0508333333335 minutes

Metrics

1529/1657 aka 92.27519613759807%

better: 0/1657 aka 0.0%

best (0.5m): 0/1657 aka 0.0%

global mae x: 1.160

global mae y: 0.066

total # resets: 7309.000

reset per minute : 5.090

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.356 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 1.190 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

660 segments || 559.7199999999999 minutes

Metrics

656/660 aka 99.39393939393939%

better: 0/660 aka 0.0%

best (0.5m): 0/660 aka 0.0%

global mae x: 1.197

global mae y: 0.024

total # resets: 2580.000

reset per minute : 4.609

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.293 m/(ss)

Mean abs long acceleration GT: 0.286 m/(ss)

Mean abs long jerk: 1.008 m/(sss)

Mean abs long jerk GT: 0.447 m/(sss)

Hard acceleration

305 segments || 16.138333333333332 minutes

Metrics

297/305 aka 97.37704918032787%

better: 59/305 aka 19.34426229508197%

best (0.5m): 27/305 aka 8.852459016393443%

global mae x: 1.108

global mae y: 0.183

total # resets: 139.000

reset per minute : 8.613

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.280 m/(ss)

Mean abs long acceleration GT: 2.018 m/(ss)

Mean abs long jerk: 2.010 m/(sss)

Mean abs long jerk GT: 1.390 m/(sss)

Hard deceleration

478 segments || 28.73 minutes

Metrics

468/478 aka 97.90794979079497%

better: 82/478 aka 17.15481171548117%

best (0.5m): 46/478 aka 9.623430962343097%

global mae x: 1.154

global mae y: 0.137

total # resets: 187.000

reset per minute : 6.509

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.449 m/(ss)

Mean abs long acceleration GT: 2.107 m/(ss)

Mean abs long jerk: 1.873 m/(sss)

Mean abs long jerk GT: 1.326 m/(sss)

No lead

502 segments || 204.61833333333334 minutes

Metrics

486/502 aka 96.81274900398407%

better: 19/502 aka 3.7848605577689245%

best (0.5m): 12/502 aka 2.3904382470119523%

global mae x: 1.146

global mae y: 0.071

total # resets: 1166.000

reset per minute : 5.698

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.310 m/(ss)

Mean abs long acceleration GT: 0.312 m/(ss)

Mean abs long jerk: 1.180 m/(sss)

Mean abs long jerk GT: 0.537 m/(sss)

Come to stop

546 segments || 110.31333333333333 minutes

Metrics

535/546 aka 97.98534798534799%

better: 62/546 aka 11.355311355311356%

best (0.5m): 17/546 aka 3.1135531135531136%

global mae x: 1.095

global mae y: 0.110

total # resets: 317.000

reset per minute : 2.874

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.435 m/(ss)

Mean abs long acceleration GT: 0.505 m/(ss)

Mean abs long jerk: 0.810 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

452 segments || 17.249166666666667 minutes

Metrics

450/452 aka 99.5575221238938%

better: 138/452 aka 30.530973451327434%

best (0.5m): 69/452 aka 15.265486725663717%

global mae x: 1.090

global mae y: 0.201

total # resets: 130.000

reset per minute : 7.537

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.178 m/(ss)

Mean abs long acceleration GT: 1.393 m/(ss)

Mean abs long jerk: 1.797 m/(sss)

Mean abs long jerk GT: 0.832 m/(sss)

Strong turns (>2 m/ss)

550 segments || 41.19416666666667 minutes

Metrics

519/550 aka 94.36363636363636%

better: 122/550 aka 22.181818181818183%

best (0.5m): 66/550 aka 12.0%

global mae x: 1.113

global mae y: 0.432

total # resets: 197.000

reset per minute : 4.782

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.504 m/(ss)

Mean abs long acceleration GT: 0.524 m/(ss)

Mean abs long jerk: 1.788 m/(sss)

Mean abs long jerk GT: 0.800 m/(sss)