Model analysis report for

▶
root: {} 11 keys
- report_name: 'rldriving_analyselong'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- shuffle: false
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 1000
  - job_name: 'rldriving_analyselong'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-harald_2025-04-28_22-59-34'
  - force_local: false
- route_list: '/home/batman/xx/routes/default_test_2k'
- ▶
  rollout: {} 10 keys
  - type: 'TinySimRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 30 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.tinysim_env.TinySimEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: false
    - fps: 20
    - allow_lat: false
    - allow_long: true
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: false
    - future_observation_range_seconds: null
    - max_x_error: 4.0
    - max_y_error: 3.0
    - max_yaw_error: 0.2
    - reset_every: null
    - noise_model: 'NONE'
    - photo_noise_model: 'NONE'
    - augment_mode: 'DEPTH'
    - inpaint_mode: 'NONE'
    - avoid_hood: false
    - cache_granularity: true
    - worldmodel: null
  - ▶
    agent: {} 10 keys
    - supercombo: '154640c6-9151-44f9-bd44-c3d39aaf52d7/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - use_raw_plan_long: true
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1657 segments || 1436.0508333333335 minutes

Metrics

1589/1657 aka 95.89619794809897%

better: 0/1657 aka 0.0%

best (0.5m): 0/1657 aka 0.0%

global mae x: 1.107

global mae y: 0.066

total # resets: 6293.000

reset per minute : 4.382

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.429 m/(ss)

Mean abs long acceleration GT: 0.372 m/(ss)

Mean abs long jerk: 1.439 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

660 segments || 559.7199999999999 minutes

Metrics

656/660 aka 99.39393939393939%

better: 0/660 aka 0.0%

best (0.5m): 0/660 aka 0.0%

global mae x: 1.100

global mae y: 0.023

total # resets: 2342.000

reset per minute : 4.184

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.356 m/(ss)

Mean abs long acceleration GT: 0.286 m/(ss)

Mean abs long jerk: 1.245 m/(sss)

Mean abs long jerk GT: 0.447 m/(sss)

Hard acceleration

303 segments || 16.065833333333334 minutes

Metrics

298/303 aka 98.34983498349835%

better: 47/303 aka 15.511551155115512%

best (0.5m): 18/303 aka 5.9405940594059405%

global mae x: 1.106

global mae y: 0.178

total # resets: 160.000

reset per minute : 9.959

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.453 m/(ss)

Mean abs long acceleration GT: 2.015 m/(ss)

Mean abs long jerk: 2.574 m/(sss)

Mean abs long jerk GT: 1.400 m/(sss)

Hard deceleration

477 segments || 28.738333333333333 minutes

Metrics

470/477 aka 98.53249475890985%

better: 86/477 aka 18.029350104821802%

best (0.5m): 44/477 aka 9.224318658280922%

global mae x: 1.104

global mae y: 0.147

total # resets: 184.000

reset per minute : 6.403

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.566 m/(ss)

Mean abs long acceleration GT: 2.106 m/(ss)

Mean abs long jerk: 2.490 m/(sss)

Mean abs long jerk GT: 1.327 m/(sss)

No lead

502 segments || 204.61833333333334 minutes

Metrics

485/502 aka 96.61354581673307%

better: 28/502 aka 5.577689243027889%

best (0.5m): 11/502 aka 2.191235059760956%

global mae x: 1.074

global mae y: 0.069

total # resets: 1028.000

reset per minute : 5.024

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.376 m/(ss)

Mean abs long acceleration GT: 0.313 m/(ss)

Mean abs long jerk: 1.305 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

546 segments || 110.31333333333333 minutes

Metrics

539/546 aka 98.71794871794872%

better: 38/546 aka 6.95970695970696%

best (0.5m): 8/546 aka 1.465201465201465%

global mae x: 0.961

global mae y: 0.129

total # resets: 308.000

reset per minute : 2.792

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.493 m/(ss)

Mean abs long acceleration GT: 0.505 m/(ss)

Mean abs long jerk: 1.256 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

452 segments || 17.249166666666667 minutes

Metrics

448/452 aka 99.11504424778761%

better: 107/452 aka 23.672566371681416%

best (0.5m): 55/452 aka 12.168141592920353%

global mae x: 1.023

global mae y: 0.193

total # resets: 207.000

reset per minute : 12.001

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.442 m/(ss)

Mean abs long acceleration GT: 1.389 m/(ss)

Mean abs long jerk: 2.695 m/(sss)

Mean abs long jerk GT: 0.833 m/(sss)

Strong turns (>2 m/ss)

549 segments || 41.18666666666666 minutes

Metrics

527/549 aka 95.99271402550092%

better: 136/549 aka 24.772313296903462%

best (0.5m): 62/549 aka 11.293260473588342%

global mae x: 1.116

global mae y: 0.441

total # resets: 151.000

reset per minute : 3.666

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.586 m/(ss)

Mean abs long acceleration GT: 0.523 m/(ss)

Mean abs long jerk: 1.923 m/(sss)

Mean abs long jerk GT: 0.798 m/(sss)