Model analysis report for

▶
root: {} 10 keys
- report_name: 'mlsimdriving_driving.analyse_long_rl'
- route_list: '/home/batman/xx/routes/default_test_2k'
- output_dir: null
- save_tmp: false
- read_only: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'mlsimdriving_driving.analyse_long_rl'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-harald_2025-06-15_15-45-45'
  - force_local: false
- ▶
  rollout: {} 10 keys
  - type: 'TinySimRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: []
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 30 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - start: null
    - end: null
    - env: 'xx.arena.envs.tinysim_env.TinySimEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: false
    - fps: 20
    - allow_lat: false
    - allow_long: true
    - min_hist: 40
    - compressor_model: null
    - compressor_mode: 'NONE'
    - quantized_compressor_latents: false
    - yolo_model: false
    - lpips_model: false
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - max_x_error: 4.0
    - max_y_error: 3.0
    - max_yaw_error: 0.2
    - reset_every: null
    - noise_model: 'NONE'
    - photo_noise_model: 'NONE'
    - augment_mode: 'DEPTH'
    - inpaint_mode: 'NONE'
    - avoid_hood: false
    - cache_granularity: true
    - worldmodel: null
  - ▶
    agent: {} 12 keys
    - supercombo: 'f7db6a09-43c5-4db9-b856-7fe1a1c231eb/400'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - calibrate_speed: false
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: true
    - use_raw_plan_long: true
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

Plan analysis

All segments (whole segments)

1656 segments || 1435.1841666666667 minutes

Metrics

1550/1656 aka 93.59903381642512%

better: 0/1656 aka 0.0%

best (0.5m): 0/1656 aka 0.0%

global mae x: 1.121

global mae y: 0.064

total # resets: 6557.000

reset per minute : 4.569

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.297 m/(ss)

Mean abs long acceleration GT: 0.371 m/(ss)

Mean abs long jerk: 1.024 m/(sss)

Mean abs long jerk GT: 0.548 m/(sss)

Easy straight (whole segments)

660 segments || 559.7199999999999 minutes

Metrics

657/660 aka 99.54545454545455%

better: 0/660 aka 0.0%

best (0.5m): 0/660 aka 0.0%

global mae x: 1.146

global mae y: 0.023

total # resets: 2217.000

reset per minute : 3.961

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.248 m/(ss)

Mean abs long acceleration GT: 0.286 m/(ss)

Mean abs long jerk: 0.879 m/(sss)

Mean abs long jerk GT: 0.447 m/(sss)

Hard acceleration

303 segments || 15.9025 minutes

Metrics

296/303 aka 97.6897689768977%

better: 49/303 aka 16.17161716171617%

best (0.5m): 24/303 aka 7.920792079207921%

global mae x: 1.112

global mae y: 0.185

total # resets: 155.000

reset per minute : 9.747

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.046 m/(ss)

Mean abs long acceleration GT: 2.017 m/(ss)

Mean abs long jerk: 1.883 m/(sss)

Mean abs long jerk GT: 1.398 m/(sss)

Hard deceleration

478 segments || 28.785833333333336 minutes

Metrics

468/478 aka 97.90794979079497%

better: 74/478 aka 15.481171548117155%

best (0.5m): 32/478 aka 6.694560669456067%

global mae x: 1.108

global mae y: 0.137

total # resets: 238.000

reset per minute : 8.268

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.405 m/(ss)

Mean abs long acceleration GT: 2.107 m/(ss)

Mean abs long jerk: 1.862 m/(sss)

Mean abs long jerk GT: 1.326 m/(sss)

No lead

502 segments || 205.00750000000002 minutes

Metrics

482/502 aka 96.01593625498008%

better: 30/502 aka 5.9760956175298805%

best (0.5m): 20/502 aka 3.9840637450199203%

global mae x: 1.091

global mae y: 0.069

total # resets: 1029.000

reset per minute : 5.019

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.243 m/(ss)

Mean abs long acceleration GT: 0.312 m/(ss)

Mean abs long jerk: 0.998 m/(sss)

Mean abs long jerk GT: 0.536 m/(sss)

Come to stop

544 segments || 109.86916666666666 minutes

Metrics

531/544 aka 97.61029411764706%

better: 60/544 aka 11.029411764705882%

best (0.5m): 17/544 aka 3.125%

global mae x: 1.099

global mae y: 0.118

total # resets: 318.000

reset per minute : 2.894

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.441 m/(ss)

Mean abs long acceleration GT: 0.504 m/(ss)

Mean abs long jerk: 0.841 m/(sss)

Mean abs long jerk GT: 0.494 m/(sss)

Start from stop

450 segments || 17.1825 minutes

Metrics

444/450 aka 98.66666666666667%

better: 105/450 aka 23.333333333333332%

best (0.5m): 50/450 aka 11.11111111111111%

global mae x: 1.118

global mae y: 0.207

total # resets: 156.000

reset per minute : 9.079

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 1.113 m/(ss)

Mean abs long acceleration GT: 1.392 m/(ss)

Mean abs long jerk: 1.645 m/(sss)

Mean abs long jerk GT: 0.832 m/(sss)

Strong turns (>2 m/ss)

548 segments || 41.19916666666666 minutes

Metrics

519/548 aka 94.7080291970803%

better: 125/548 aka 22.81021897810219%

best (0.5m): 59/548 aka 10.766423357664234%

global mae x: 1.066

global mae y: 0.414

total # resets: 179.000

reset per minute : 4.345

Mean model 0 path x errs: 0.000 m

Mean abs long acceleration: 0.395 m/(ss)

Mean abs long acceleration GT: 0.518 m/(ss)

Mean abs long jerk: 1.311 m/(sss)

Mean abs long jerk GT: 0.799 m/(sss)