Model analysis report for

• ▶
  root: {} 12 keys
  • report_name: 'rldriving_driving.analyse_disagreements'
  • route_list: '/home/batman/xx/routes/default_test_2k'
  • output_dir: null
  • save_tmp: false
  • read_only: false
  • limit: 10000
  • use_virtual_webgl: false
  • format: 'HTML'
  • ▶
    miniray: {} 8 keys
    • priority: 10
    • job_name: 'rldriving_driving.analyse_disagreements'
    • queue_name: 'remote2'
    • submit_host: 'redis.comma.internal'
    • result_host: 'data-ram.comma.internal'
    • no_unique_id: false
    • codedir: '/code.nfs/xx'
    • force_local: false
  • codedir2: '/code.nfs/xx'
  • ▶
    rollout: {} 10 keys
    • type: 'RawVideoRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 20 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: true
      • fps: 20
      • allow_lat: true
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • with_gt: false
    • ▶
      agent: {} 12 keys
      • supercombo: '6a8a3da8-c264-4f91-b0a6-d04722cccfce/700'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: true
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: false
      • use_raw_plan_long: false
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys
  • ▶
    rollout2: {} 10 keys
    • type: 'RawVideoRL'
    • nowindow: false
    • debug: false
    • debug_render: false
    • save_observation_keys: []
    • output_dir: null
    • rerun: false
    • ▶
      env: {} 20 keys
      • debug_window_size: [512, 512]
      • ▶
        ep: {} 7 keys
        • segment: ''
        • augment_desire: false
        • depth_type: 'EASYDEPTH'
        • calibration_source: 'LOG'
        • pipeline_dir: 'http://data-ssd.comma.life/runner/training_2024_06'
        • feature_dir: null
        • ▶
          input_paths: {} 0 keys
      • start: null
      • end: null
      • env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
      • get_imgs: true
      • allow_resets: true
      • zero_desire: true
      • fps: 20
      • allow_lat: true
      • allow_long: true
      • min_hist: 40
      • compressor_model: null
      • compressor_mode: 'NONE'
      • quantized_compressor_latents: false
      • yolo_model: false
      • lpips_model: false
      • sharpness_metric: true
      • future_observation_range_seconds: null
      • with_gt: false
    • ▶
      agent: {} 12 keys
      • supercombo: 'default_supercombo'
      • restrict_outputs: null
      • exclude_outputs: ['imgs']
      • calibrate_speed: false
      • use_allow_throttle: false
      • use_prev_desired_curv: true
      • model_trained_fps: 5
      • use_worldmodel_as_agent: false
      • use_raw_plan_lat: false
      • use_raw_plan_long: false
      • desired_curv_smooth_seconds: false
      • desired_accel_smooth_seconds: false
    • ▶
      shadow_agent: {} 0 keys

total 1658 segments

Easy straights lane positioning (>20mph)

- 43 chunks (8 seconds each)

--- (GT) avg = 0.153(m) from lane center

--- (6a8a3da8) avg = -0.0000209(1/m) desired curvature

--- (default_supercombo) avg = -0.0000593(1/m) desired curvature

examples with (6a8a3da8) more steer:

examples with (default_supercombo) more steer:

\

Turns lane positioning

- 40 chunks (8 seconds each)

--- (GT) avg = 0.407(m) from lane center

--- (6a8a3da8) avg = 0.0018472(1/m) desired curvature

--- (default_supercombo) avg = 0.0017691(1/m) desired curvature

examples with (6a8a3da8) more steer:

examples with (default_supercombo) more steer:

\

Highway straights speed convergence (50-80mph)

- 27 chunks (8 seconds each)

--- (6a8a3da8) avg = 0.0113057(m/s^2) vs GT

--- (default_supercombo) avg = 0.0076334(m/s^2) vs GT

examples with (6a8a3da8) faster:

examples with (default_supercombo) faster:

\

City straights speed convergence (20-40mph)

- 31 chunks (8 seconds each)

--- (6a8a3da8) avg = -0.0535332(m/s^2) vs GT

--- (default_supercombo) avg = -0.0598615(m/s^2) vs GT

examples with (6a8a3da8) faster:

examples with (default_supercombo) faster:

\

Slowdown for turns (decel performance)

- 56 chunks (8 seconds each)

--- (6a8a3da8) avg = -0.0666536(m/s^2) vs GT

--- (default_supercombo) avg = -0.0797893(m/s^2) vs GT

examples with (6a8a3da8) less brake:

examples with (default_supercombo) less brake:

\

Come to stop (decel performance)

- 117 chunks (8 seconds each)

--- (6a8a3da8) avg = 0.1208521(m/s^2) vs GT

--- (default_supercombo) avg = 0.0955720(m/s^2) vs GT

examples with (6a8a3da8) less brake:

examples with (default_supercombo) less brake:

\

Start from stop (accel performance)

- 125 chunks (8 seconds each)

--- (6a8a3da8) avg = -0.0293432(m/s^2) vs GT

--- (default_supercombo) avg = -0.0378940(m/s^2) vs GT

examples with (6a8a3da8) more accel:

examples with (default_supercombo) more accel:

\