Model analysis report for

▶
root: {} 16 keys
- report_name: 'kl_compressor_worldmodel.analyse_vae'
- route_list: '/code.nfs/branches/caches/workstation-yassine_2025-09-08_17-24-15/xx/datasets/lists/train_500k_20240617_test_2k.txt'
- output_dir: null
- save_tmp: false
- read_only: false
- shuffle: false
- limit: 10000
- use_virtual_webgl: false
- format: 'HTML'
- ▶
  miniray: {} 8 keys
  - priority: 10
  - job_name: 'kl_compressor_worldmodel.analyse_vae'
  - queue_name: 'remote2'
  - submit_host: 'redis.comma.internal'
  - result_host: 'data-ram.comma.internal'
  - no_unique_id: false
  - codedir: 'caches/workstation-yassine_2025-09-08_17-24-15'
  - force_local: false
- baseline: '69e6e7e1-4100-48c1-a041-b01cc7940f3b/110'
- ckpt: '4672da0d-19f5-44f8-a5fb-2215981c9c0e/50'
- viz_top_n: 10
- viz_len: 4
- fps: 5
- ▶
  rollout: {} 10 keys
  - type: 'RawVideoRL'
  - nowindow: false
  - debug: false
  - debug_render: false
  - save_observation_keys: ['metrics']
  - output_dir: null
  - rerun: false
  - ▶
    env: {} 22 keys
    - debug_window_size: [512, 512]
    - ▶
      ep: {} 7 keys
      - segment: ''
      - augment_desire: false
      - depth_type: 'EASYDEPTH'
      - calibration_source: 'LOG'
      - pipeline_dir: 'http://data-ssd.comma.life/runner/training_2025_07'
      - feature_dir: null
      - ▶
        input_paths: {} 0 keys
    - ▶
      physics: {} 3 keys
      - type: 'BICYCLE'
      - ▶
        bicycle_model_config: {} 1 keys
        
        vehicle_platform: 'HONDA_CIVIC'
      - ▶
        ml_model_config: {} 8 keys
        
        platform: 'TOYOTA_COROLLA_TSS2'
        
        eid: '3bec45a4-9512-4c7f-bdbc-9415e5890fb6/100'
        
        y_range: [-5.0, 5.0]
        
        vocab_size: 2048
        
        temperature: 0.5
        
        use_gt_features: false
        
        platform_vec_size: 256
        
        platform_map_path: '/code.nfs/branches/caches/workstation-yassine_2025-09-08_17-24-15/xx/arena/envs/lib/physics_models/assets/platform_map.txt'
    - start: null
    - end: null
    - env: 'xx.arena.envs.rawvideo_env.RawVideoEnv'
    - get_imgs: true
    - allow_resets: true
    - zero_desire: true
    - fps: 5
    - allow_lat: true
    - allow_long: false
    - min_hist: 40
    - compressor_model: null
    - compressor_decoder_model: null
    - compressor_mode: 'ENCODE_AND_DECODE'
    - quantized_compressor_latents: false
    - yolo_model: true
    - lpips_model: true
    - sharpness_metric: true
    - future_observation_range_seconds: null
    - with_gt: false
  - ▶
    agent: {} 10 keys
    - supercombo: 'default_vision'
    - restrict_outputs: null
    - exclude_outputs: ['imgs']
    - use_allow_throttle: false
    - use_prev_desired_curv: false
    - model_trained_fps: 5
    - use_worldmodel_as_agent: false
    - use_raw_plan_lat: false
    - desired_curv_smooth_seconds: false
    - desired_accel_smooth_seconds: false
  - ▶
    shadow_agent: {} 0 keys

▶
root: {} 4 keys
- model_trained_fps: 5
- policy_model: null
- use_raw_plan_lat: false
- vision_model: '055a0882-095f-43a3-b0ee-3dd80a2a5f53/500'

Sample Metrics

	None	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
lpips:
mean	3.2804812231731443e-16	0.0587	0.0639
sharpness:
mean	313.2813769887137	258.2389	255.9282
yolo_conf:
mean	0.44090444	0.4053	0.4155

Distribution Metrics

	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
narrow rFID-16.1k	4.9811	5.0798
big rFID-16.1k	4.9232	4.8870

VM Disagreements

	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
BRAKE_PRESS + GAS_PRESS kl div (2s + 4s + 6s avg, bits):
nanmean	0.0098	0.0148
nanmax	5.8608	6.1366
nanstd	0.0333	0.0425
lead_prob kl div (bits):
nanmean	0.0058	0.0085
nanmax	2.1644	3.4026
nanstd	0.0272	0.0370
Plan.VELOCITY at t=0 (m/s):
nanmean	0.1083	0.1348
nanmax	23.3123	24.6580
nanstd	0.0797	0.0939
Plan L1 (no unit):
nanmean	0.0792	0.0967
nanmax	6.7245	7.1462
nanstd	0.0777	0.0823

Top 10 BRAKE_PRESS + GAS_PRESS kl div (2s + 4s + 6s avg, bits) disagreements

▶

None	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
ea0d8e2a4cf9ce46\|2021-11-02--14-23-34/31--536	max disagree: 5.8608	max disagree: 2.9123



b794e7af372e5724\|2022-05-23--13-12-31/6--192	max disagree: 2.4744	max disagree: 1.3197



ee7126c8e2af1b48\|2024-01-22--09-29-31/5--496	max disagree: 2.2807	max disagree: 1.2207



2611ca299b6b5939\|2023-09-22--15-50-09/0--1040	max disagree: 2.2750	max disagree: 1.5400



8faf799674fa044b\|2023-07-18--19-49-14/15--864	max disagree: 2.2710	max disagree: 2.0492



3cfdec54aa035f3f\|2021-09-27--16-23-30/2--352	max disagree: 2.2489	max disagree: 2.7284



ee7126c8e2af1b48\|2024-01-26--18-27-07/37--36	max disagree: 2.1185	max disagree: 2.2301



4c833174af1f11d9\|00000001--ac40d4e315/2--504	max disagree: 2.0859	max disagree: 2.1196



62b2a39186dc9eab\|2021-04-18--12-13-55/4--1168	max disagree: 2.0458	max disagree: 1.7956



bf0fac55013ab937\|2021-12-09--10-55-41/49--4	max disagree: 2.0442	max disagree: 1.0500

Top 10 lead_prob kl div (bits) disagreements

▶

None	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
a092c3532573731f\|2022-06-24--20-30-54/47--276	max disagree: 2.1644	max disagree: 1.6027



91a5f9bc5181e3ba\|2024-01-23--06-02-57/5--944	max disagree: 1.9728	max disagree: 2.1172



d09c814b908913a2\|2023-04-13--21-08-58/13--560	max disagree: 1.9611	max disagree: 1.4191



d8da379d26d61d63\|2022-12-09--09-20-06/1--864	max disagree: 1.9096	max disagree: 2.3306



57e4cb70ba976d3e\|2023-11-22--14-06-15/9--704	max disagree: 1.8820	max disagree: 0.9709



fcb9ba4c67b2c0fb\|2023-10-27--18-01-25/2--1020	max disagree: 1.7827	max disagree: 1.8271



a99416f1dc16ad80\|2021-12-04--16-26-02/159--224	max disagree: 1.6676	max disagree: 0.8541



55cbd4260394fa17\|2022-12-30--19-11-00/6--668	max disagree: 1.6537	max disagree: 1.2808



7914a2f4fbbf103b\|2021-04-08--00-09-04/63--480	max disagree: 1.6514	max disagree: 2.3047



516d6c3bd3ce6315\|2022-11-19--11-33-40/33--488	max disagree: 1.4931	max disagree: 1.0522

Top 10 Plan.VELOCITY at t=0 (m/s) disagreements

▶

None	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
be40ce5f5b2e0871\|2020-12-20--17-34-33/177--808	max disagree: 23.3123	max disagree: 24.6580



4a79abbddaacfcdb\|2024-03-23--23-03-29/19--100	max disagree: 7.1132	max disagree: 6.7889



873884f8a54c5558\|2021-10-12--16-03-45/17--4	max disagree: 6.3976	max disagree: 7.5990



4a229ee1af08a9f8\|2021-06-08--15-01-51/21--40	max disagree: 6.3405	max disagree: 3.3992



f2a594dd91e37527\|2022-04-10--13-29-48/41--268	max disagree: 4.4212	max disagree: 2.6486



26316d513b2e7140\|2020-12-21--03-43-20/0--180	max disagree: 4.3111	max disagree: 5.6656



4a444eab146265fe\|2022-06-01--14-54-45/9--156	max disagree: 4.0989	max disagree: 4.0240



c4870daa2d1dd14d\|2021-11-10--06-12-30/47--1132	max disagree: 4.0626	max disagree: 6.4621



b3ac0e70e5b188a7\|2024-03-01--15-44-09/43--508	max disagree: 3.8962	max disagree: 2.2719



841f184fa5c90ec4\|2022-02-19--21-09-33/32--392	max disagree: 3.7265	max disagree: 3.8689

Top 10 Plan L1 (no unit) disagreements

▶

None	69e6e7e1-4100-48c1-a041-b01cc7940f3b/110	4672da0d-19f5-44f8-a5fb-2215981c9c0e/50
be40ce5f5b2e0871\|2020-12-20--17-34-33/177--808	max disagree: 6.7245	max disagree: 7.1462



cbd6df88df64ed1e\|2023-10-21--16-17-45/10--324	max disagree: 3.8346	max disagree: 3.6994



92e13d6c269f512d\|2023-02-18--04-31-33/82--720	max disagree: 3.6868	max disagree: 3.5956



436363c17591bdd7\|2021-01-25--16-29-01/18--368	max disagree: 3.5168	max disagree: 3.5176



d5e5da41ff5086ec\|2022-05-21--09-59-21/9--736	max disagree: 3.4769	max disagree: 3.3786



d934d61074e5ca75\|2021-01-04--09-08-44/2--620	max disagree: 3.4470	max disagree: 3.5644



026b6d18fba6417f\|2021-06-03--16-38-58/4--916	max disagree: 3.4425	max disagree: 3.4480



4a080fe908e25eb2\|2022-12-23--15-28-55/3--1160	max disagree: 3.3793	max disagree: 3.3951



88b664a0adc4a54e\|2021-11-17--11-52-43/6--376	max disagree: 3.2862	max disagree: 3.3288



88b664a0adc4a54e\|2021-11-17--12-19-26/27--868	max disagree: 3.2656	max disagree: 3.1801