diff --git "a/logs/training_v2_final.log" "b/logs/training_v2_final.log"
new file mode 100644--- /dev/null
+++ "b/logs/training_v2_final.log"
@@ -0,0 +1,572 @@
+
+============================================================
+🎯 OCULUS EXTENDED DETECTION TRAINER
+============================================================
+
+[Loading Model]
+  ✨ Resuming from V2 checkpoint: checkpoints/oculus_detection_v2/final
+  ✓ Loaded projector from checkpoints/oculus_detection_v2/final
+  ✓ Loaded pre-trained detection heads
+[Oculus] Loading vision encoders...
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+
Loading weights:   0%|          | 0/439 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/439 [00:00<00:00, 3457.79it/s, Materializing param=embeddings.cls_token]
Loading weights:   0%|          | 1/439 [00:00<00:00, 1005.83it/s, Materializing param=embeddings.cls_token]
Loading weights:   0%|          | 2/439 [00:00<00:00, 1036.78it/s, Materializing param=embeddings.mask_token]
Loading weights:   0%|          | 2/439 [00:00<00:00, 895.17it/s, Materializing param=embeddings.mask_token] 
Loading weights:   1%|          | 3/439 [00:00<00:00, 1077.12it/s, Materializing param=embeddings.patch_embeddings.projection.bias]
Loading weights:   1%|          | 3/439 [00:00<00:00, 853.66it/s, Materializing param=embeddings.patch_embeddings.projection.bias] 
Loading weights:   1%|          | 4/439 [00:00<00:00, 912.60it/s, Materializing param=embeddings.patch_embeddings.projection.weight]
Loading weights:   1%|          | 4/439 [00:00<00:00, 834.69it/s, Materializing param=embeddings.patch_embeddings.projection.weight]
Loading weights:   1%|          | 5/439 [00:00<00:00, 939.84it/s, Materializing param=embeddings.position_embeddings]               
Loading weights:   1%|          | 5/439 [00:00<00:00, 886.78it/s, Materializing param=embeddings.position_embeddings]
Loading weights:   1%|▏         | 6/439 [00:00<00:00, 1028.94it/s, Materializing param=encoder.layer.0.attention.attention.key.bias]
Loading weights:   1%|▏         | 6/439 [00:00<00:00, 961.37it/s, Materializing param=encoder.layer.0.attention.attention.key.bias] 
Loading weights:   2%|▏         | 7/439 [00:00<00:00, 1052.60it/s, Materializing param=encoder.layer.0.attention.attention.key.weight]
Loading weights:   2%|▏         | 7/439 [00:00<00:00, 1042.40it/s, Materializing param=encoder.layer.0.attention.attention.key.weight]
Loading weights:   2%|▏         | 8/439 [00:00<00:00, 1173.68it/s, Materializing param=encoder.layer.0.attention.attention.query.bias]
Loading weights:   2%|▏         | 8/439 [00:00<00:00, 1155.54it/s, Materializing param=encoder.layer.0.attention.attention.query.bias]
Loading weights:   2%|▏         | 9/439 [00:00<00:00, 1273.66it/s, Materializing param=encoder.layer.0.attention.attention.query.weight]
Loading weights:   2%|▏         | 9/439 [00:00<00:00, 1168.98it/s, Materializing param=encoder.layer.0.attention.attention.query.weight]
Loading weights:   2%|▏         | 10/439 [00:00<00:00, 1229.21it/s, Materializing param=encoder.layer.0.attention.attention.value.bias] 
Loading weights:   2%|▏         | 10/439 [00:00<00:00, 1219.20it/s, Materializing param=encoder.layer.0.attention.attention.value.bias]
Loading weights:   3%|▎         | 11/439 [00:00<00:00, 1282.16it/s, Materializing param=encoder.layer.0.attention.attention.value.weight]
Loading weights:   3%|▎         | 11/439 [00:00<00:00, 1271.21it/s, Materializing param=encoder.layer.0.attention.attention.value.weight]
Loading weights:   3%|▎         | 12/439 [00:00<00:00, 1358.22it/s, Materializing param=encoder.layer.0.attention.output.dense.bias]     
Loading weights:   3%|▎         | 12/439 [00:00<00:00, 1271.68it/s, Materializing param=encoder.layer.0.attention.output.dense.bias]
Loading weights:   3%|▎         | 13/439 [00:00<00:00, 1315.21it/s, Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:   3%|▎         | 13/439 [00:00<00:00, 1270.50it/s, Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:   3%|▎         | 14/439 [00:00<00:00, 1317.25it/s, Materializing param=encoder.layer.0.layer_scale1.lambda1]         
Loading weights:   3%|▎         | 14/439 [00:00<00:00, 1304.11it/s, Materializing param=encoder.layer.0.layer_scale1.lambda1]
Loading weights:   3%|▎         | 15/439 [00:00<00:00, 1306.01it/s, Materializing param=encoder.layer.0.layer_scale2.lambda1]
Loading weights:   3%|▎         | 15/439 [00:00<00:00, 1261.85it/s, Materializing param=encoder.layer.0.layer_scale2.lambda1]
Loading weights:   4%|▎         | 16/439 [00:00<00:00, 1302.40it/s, Materializing param=encoder.layer.0.mlp.fc1.bias]        
Loading weights:   4%|▎         | 16/439 [00:00<00:00, 1259.53it/s, Materializing param=encoder.layer.0.mlp.fc1.bias]
Loading weights:   4%|▍         | 17/439 [00:00<00:00, 1285.34it/s, Materializing param=encoder.layer.0.mlp.fc1.weight]
Loading weights:   4%|▍         | 17/439 [00:00<00:00, 1254.98it/s, Materializing param=encoder.layer.0.mlp.fc1.weight]
Loading weights:   4%|▍         | 18/439 [00:00<00:00, 1280.94it/s, Materializing param=encoder.layer.0.mlp.fc2.bias]  
Loading weights:   4%|▍         | 18/439 [00:00<00:00, 1274.61it/s, Materializing param=encoder.layer.0.mlp.fc2.bias]
Loading weights:   4%|▍         | 19/439 [00:00<00:00, 1307.71it/s, Materializing param=encoder.layer.0.mlp.fc2.weight]
Loading weights:   4%|▍         | 19/439 [00:00<00:00, 1303.86it/s, Materializing param=encoder.layer.0.mlp.fc2.weight]
Loading weights:   5%|▍         | 20/439 [00:00<00:00, 1285.10it/s, Materializing param=encoder.layer.0.norm1.bias]    
Loading weights:   5%|▍         | 20/439 [00:00<00:00, 1277.21it/s, Materializing param=encoder.layer.0.norm1.bias]
Loading weights:   5%|▍         | 21/439 [00:00<00:00, 1314.22it/s, Materializing param=encoder.layer.0.norm1.weight]
Loading weights:   5%|▍         | 21/439 [00:00<00:00, 1310.99it/s, Materializing param=encoder.layer.0.norm1.weight]
Loading weights:   5%|▌         | 22/439 [00:00<00:00, 1295.94it/s, Materializing param=encoder.layer.0.norm2.bias]  
Loading weights:   5%|▌         | 22/439 [00:00<00:00, 1273.72it/s, Materializing param=encoder.layer.0.norm2.bias]
Loading weights:   5%|▌         | 23/439 [00:00<00:00, 1297.15it/s, Materializing param=encoder.layer.0.norm2.weight]
Loading weights:   5%|▌         | 23/439 [00:00<00:00, 1270.35it/s, Materializing param=encoder.layer.0.norm2.weight]
Loading weights:   5%|▌         | 24/439 [00:00<00:00, 1315.57it/s, Materializing param=encoder.layer.1.attention.attention.key.bias]
Loading weights:   5%|▌         | 24/439 [00:00<00:00, 1310.62it/s, Materializing param=encoder.layer.1.attention.attention.key.bias]
Loading weights:   6%|▌         | 25/439 [00:00<00:00, 1325.77it/s, Materializing param=encoder.layer.1.attention.attention.key.weight]
Loading weights:   6%|▌         | 25/439 [00:00<00:00, 1322.87it/s, Materializing param=encoder.layer.1.attention.attention.key.weight]
Loading weights:   6%|▌         | 26/439 [00:00<00:00, 1331.82it/s, Materializing param=encoder.layer.1.attention.attention.query.bias]
Loading weights:   6%|▌         | 26/439 [00:00<00:00, 1327.07it/s, Materializing param=encoder.layer.1.attention.attention.query.bias]
Loading weights:   6%|▌         | 27/439 [00:00<00:00, 1315.39it/s, Materializing param=encoder.layer.1.attention.attention.query.weight]
Loading weights:   6%|▌         | 27/439 [00:00<00:00, 1308.77it/s, Materializing param=encoder.layer.1.attention.attention.query.weight]
Loading weights:   6%|▋         | 28/439 [00:00<00:00, 1328.83it/s, Materializing param=encoder.layer.1.attention.attention.value.bias]  
Loading weights:   6%|▋         | 28/439 [00:00<00:00, 1309.76it/s, Materializing param=encoder.layer.1.attention.attention.value.bias]
Loading weights:   7%|▋         | 29/439 [00:00<00:00, 1348.95it/s, Materializing param=encoder.layer.1.attention.attention.value.weight]
Loading weights:   7%|▋         | 29/439 [00:00<00:00, 1322.87it/s, Materializing param=encoder.layer.1.attention.attention.value.weight]
Loading weights:   7%|▋         | 30/439 [00:00<00:00, 1305.92it/s, Materializing param=encoder.layer.1.attention.output.dense.bias]     
Loading weights:   7%|▋         | 30/439 [00:00<00:00, 1301.64it/s, Materializing param=encoder.layer.1.attention.output.dense.bias]
Loading weights:   7%|▋         | 31/439 [00:00<00:00, 1322.36it/s, Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 31/439 [00:00<00:00, 1299.73it/s, Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 32/439 [00:00<00:00, 1328.94it/s, Materializing param=encoder.layer.1.layer_scale1.lambda1]         
Loading weights:   7%|▋         | 32/439 [00:00<00:00, 1326.25it/s, Materializing param=encoder.layer.1.layer_scale1.lambda1]
Loading weights:   8%|▊         | 33/439 [00:00<00:00, 1280.80it/s, Materializing param=encoder.layer.1.layer_scale2.lambda1]
Loading weights:   8%|▊         | 33/439 [00:00<00:00, 1262.51it/s, Materializing param=encoder.layer.1.layer_scale2.lambda1]
Loading weights:   8%|▊         | 34/439 [00:00<00:00, 1233.58it/s, Materializing param=encoder.layer.1.mlp.fc1.bias]        
Loading weights:   8%|▊         | 34/439 [00:00<00:00, 1210.18it/s, Materializing param=encoder.layer.1.mlp.fc1.bias]
Loading weights:   8%|▊         | 35/439 [00:00<00:00, 1237.44it/s, Materializing param=encoder.layer.1.mlp.fc1.weight]
Loading weights:   8%|▊         | 35/439 [00:00<00:00, 1213.67it/s, Materializing param=encoder.layer.1.mlp.fc1.weight]
Loading weights:   8%|▊         | 36/439 [00:00<00:00, 1241.03it/s, Materializing param=encoder.layer.1.mlp.fc2.bias]  
Loading weights:   8%|▊         | 36/439 [00:00<00:00, 1237.36it/s, Materializing param=encoder.layer.1.mlp.fc2.bias]
Loading weights:   8%|▊         | 37/439 [00:00<00:00, 1264.82it/s, Materializing param=encoder.layer.1.mlp.fc2.weight]
Loading weights:   8%|▊         | 37/439 [00:00<00:00, 1232.72it/s, Materializing param=encoder.layer.1.mlp.fc2.weight]
Loading weights:   9%|▊         | 38/439 [00:00<00:00, 1261.28it/s, Materializing param=encoder.layer.1.norm1.bias]    
Loading weights:   9%|▊         | 38/439 [00:00<00:00, 1247.90it/s, Materializing param=encoder.layer.1.norm1.bias]
Loading weights:   9%|▉         | 39/439 [00:00<00:00, 1246.04it/s, Materializing param=encoder.layer.1.norm1.weight]
Loading weights:   9%|▉         | 39/439 [00:00<00:00, 1243.14it/s, Materializing param=encoder.layer.1.norm1.weight]
Loading weights:   9%|▉         | 40/439 [00:00<00:00, 1271.97it/s, Materializing param=encoder.layer.1.norm2.bias]  
Loading weights:   9%|▉         | 40/439 [00:00<00:00, 1265.14it/s, Materializing param=encoder.layer.1.norm2.bias]
Loading weights:   9%|▉         | 41/439 [00:00<00:00, 1278.52it/s, Materializing param=encoder.layer.1.norm2.weight]
Loading weights:   9%|▉         | 41/439 [00:00<00:00, 1240.91it/s, Materializing param=encoder.layer.1.norm2.weight]
Loading weights:  10%|▉         | 42/439 [00:00<00:00, 1265.05it/s, Materializing param=encoder.layer.2.attention.attention.key.bias]
Loading weights:  10%|▉         | 42/439 [00:00<00:00, 1253.13it/s, Materializing param=encoder.layer.2.attention.attention.key.bias]
Loading weights:  10%|▉         | 43/439 [00:00<00:00, 1268.36it/s, Materializing param=encoder.layer.2.attention.attention.key.weight]
Loading weights:  10%|▉         | 43/439 [00:00<00:00, 1265.93it/s, Materializing param=encoder.layer.2.attention.attention.key.weight]
Loading weights:  10%|█         | 44/439 [00:00<00:00, 1290.58it/s, Materializing param=encoder.layer.2.attention.attention.query.bias]
Loading weights:  10%|█         | 44/439 [00:00<00:00, 1274.06it/s, Materializing param=encoder.layer.2.attention.attention.query.bias]
Loading weights:  10%|█         | 45/439 [00:00<00:00, 1255.54it/s, Materializing param=encoder.layer.2.attention.attention.query.weight]
Loading weights:  10%|█         | 45/439 [00:00<00:00, 1242.85it/s, Materializing param=encoder.layer.2.attention.attention.query.weight]
Loading weights:  10%|█         | 46/439 [00:00<00:00, 1239.48it/s, Materializing param=encoder.layer.2.attention.attention.value.bias]  
Loading weights:  10%|█         | 46/439 [00:00<00:00, 1206.37it/s, Materializing param=encoder.layer.2.attention.attention.value.bias]
Loading weights:  11%|█         | 47/439 [00:00<00:00, 1228.88it/s, Materializing param=encoder.layer.2.attention.attention.value.weight]
Loading weights:  11%|█         | 47/439 [00:00<00:00, 1226.38it/s, Materializing param=encoder.layer.2.attention.attention.value.weight]
Loading weights:  11%|█         | 48/439 [00:00<00:00, 1241.33it/s, Materializing param=encoder.layer.2.attention.output.dense.bias]     
Loading weights:  11%|█         | 48/439 [00:00<00:00, 1197.39it/s, Materializing param=encoder.layer.2.attention.output.dense.bias]
Loading weights:  11%|█         | 49/439 [00:00<00:00, 1200.07it/s, Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  11%|█         | 49/439 [00:00<00:00, 1196.58it/s, Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  11%|█▏        | 50/439 [00:00<00:00, 1199.26it/s, Materializing param=encoder.layer.2.layer_scale1.lambda1]         
Loading weights:  11%|█▏        | 50/439 [00:00<00:00, 1173.82it/s, Materializing param=encoder.layer.2.layer_scale1.lambda1]
Loading weights:  12%|█▏        | 51/439 [00:00<00:00, 1186.46it/s, Materializing param=encoder.layer.2.layer_scale2.lambda1]
Loading weights:  12%|█▏        | 51/439 [00:00<00:00, 1184.28it/s, Materializing param=encoder.layer.2.layer_scale2.lambda1]
Loading weights:  12%|█▏        | 52/439 [00:00<00:00, 1202.14it/s, Materializing param=encoder.layer.2.mlp.fc1.bias]        
Loading weights:  12%|█▏        | 52/439 [00:00<00:00, 1184.50it/s, Materializing param=encoder.layer.2.mlp.fc1.bias]
Loading weights:  12%|█▏        | 53/439 [00:00<00:00, 1198.11it/s, Materializing param=encoder.layer.2.mlp.fc1.weight]
Loading weights:  12%|█▏        | 53/439 [00:00<00:00, 1188.60it/s, Materializing param=encoder.layer.2.mlp.fc1.weight]
Loading weights:  12%|█▏        | 54/439 [00:00<00:00, 1191.24it/s, Materializing param=encoder.layer.2.mlp.fc2.bias]  
Loading weights:  12%|█▏        | 54/439 [00:00<00:00, 1187.80it/s, Materializing param=encoder.layer.2.mlp.fc2.bias]
Loading weights:  13%|█▎        | 55/439 [00:00<00:00, 1206.03it/s, Materializing param=encoder.layer.2.mlp.fc2.weight]
Loading weights:  13%|█▎        | 55/439 [00:00<00:00, 1204.16it/s, Materializing param=encoder.layer.2.mlp.fc2.weight]
Loading weights:  13%|█▎        | 56/439 [00:00<00:00, 1184.46it/s, Materializing param=encoder.layer.2.norm1.bias]    
Loading weights:  13%|█▎        | 56/439 [00:00<00:00, 1183.18it/s, Materializing param=encoder.layer.2.norm1.bias]
Loading weights:  13%|█▎        | 57/439 [00:00<00:00, 1201.14it/s, Materializing param=encoder.layer.2.norm1.weight]
Loading weights:  13%|█▎        | 57/439 [00:00<00:00, 1191.66it/s, Materializing param=encoder.layer.2.norm1.weight]
Loading weights:  13%|█▎        | 58/439 [00:00<00:00, 1210.22it/s, Materializing param=encoder.layer.2.norm2.bias]  
Loading weights:  13%|█▎        | 58/439 [00:00<00:00, 1208.66it/s, Materializing param=encoder.layer.2.norm2.bias]
Loading weights:  13%|█▎        | 59/439 [00:00<00:00, 1222.48it/s, Materializing param=encoder.layer.2.norm2.weight]
Loading weights:  13%|█▎        | 59/439 [00:00<00:00, 1212.01it/s, Materializing param=encoder.layer.2.norm2.weight]
Loading weights:  14%|█▎        | 60/439 [00:00<00:00, 1210.01it/s, Materializing param=encoder.layer.3.attention.attention.key.bias]
Loading weights:  14%|█▎        | 60/439 [00:00<00:00, 1202.67it/s, Materializing param=encoder.layer.3.attention.attention.key.bias]
Loading weights:  14%|█▍        | 61/439 [00:00<00:00, 1201.49it/s, Materializing param=encoder.layer.3.attention.attention.key.weight]
Loading weights:  14%|█▍        | 61/439 [00:00<00:00, 1198.56it/s, Materializing param=encoder.layer.3.attention.attention.key.weight]
Loading weights:  14%|█▍        | 62/439 [00:00<00:00, 1214.80it/s, Materializing param=encoder.layer.3.attention.attention.query.bias]
Loading weights:  14%|█▍        | 62/439 [00:00<00:00, 1211.81it/s, Materializing param=encoder.layer.3.attention.attention.query.bias]
Loading weights:  14%|█▍        | 63/439 [00:00<00:00, 1203.04it/s, Materializing param=encoder.layer.3.attention.attention.query.weight]
Loading weights:  14%|█▍        | 63/439 [00:00<00:00, 1181.81it/s, Materializing param=encoder.layer.3.attention.attention.query.weight]
Loading weights:  15%|█▍        | 64/439 [00:00<00:00, 1187.01it/s, Materializing param=encoder.layer.3.attention.attention.value.bias]  
Loading weights:  15%|█▍        | 64/439 [00:00<00:00, 1184.48it/s, Materializing param=encoder.layer.3.attention.attention.value.bias]
Loading weights:  15%|█▍        | 65/439 [00:00<00:00, 1194.76it/s, Materializing param=encoder.layer.3.attention.attention.value.weight]
Loading weights:  15%|█▍        | 65/439 [00:00<00:00, 1192.22it/s, Materializing param=encoder.layer.3.attention.attention.value.weight]
Loading weights:  15%|█▌        | 66/439 [00:00<00:00, 1190.47it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]     
Loading weights:  15%|█▌        | 66/439 [00:00<00:00, 1188.91it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]
Loading weights:  15%|█▌        | 67/439 [00:00<00:00, 1202.11it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  15%|█▌        | 67/439 [00:00<00:00, 1200.26it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  15%|█▌        | 68/439 [00:00<00:00, 1216.18it/s, Materializing param=encoder.layer.3.layer_scale1.lambda1]         
Loading weights:  15%|█▌        | 68/439 [00:00<00:00, 1215.39it/s, Materializing param=encoder.layer.3.layer_scale1.lambda1]
Loading weights:  16%|█▌        | 69/439 [00:00<00:00, 1231.79it/s, Materializing param=encoder.layer.3.layer_scale2.lambda1]
Loading weights:  16%|█▌        | 69/439 [00:00<00:00, 1231.08it/s, Materializing param=encoder.layer.3.layer_scale2.lambda1]
Loading weights:  16%|█▌        | 70/439 [00:00<00:00, 1247.74it/s, Materializing param=encoder.layer.3.mlp.fc1.bias]        
Loading weights:  16%|█▌        | 70/439 [00:00<00:00, 1247.08it/s, Materializing param=encoder.layer.3.mlp.fc1.bias]
Loading weights:  16%|█▌        | 71/439 [00:00<00:00, 1263.70it/s, Materializing param=encoder.layer.3.mlp.fc1.weight]
Loading weights:  16%|█▌        | 71/439 [00:00<00:00, 1263.03it/s, Materializing param=encoder.layer.3.mlp.fc1.weight]
Loading weights:  16%|█▋        | 72/439 [00:00<00:00, 1279.59it/s, Materializing param=encoder.layer.3.mlp.fc2.bias]  
Loading weights:  16%|█▋        | 72/439 [00:00<00:00, 1278.91it/s, Materializing param=encoder.layer.3.mlp.fc2.bias]
Loading weights:  17%|█▋        | 73/439 [00:00<00:00, 1295.54it/s, Materializing param=encoder.layer.3.mlp.fc2.weight]
Loading weights:  17%|█▋        | 73/439 [00:00<00:00, 1294.85it/s, Materializing param=encoder.layer.3.mlp.fc2.weight]
Loading weights:  17%|█▋        | 74/439 [00:00<00:00, 1311.47it/s, Materializing param=encoder.layer.3.norm1.bias]    
Loading weights:  17%|█▋        | 74/439 [00:00<00:00, 1310.77it/s, Materializing param=encoder.layer.3.norm1.bias]
Loading weights:  17%|█▋        | 75/439 [00:00<00:00, 1327.38it/s, Materializing param=encoder.layer.3.norm1.weight]
Loading weights:  17%|█▋        | 75/439 [00:00<00:00, 1326.66it/s, Materializing param=encoder.layer.3.norm1.weight]
Loading weights:  17%|█▋        | 76/439 [00:00<00:00, 1343.25it/s, Materializing param=encoder.layer.3.norm2.bias]  
Loading weights:  17%|█▋        | 76/439 [00:00<00:00, 1342.54it/s, Materializing param=encoder.layer.3.norm2.bias]
Loading weights:  18%|█▊        | 77/439 [00:00<00:00, 1358.76it/s, Materializing param=encoder.layer.3.norm2.weight]
Loading weights:  18%|█▊        | 77/439 [00:00<00:00, 1358.04it/s, Materializing param=encoder.layer.3.norm2.weight]
Loading weights:  18%|█▊        | 78/439 [00:00<00:00, 1374.55it/s, Materializing param=encoder.layer.4.attention.attention.key.bias]
Loading weights:  18%|█▊        | 78/439 [00:00<00:00, 1373.77it/s, Materializing param=encoder.layer.4.attention.attention.key.bias]
Loading weights:  18%|█▊        | 79/439 [00:00<00:00, 1389.96it/s, Materializing param=encoder.layer.4.attention.attention.key.weight]
Loading weights:  18%|█▊        | 79/439 [00:00<00:00, 1389.20it/s, Materializing param=encoder.layer.4.attention.attention.key.weight]
Loading weights:  18%|█▊        | 80/439 [00:00<00:00, 1405.53it/s, Materializing param=encoder.layer.4.attention.attention.query.bias]
Loading weights:  18%|█▊        | 80/439 [00:00<00:00, 1404.79it/s, Materializing param=encoder.layer.4.attention.attention.query.bias]
Loading weights:  18%|█▊        | 81/439 [00:00<00:00, 1421.02it/s, Materializing param=encoder.layer.4.attention.attention.query.weight]
Loading weights:  18%|█▊        | 81/439 [00:00<00:00, 1420.25it/s, Materializing param=encoder.layer.4.attention.attention.query.weight]
Loading weights:  19%|█▊        | 82/439 [00:00<00:00, 1436.45it/s, Materializing param=encoder.layer.4.attention.attention.value.bias]  
Loading weights:  19%|█▊        | 82/439 [00:00<00:00, 1435.67it/s, Materializing param=encoder.layer.4.attention.attention.value.bias]
Loading weights:  19%|█▉        | 83/439 [00:00<00:00, 1451.91it/s, Materializing param=encoder.layer.4.attention.attention.value.weight]
Loading weights:  19%|█▉        | 83/439 [00:00<00:00, 1451.12it/s, Materializing param=encoder.layer.4.attention.attention.value.weight]
Loading weights:  19%|█▉        | 84/439 [00:00<00:00, 1467.35it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]     
Loading weights:  19%|█▉        | 84/439 [00:00<00:00, 1466.58it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]
Loading weights:  19%|█▉        | 85/439 [00:00<00:00, 1482.75it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  19%|█▉        | 85/439 [00:00<00:00, 1481.97it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  20%|█▉        | 86/439 [00:00<00:00, 1498.07it/s, Materializing param=encoder.layer.4.layer_scale1.lambda1]         
Loading weights:  20%|█▉        | 86/439 [00:00<00:00, 1497.29it/s, Materializing param=encoder.layer.4.layer_scale1.lambda1]
Loading weights:  20%|█▉        | 87/439 [00:00<00:00, 1513.25it/s, Materializing param=encoder.layer.4.layer_scale2.lambda1]
Loading weights:  20%|█▉        | 87/439 [00:00<00:00, 1512.46it/s, Materializing param=encoder.layer.4.layer_scale2.lambda1]
Loading weights:  20%|██        | 88/439 [00:00<00:00, 1528.49it/s, Materializing param=encoder.layer.4.mlp.fc1.bias]        
Loading weights:  20%|██        | 88/439 [00:00<00:00, 1527.72it/s, Materializing param=encoder.layer.4.mlp.fc1.bias]
Loading weights:  20%|██        | 89/439 [00:00<00:00, 1543.58it/s, Materializing param=encoder.layer.4.mlp.fc1.weight]
Loading weights:  20%|██        | 89/439 [00:00<00:00, 1542.81it/s, Materializing param=encoder.layer.4.mlp.fc1.weight]
Loading weights:  21%|██        | 90/439 [00:00<00:00, 1558.74it/s, Materializing param=encoder.layer.4.mlp.fc2.bias]  
Loading weights:  21%|██        | 90/439 [00:00<00:00, 1557.95it/s, Materializing param=encoder.layer.4.mlp.fc2.bias]
Loading weights:  21%|██        | 91/439 [00:00<00:00, 1574.01it/s, Materializing param=encoder.layer.4.mlp.fc2.weight]
Loading weights:  21%|██        | 91/439 [00:00<00:00, 1573.22it/s, Materializing param=encoder.layer.4.mlp.fc2.weight]
Loading weights:  21%|██        | 92/439 [00:00<00:00, 1589.19it/s, Materializing param=encoder.layer.4.norm1.bias]    
Loading weights:  21%|██        | 92/439 [00:00<00:00, 1588.20it/s, Materializing param=encoder.layer.4.norm1.bias]
Loading weights:  21%|██        | 93/439 [00:00<00:00, 1604.22it/s, Materializing param=encoder.layer.4.norm1.weight]
Loading weights:  21%|██        | 93/439 [00:00<00:00, 1603.39it/s, Materializing param=encoder.layer.4.norm1.weight]
Loading weights:  21%|██▏       | 94/439 [00:00<00:00, 1619.18it/s, Materializing param=encoder.layer.4.norm2.bias]  
Loading weights:  21%|██▏       | 94/439 [00:00<00:00, 1618.34it/s, Materializing param=encoder.layer.4.norm2.bias]
Loading weights:  22%|██▏       | 95/439 [00:00<00:00, 1634.29it/s, Materializing param=encoder.layer.4.norm2.weight]
Loading weights:  22%|██▏       | 95/439 [00:00<00:00, 1633.48it/s, Materializing param=encoder.layer.4.norm2.weight]
Loading weights:  22%|██▏       | 96/439 [00:00<00:00, 1649.42it/s, Materializing param=encoder.layer.5.attention.attention.key.bias]
Loading weights:  22%|██▏       | 96/439 [00:00<00:00, 1648.54it/s, Materializing param=encoder.layer.5.attention.attention.key.bias]
Loading weights:  22%|██▏       | 97/439 [00:00<00:00, 1664.17it/s, Materializing param=encoder.layer.5.attention.attention.key.weight]
Loading weights:  22%|██▏       | 97/439 [00:00<00:00, 1663.29it/s, Materializing param=encoder.layer.5.attention.attention.key.weight]
Loading weights:  22%|██▏       | 98/439 [00:00<00:00, 1679.03it/s, Materializing param=encoder.layer.5.attention.attention.query.bias]
Loading weights:  22%|██▏       | 98/439 [00:00<00:00, 1678.16it/s, Materializing param=encoder.layer.5.attention.attention.query.bias]
Loading weights:  23%|██▎       | 99/439 [00:00<00:00, 1693.93it/s, Materializing param=encoder.layer.5.attention.attention.query.weight]
Loading weights:  23%|██▎       | 99/439 [00:00<00:00, 1693.03it/s, Materializing param=encoder.layer.5.attention.attention.query.weight]
Loading weights:  23%|██▎       | 100/439 [00:00<00:00, 1708.66it/s, Materializing param=encoder.layer.5.attention.attention.value.bias] 
Loading weights:  23%|██▎       | 100/439 [00:00<00:00, 1707.74it/s, Materializing param=encoder.layer.5.attention.attention.value.bias]
Loading weights:  23%|██▎       | 101/439 [00:00<00:00, 1723.34it/s, Materializing param=encoder.layer.5.attention.attention.value.weight]
Loading weights:  23%|██▎       | 101/439 [00:00<00:00, 1722.23it/s, Materializing param=encoder.layer.5.attention.attention.value.weight]
Loading weights:  23%|██▎       | 102/439 [00:00<00:00, 1737.77it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]     
Loading weights:  23%|██▎       | 102/439 [00:00<00:00, 1736.84it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]
Loading weights:  23%|██▎       | 103/439 [00:00<00:00, 1752.47it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  23%|██▎       | 103/439 [00:00<00:00, 1751.55it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  24%|██▎       | 104/439 [00:00<00:00, 1766.87it/s, Materializing param=encoder.layer.5.layer_scale1.lambda1]         
Loading weights:  24%|██▎       | 104/439 [00:00<00:00, 1766.00it/s, Materializing param=encoder.layer.5.layer_scale1.lambda1]
Loading weights:  24%|██▍       | 105/439 [00:00<00:00, 1781.44it/s, Materializing param=encoder.layer.5.layer_scale2.lambda1]
Loading weights:  24%|██▍       | 105/439 [00:00<00:00, 1780.53it/s, Materializing param=encoder.layer.5.layer_scale2.lambda1]
Loading weights:  24%|██▍       | 106/439 [00:00<00:00, 1796.00it/s, Materializing param=encoder.layer.5.mlp.fc1.bias]        
Loading weights:  24%|██▍       | 106/439 [00:00<00:00, 1795.09it/s, Materializing param=encoder.layer.5.mlp.fc1.bias]
Loading weights:  24%|██▍       | 107/439 [00:00<00:00, 1810.43it/s, Materializing param=encoder.layer.5.mlp.fc1.weight]
Loading weights:  24%|██▍       | 107/439 [00:00<00:00, 1809.11it/s, Materializing param=encoder.layer.5.mlp.fc1.weight]
Loading weights:  25%|██▍       | 108/439 [00:00<00:00, 1824.54it/s, Materializing param=encoder.layer.5.mlp.fc2.bias]  
Loading weights:  25%|██▍       | 108/439 [00:00<00:00, 1823.61it/s, Materializing param=encoder.layer.5.mlp.fc2.bias]
Loading weights:  25%|██▍       | 109/439 [00:00<00:00, 1838.98it/s, Materializing param=encoder.layer.5.mlp.fc2.weight]
Loading weights:  25%|██▍       | 109/439 [00:00<00:00, 1838.08it/s, Materializing param=encoder.layer.5.mlp.fc2.weight]
Loading weights:  25%|██▌       | 110/439 [00:00<00:00, 1853.47it/s, Materializing param=encoder.layer.5.norm1.bias]    
Loading weights:  25%|██▌       | 110/439 [00:00<00:00, 1852.56it/s, Materializing param=encoder.layer.5.norm1.bias]
Loading weights:  25%|██▌       | 111/439 [00:00<00:00, 1868.02it/s, Materializing param=encoder.layer.5.norm1.weight]
Loading weights:  25%|██▌       | 111/439 [00:00<00:00, 1867.11it/s, Materializing param=encoder.layer.5.norm1.weight]
Loading weights:  26%|██▌       | 112/439 [00:00<00:00, 1882.44it/s, Materializing param=encoder.layer.5.norm2.bias]  
Loading weights:  26%|██▌       | 112/439 [00:00<00:00, 1881.46it/s, Materializing param=encoder.layer.5.norm2.bias]
Loading weights:  26%|██▌       | 113/439 [00:00<00:00, 1896.80it/s, Materializing param=encoder.layer.5.norm2.weight]
Loading weights:  26%|██▌       | 113/439 [00:00<00:00, 1895.88it/s, Materializing param=encoder.layer.5.norm2.weight]
Loading weights:  26%|██▌       | 114/439 [00:00<00:00, 1911.11it/s, Materializing param=encoder.layer.6.attention.attention.key.bias]
Loading weights:  26%|██▌       | 114/439 [00:00<00:00, 1910.12it/s, Materializing param=encoder.layer.6.attention.attention.key.bias]
Loading weights:  26%|██▌       | 115/439 [00:00<00:00, 1925.13it/s, Materializing param=encoder.layer.6.attention.attention.key.weight]
Loading weights:  26%|██▌       | 115/439 [00:00<00:00, 1924.13it/s, Materializing param=encoder.layer.6.attention.attention.key.weight]
Loading weights:  26%|██▋       | 116/439 [00:00<00:00, 1939.21it/s, Materializing param=encoder.layer.6.attention.attention.query.bias]
Loading weights:  26%|██▋       | 116/439 [00:00<00:00, 1938.26it/s, Materializing param=encoder.layer.6.attention.attention.query.bias]
Loading weights:  27%|██▋       | 117/439 [00:00<00:00, 1953.38it/s, Materializing param=encoder.layer.6.attention.attention.query.weight]
Loading weights:  27%|██▋       | 117/439 [00:00<00:00, 1952.37it/s, Materializing param=encoder.layer.6.attention.attention.query.weight]
Loading weights:  27%|██▋       | 118/439 [00:00<00:00, 1967.42it/s, Materializing param=encoder.layer.6.attention.attention.value.bias]  
Loading weights:  27%|██▋       | 118/439 [00:00<00:00, 1966.40it/s, Materializing param=encoder.layer.6.attention.attention.value.bias]
Loading weights:  27%|██▋       | 119/439 [00:00<00:00, 1981.37it/s, Materializing param=encoder.layer.6.attention.attention.value.weight]
Loading weights:  27%|██▋       | 119/439 [00:00<00:00, 1980.39it/s, Materializing param=encoder.layer.6.attention.attention.value.weight]
Loading weights:  27%|██▋       | 120/439 [00:00<00:00, 1995.08it/s, Materializing param=encoder.layer.6.attention.output.dense.bias]     
Loading weights:  27%|██▋       | 120/439 [00:00<00:00, 1994.07it/s, Materializing param=encoder.layer.6.attention.output.dense.bias]
Loading weights:  28%|██▊       | 121/439 [00:00<00:00, 2008.90it/s, Materializing param=encoder.layer.6.attention.output.dense.weight]
Loading weights:  28%|██▊       | 121/439 [00:00<00:00, 2007.89it/s, Materializing param=encoder.layer.6.attention.output.dense.weight]
Loading weights:  28%|██▊       | 122/439 [00:00<00:00, 2022.85it/s, Materializing param=encoder.layer.6.layer_scale1.lambda1]         
Loading weights:  28%|██▊       | 122/439 [00:00<00:00, 2021.57it/s, Materializing param=encoder.layer.6.layer_scale1.lambda1]
Loading weights:  28%|██▊       | 123/439 [00:00<00:00, 2036.59it/s, Materializing param=encoder.layer.6.layer_scale2.lambda1]
Loading weights:  28%|██▊       | 123/439 [00:00<00:00, 2035.55it/s, Materializing param=encoder.layer.6.layer_scale2.lambda1]
Loading weights:  28%|██▊       | 124/439 [00:00<00:00, 2050.39it/s, Materializing param=encoder.layer.6.mlp.fc1.bias]        
Loading weights:  28%|██▊       | 124/439 [00:00<00:00, 2049.34it/s, Materializing param=encoder.layer.6.mlp.fc1.bias]
Loading weights:  28%|██▊       | 125/439 [00:00<00:00, 2064.20it/s, Materializing param=encoder.layer.6.mlp.fc1.weight]
Loading weights:  28%|██▊       | 125/439 [00:00<00:00, 2063.18it/s, Materializing param=encoder.layer.6.mlp.fc1.weight]
Loading weights:  29%|██▊       | 126/439 [00:00<00:00, 2078.04it/s, Materializing param=encoder.layer.6.mlp.fc2.bias]  
Loading weights:  29%|██▊       | 126/439 [00:00<00:00, 2077.08it/s, Materializing param=encoder.layer.6.mlp.fc2.bias]
Loading weights:  29%|██▉       | 127/439 [00:00<00:00, 2091.91it/s, Materializing param=encoder.layer.6.mlp.fc2.weight]
Loading weights:  29%|██▉       | 127/439 [00:00<00:00, 2090.87it/s, Materializing param=encoder.layer.6.mlp.fc2.weight]
Loading weights:  29%|██▉       | 128/439 [00:00<00:00, 2105.77it/s, Materializing param=encoder.layer.6.norm1.bias]    
Loading weights:  29%|██▉       | 128/439 [00:00<00:00, 2104.77it/s, Materializing param=encoder.layer.6.norm1.bias]
Loading weights:  29%|██▉       | 129/439 [00:00<00:00, 2119.65it/s, Materializing param=encoder.layer.6.norm1.weight]
Loading weights:  29%|██▉       | 129/439 [00:00<00:00, 2118.64it/s, Materializing param=encoder.layer.6.norm1.weight]
Loading weights:  30%|██▉       | 130/439 [00:00<00:00, 2133.45it/s, Materializing param=encoder.layer.6.norm2.bias]  
Loading weights:  30%|██▉       | 130/439 [00:00<00:00, 2132.40it/s, Materializing param=encoder.layer.6.norm2.bias]
Loading weights:  30%|██▉       | 131/439 [00:00<00:00, 2147.12it/s, Materializing param=encoder.layer.6.norm2.weight]
Loading weights:  30%|██▉       | 131/439 [00:00<00:00, 2145.88it/s, Materializing param=encoder.layer.6.norm2.weight]
Loading weights:  30%|███       | 132/439 [00:00<00:00, 2159.79it/s, Materializing param=encoder.layer.7.attention.attention.key.bias]
Loading weights:  30%|███       | 132/439 [00:00<00:00, 2158.65it/s, Materializing param=encoder.layer.7.attention.attention.key.bias]
Loading weights:  30%|███       | 133/439 [00:00<00:00, 2173.10it/s, Materializing param=encoder.layer.7.attention.attention.key.weight]
Loading weights:  30%|███       | 133/439 [00:00<00:00, 2172.02it/s, Materializing param=encoder.layer.7.attention.attention.key.weight]
Loading weights:  31%|███       | 134/439 [00:00<00:00, 2186.53it/s, Materializing param=encoder.layer.7.attention.attention.query.bias]
Loading weights:  31%|███       | 134/439 [00:00<00:00, 2185.43it/s, Materializing param=encoder.layer.7.attention.attention.query.bias]
Loading weights:  31%|███       | 135/439 [00:00<00:00, 2200.02it/s, Materializing param=encoder.layer.7.attention.attention.query.weight]
Loading weights:  31%|███       | 135/439 [00:00<00:00, 2198.90it/s, Materializing param=encoder.layer.7.attention.attention.query.weight]
Loading weights:  31%|███       | 136/439 [00:00<00:00, 2213.43it/s, Materializing param=encoder.layer.7.attention.attention.value.bias]  
Loading weights:  31%|███       | 136/439 [00:00<00:00, 2212.31it/s, Materializing param=encoder.layer.7.attention.attention.value.bias]
Loading weights:  31%|███       | 137/439 [00:00<00:00, 2226.84it/s, Materializing param=encoder.layer.7.attention.attention.value.weight]
Loading weights:  31%|███       | 137/439 [00:00<00:00, 2225.75it/s, Materializing param=encoder.layer.7.attention.attention.value.weight]
Loading weights:  31%|███▏      | 138/439 [00:00<00:00, 2239.99it/s, Materializing param=encoder.layer.7.attention.output.dense.bias]     
Loading weights:  31%|███▏      | 138/439 [00:00<00:00, 2238.87it/s, Materializing param=encoder.layer.7.attention.output.dense.bias]
Loading weights:  32%|███▏      | 139/439 [00:00<00:00, 2253.19it/s, Materializing param=encoder.layer.7.attention.output.dense.weight]
Loading weights:  32%|███▏      | 139/439 [00:00<00:00, 2252.14it/s, Materializing param=encoder.layer.7.attention.output.dense.weight]
Loading weights:  32%|███▏      | 140/439 [00:00<00:00, 2266.57it/s, Materializing param=encoder.layer.7.layer_scale1.lambda1]         
Loading weights:  32%|███▏      | 140/439 [00:00<00:00, 2265.48it/s, Materializing param=encoder.layer.7.layer_scale1.lambda1]
Loading weights:  32%|███▏      | 141/439 [00:00<00:00, 2280.00it/s, Materializing param=encoder.layer.7.layer_scale2.lambda1]
Loading weights:  32%|███▏      | 141/439 [00:00<00:00, 2278.92it/s, Materializing param=encoder.layer.7.layer_scale2.lambda1]
Loading weights:  32%|███▏      | 142/439 [00:00<00:00, 2293.20it/s, Materializing param=encoder.layer.7.mlp.fc1.bias]        
Loading weights:  32%|███▏      | 142/439 [00:00<00:00, 2292.13it/s, Materializing param=encoder.layer.7.mlp.fc1.bias]
Loading weights:  33%|███▎      | 143/439 [00:00<00:00, 2306.52it/s, Materializing param=encoder.layer.7.mlp.fc1.weight]
Loading weights:  33%|███▎      | 143/439 [00:00<00:00, 2305.44it/s, Materializing param=encoder.layer.7.mlp.fc1.weight]
Loading weights:  33%|███▎      | 144/439 [00:00<00:00, 2319.84it/s, Materializing param=encoder.layer.7.mlp.fc2.bias]  
Loading weights:  33%|███▎      | 144/439 [00:00<00:00, 2318.73it/s, Materializing param=encoder.layer.7.mlp.fc2.bias]
Loading weights:  33%|███▎      | 145/439 [00:00<00:00, 2333.14it/s, Materializing param=encoder.layer.7.mlp.fc2.weight]
Loading weights:  33%|███▎      | 145/439 [00:00<00:00, 2332.01it/s, Materializing param=encoder.layer.7.mlp.fc2.weight]
Loading weights:  33%|███▎      | 146/439 [00:00<00:00, 2346.36it/s, Materializing param=encoder.layer.7.norm1.bias]    
Loading weights:  33%|███▎      | 146/439 [00:00<00:00, 2345.26it/s, Materializing param=encoder.layer.7.norm1.bias]
Loading weights:  33%|███▎      | 147/439 [00:00<00:00, 2359.66it/s, Materializing param=encoder.layer.7.norm1.weight]
Loading weights:  33%|███▎      | 147/439 [00:00<00:00, 2358.52it/s, Materializing param=encoder.layer.7.norm1.weight]
Loading weights:  34%|███▎      | 148/439 [00:00<00:00, 2372.77it/s, Materializing param=encoder.layer.7.norm2.bias]  
Loading weights:  34%|███▎      | 148/439 [00:00<00:00, 2371.67it/s, Materializing param=encoder.layer.7.norm2.bias]
Loading weights:  34%|███▍      | 149/439 [00:00<00:00, 2386.01it/s, Materializing param=encoder.layer.7.norm2.weight]
Loading weights:  34%|███▍      | 149/439 [00:00<00:00, 2384.91it/s, Materializing param=encoder.layer.7.norm2.weight]
Loading weights:  34%|███▍      | 150/439 [00:00<00:00, 2399.22it/s, Materializing param=encoder.layer.8.attention.attention.key.bias]
Loading weights:  34%|███▍      | 150/439 [00:00<00:00, 2398.08it/s, Materializing param=encoder.layer.8.attention.attention.key.bias]
Loading weights:  34%|███▍      | 151/439 [00:00<00:00, 2411.68it/s, Materializing param=encoder.layer.8.attention.attention.key.weight]
Loading weights:  34%|███▍      | 151/439 [00:00<00:00, 2410.48it/s, Materializing param=encoder.layer.8.attention.attention.key.weight]
Loading weights:  35%|███▍      | 152/439 [00:00<00:00, 2424.43it/s, Materializing param=encoder.layer.8.attention.attention.query.bias]
Loading weights:  35%|███▍      | 152/439 [00:00<00:00, 2423.26it/s, Materializing param=encoder.layer.8.attention.attention.query.bias]
Loading weights:  35%|███▍      | 153/439 [00:00<00:00, 2435.13it/s, Materializing param=encoder.layer.8.attention.attention.query.weight]
Loading weights:  35%|███▍      | 153/439 [00:00<00:00, 2432.58it/s, Materializing param=encoder.layer.8.attention.attention.query.weight]
Loading weights:  35%|███▌      | 154/439 [00:00<00:00, 2445.49it/s, Materializing param=encoder.layer.8.attention.attention.value.bias]  
Loading weights:  35%|███▌      | 154/439 [00:00<00:00, 2443.93it/s, Materializing param=encoder.layer.8.attention.attention.value.bias]
Loading weights:  35%|███▌      | 155/439 [00:00<00:00, 2457.55it/s, Materializing param=encoder.layer.8.attention.attention.value.weight]
Loading weights:  35%|███▌      | 155/439 [00:00<00:00, 2455.95it/s, Materializing param=encoder.layer.8.attention.attention.value.weight]
Loading weights:  36%|███▌      | 156/439 [00:00<00:00, 2469.75it/s, Materializing param=encoder.layer.8.attention.output.dense.bias]     
Loading weights:  36%|███▌      | 156/439 [00:00<00:00, 2468.50it/s, Materializing param=encoder.layer.8.attention.output.dense.bias]
Loading weights:  36%|███▌      | 157/439 [00:00<00:00, 2482.32it/s, Materializing param=encoder.layer.8.attention.output.dense.weight]
Loading weights:  36%|███▌      | 157/439 [00:00<00:00, 2481.11it/s, Materializing param=encoder.layer.8.attention.output.dense.weight]
Loading weights:  36%|███▌      | 158/439 [00:00<00:00, 2494.86it/s, Materializing param=encoder.layer.8.layer_scale1.lambda1]         
Loading weights:  36%|███▌      | 158/439 [00:00<00:00, 2493.64it/s, Materializing param=encoder.layer.8.layer_scale1.lambda1]
Loading weights:  36%|███▌      | 159/439 [00:00<00:00, 2507.52it/s, Materializing param=encoder.layer.8.layer_scale2.lambda1]
Loading weights:  36%|███▌      | 159/439 [00:00<00:00, 2506.38it/s, Materializing param=encoder.layer.8.layer_scale2.lambda1]
Loading weights:  36%|███▋      | 160/439 [00:00<00:00, 2520.08it/s, Materializing param=encoder.layer.8.mlp.fc1.bias]        
Loading weights:  36%|███▋      | 160/439 [00:00<00:00, 2518.81it/s, Materializing param=encoder.layer.8.mlp.fc1.bias]
Loading weights:  37%|███▋      | 161/439 [00:00<00:00, 2532.39it/s, Materializing param=encoder.layer.8.mlp.fc1.weight]
Loading weights:  37%|███▋      | 161/439 [00:00<00:00, 2531.21it/s, Materializing param=encoder.layer.8.mlp.fc1.weight]
Loading weights:  37%|███▋      | 162/439 [00:00<00:00, 2544.80it/s, Materializing param=encoder.layer.8.mlp.fc2.bias]  
Loading weights:  37%|███▋      | 162/439 [00:00<00:00, 2543.65it/s, Materializing param=encoder.layer.8.mlp.fc2.bias]
Loading weights:  37%|███▋      | 163/439 [00:00<00:00, 2557.06it/s, Materializing param=encoder.layer.8.mlp.fc2.weight]
Loading weights:  37%|███▋      | 163/439 [00:00<00:00, 2555.90it/s, Materializing param=encoder.layer.8.mlp.fc2.weight]
Loading weights:  37%|███▋      | 164/439 [00:00<00:00, 2569.51it/s, Materializing param=encoder.layer.8.norm1.bias]    
Loading weights:  37%|███▋      | 164/439 [00:00<00:00, 2568.31it/s, Materializing param=encoder.layer.8.norm1.bias]
Loading weights:  38%|███▊      | 165/439 [00:00<00:00, 2582.07it/s, Materializing param=encoder.layer.8.norm1.weight]
Loading weights:  38%|███▊      | 165/439 [00:00<00:00, 2580.86it/s, Materializing param=encoder.layer.8.norm1.weight]
Loading weights:  38%|███▊      | 166/439 [00:00<00:00, 2594.72it/s, Materializing param=encoder.layer.8.norm2.bias]  
Loading weights:  38%|███▊      | 166/439 [00:00<00:00, 2593.45it/s, Materializing param=encoder.layer.8.norm2.bias]
Loading weights:  38%|███▊      | 167/439 [00:00<00:00, 2607.13it/s, Materializing param=encoder.layer.8.norm2.weight]
Loading weights:  38%|███▊      | 167/439 [00:00<00:00, 2605.95it/s, Materializing param=encoder.layer.8.norm2.weight]
Loading weights:  38%|███▊      | 168/439 [00:00<00:00, 2619.68it/s, Materializing param=encoder.layer.9.attention.attention.key.bias]
Loading weights:  38%|███▊      | 168/439 [00:00<00:00, 2617.68it/s, Materializing param=encoder.layer.9.attention.attention.key.bias]
Loading weights:  38%|███▊      | 169/439 [00:00<00:00, 2630.75it/s, Materializing param=encoder.layer.9.attention.attention.key.weight]
Loading weights:  38%|███▊      | 169/439 [00:00<00:00, 2629.48it/s, Materializing param=encoder.layer.9.attention.attention.key.weight]
Loading weights:  39%|███▊      | 170/439 [00:00<00:00, 2642.99it/s, Materializing param=encoder.layer.9.attention.attention.query.bias]
Loading weights:  39%|███▊      | 170/439 [00:00<00:00, 2641.76it/s, Materializing param=encoder.layer.9.attention.attention.query.bias]
Loading weights:  39%|███▉      | 171/439 [00:00<00:00, 2655.35it/s, Materializing param=encoder.layer.9.attention.attention.query.weight]
Loading weights:  39%|███▉      | 171/439 [00:00<00:00, 2654.12it/s, Materializing param=encoder.layer.9.attention.attention.query.weight]
Loading weights:  39%|███▉      | 172/439 [00:00<00:00, 2667.65it/s, Materializing param=encoder.layer.9.attention.attention.value.bias]  
Loading weights:  39%|███▉      | 172/439 [00:00<00:00, 2666.37it/s, Materializing param=encoder.layer.9.attention.attention.value.bias]
Loading weights:  39%|███▉      | 173/439 [00:00<00:00, 2679.84it/s, Materializing param=encoder.layer.9.attention.attention.value.weight]
Loading weights:  39%|███▉      | 173/439 [00:00<00:00, 2678.55it/s, Materializing param=encoder.layer.9.attention.attention.value.weight]
Loading weights:  40%|███▉      | 174/439 [00:00<00:00, 2691.66it/s, Materializing param=encoder.layer.9.attention.output.dense.bias]     
Loading weights:  40%|███▉      | 174/439 [00:00<00:00, 2690.37it/s, Materializing param=encoder.layer.9.attention.output.dense.bias]
Loading weights:  40%|███▉      | 175/439 [00:00<00:00, 2703.79it/s, Materializing param=encoder.layer.9.attention.output.dense.weight]
Loading weights:  40%|███▉      | 175/439 [00:00<00:00, 2702.45it/s, Materializing param=encoder.layer.9.attention.output.dense.weight]
Loading weights:  40%|████      | 176/439 [00:00<00:00, 2715.67it/s, Materializing param=encoder.layer.9.layer_scale1.lambda1]         
Loading weights:  40%|████      | 176/439 [00:00<00:00, 2714.40it/s, Materializing param=encoder.layer.9.layer_scale1.lambda1]
Loading weights:  40%|████      | 177/439 [00:00<00:00, 2727.86it/s, Materializing param=encoder.layer.9.layer_scale2.lambda1]
Loading weights:  40%|████      | 177/439 [00:00<00:00, 2726.59it/s, Materializing param=encoder.layer.9.layer_scale2.lambda1]
Loading weights:  41%|████      | 178/439 [00:00<00:00, 2739.97it/s, Materializing param=encoder.layer.9.mlp.fc1.bias]        
Loading weights:  41%|████      | 178/439 [00:00<00:00, 2738.79it/s, Materializing param=encoder.layer.9.mlp.fc1.bias]
Loading weights:  41%|████      | 179/439 [00:00<00:00, 2752.15it/s, Materializing param=encoder.layer.9.mlp.fc1.weight]
Loading weights:  41%|████      | 179/439 [00:00<00:00, 2750.96it/s, Materializing param=encoder.layer.9.mlp.fc1.weight]
Loading weights:  41%|████      | 180/439 [00:00<00:00, 2764.08it/s, Materializing param=encoder.layer.9.mlp.fc2.bias]  
Loading weights:  41%|████      | 180/439 [00:00<00:00, 2762.85it/s, Materializing param=encoder.layer.9.mlp.fc2.bias]
Loading weights:  41%|████      | 181/439 [00:00<00:00, 2776.24it/s, Materializing param=encoder.layer.9.mlp.fc2.weight]
Loading weights:  41%|████      | 181/439 [00:00<00:00, 2775.00it/s, Materializing param=encoder.layer.9.mlp.fc2.weight]
Loading weights:  41%|████▏     | 182/439 [00:00<00:00, 2788.33it/s, Materializing param=encoder.layer.9.norm1.bias]    
Loading weights:  41%|████▏     | 182/439 [00:00<00:00, 2786.78it/s, Materializing param=encoder.layer.9.norm1.bias]
Loading weights:  42%|████▏     | 183/439 [00:00<00:00, 2799.86it/s, Materializing param=encoder.layer.9.norm1.weight]
Loading weights:  42%|████▏     | 183/439 [00:00<00:00, 2798.63it/s, Materializing param=encoder.layer.9.norm1.weight]
Loading weights:  42%|████▏     | 184/439 [00:00<00:00, 2812.03it/s, Materializing param=encoder.layer.9.norm2.bias]  
Loading weights:  42%|████▏     | 184/439 [00:00<00:00, 2810.83it/s, Materializing param=encoder.layer.9.norm2.bias]
Loading weights:  42%|████▏     | 185/439 [00:00<00:00, 2824.25it/s, Materializing param=encoder.layer.9.norm2.weight]
Loading weights:  42%|████▏     | 185/439 [00:00<00:00, 2823.01it/s, Materializing param=encoder.layer.9.norm2.weight]
Loading weights:  42%|████▏     | 186/439 [00:00<00:00, 2836.36it/s, Materializing param=encoder.layer.10.attention.attention.key.bias]
Loading weights:  42%|████▏     | 186/439 [00:00<00:00, 2834.93it/s, Materializing param=encoder.layer.10.attention.attention.key.bias]
Loading weights:  43%|████▎     | 187/439 [00:00<00:00, 2847.91it/s, Materializing param=encoder.layer.10.attention.attention.key.weight]
Loading weights:  43%|████▎     | 187/439 [00:00<00:00, 2846.56it/s, Materializing param=encoder.layer.10.attention.attention.key.weight]
Loading weights:  43%|████▎     | 188/439 [00:00<00:00, 2859.65it/s, Materializing param=encoder.layer.10.attention.attention.query.bias]
Loading weights:  43%|████▎     | 188/439 [00:00<00:00, 2858.36it/s, Materializing param=encoder.layer.10.attention.attention.query.bias]
Loading weights:  43%|████▎     | 189/439 [00:00<00:00, 2871.42it/s, Materializing param=encoder.layer.10.attention.attention.query.weight]
Loading weights:  43%|████▎     | 189/439 [00:00<00:00, 2870.02it/s, Materializing param=encoder.layer.10.attention.attention.query.weight]
Loading weights:  43%|████▎     | 190/439 [00:00<00:00, 2879.43it/s, Materializing param=encoder.layer.10.attention.attention.value.bias]  
Loading weights:  43%|████▎     | 190/439 [00:00<00:00, 2876.99it/s, Materializing param=encoder.layer.10.attention.attention.value.bias]
Loading weights:  44%|████▎     | 191/439 [00:00<00:00, 2889.17it/s, Materializing param=encoder.layer.10.attention.attention.value.weight]
Loading weights:  44%|████▎     | 191/439 [00:00<00:00, 2887.59it/s, Materializing param=encoder.layer.10.attention.attention.value.weight]
Loading weights:  44%|████▎     | 192/439 [00:00<00:00, 2900.21it/s, Materializing param=encoder.layer.10.attention.output.dense.bias]     
Loading weights:  44%|████▎     | 192/439 [00:00<00:00, 2898.68it/s, Materializing param=encoder.layer.10.attention.output.dense.bias]
Loading weights:  44%|████▍     | 193/439 [00:00<00:00, 2911.32it/s, Materializing param=encoder.layer.10.attention.output.dense.weight]
Loading weights:  44%|████▍     | 193/439 [00:00<00:00, 2909.78it/s, Materializing param=encoder.layer.10.attention.output.dense.weight]
Loading weights:  44%|████▍     | 194/439 [00:00<00:00, 2922.51it/s, Materializing param=encoder.layer.10.layer_scale1.lambda1]         
Loading weights:  44%|████▍     | 194/439 [00:00<00:00, 2921.11it/s, Materializing param=encoder.layer.10.layer_scale1.lambda1]
Loading weights:  44%|████▍     | 195/439 [00:00<00:00, 2933.56it/s, Materializing param=encoder.layer.10.layer_scale2.lambda1]
Loading weights:  44%|████▍     | 195/439 [00:00<00:00, 2932.11it/s, Materializing param=encoder.layer.10.layer_scale2.lambda1]
Loading weights:  45%|████▍     | 196/439 [00:00<00:00, 2944.62it/s, Materializing param=encoder.layer.10.mlp.fc1.bias]        
Loading weights:  45%|████▍     | 196/439 [00:00<00:00, 2943.24it/s, Materializing param=encoder.layer.10.mlp.fc1.bias]
Loading weights:  45%|████▍     | 197/439 [00:00<00:00, 2955.95it/s, Materializing param=encoder.layer.10.mlp.fc1.weight]
Loading weights:  45%|████▍     | 197/439 [00:00<00:00, 2954.01it/s, Materializing param=encoder.layer.10.mlp.fc1.weight]
Loading weights:  45%|████▌     | 198/439 [00:00<00:00, 2966.55it/s, Materializing param=encoder.layer.10.mlp.fc2.bias]  
Loading weights:  45%|████▌     | 198/439 [00:00<00:00, 2965.12it/s, Materializing param=encoder.layer.10.mlp.fc2.bias]
Loading weights:  45%|████▌     | 199/439 [00:00<00:00, 2977.84it/s, Materializing param=encoder.layer.10.mlp.fc2.weight]
Loading weights:  45%|████▌     | 199/439 [00:00<00:00, 2976.54it/s, Materializing param=encoder.layer.10.mlp.fc2.weight]
Loading weights:  46%|████▌     | 200/439 [00:00<00:00, 2989.16it/s, Materializing param=encoder.layer.10.norm1.bias]    
Loading weights:  46%|████▌     | 200/439 [00:00<00:00, 2987.74it/s, Materializing param=encoder.layer.10.norm1.bias]
Loading weights:  46%|████▌     | 201/439 [00:00<00:00, 3000.53it/s, Materializing param=encoder.layer.10.norm1.weight]
Loading weights:  46%|████▌     | 201/439 [00:00<00:00, 2999.24it/s, Materializing param=encoder.layer.10.norm1.weight]
Loading weights:  46%|████▌     | 202/439 [00:00<00:00, 3012.05it/s, Materializing param=encoder.layer.10.norm2.bias]  
Loading weights:  46%|████▌     | 202/439 [00:00<00:00, 3010.15it/s, Materializing param=encoder.layer.10.norm2.bias]
Loading weights:  46%|████▌     | 203/439 [00:00<00:00, 3022.85it/s, Materializing param=encoder.layer.10.norm2.weight]
Loading weights:  46%|████▌     | 203/439 [00:00<00:00, 3021.59it/s, Materializing param=encoder.layer.10.norm2.weight]
Loading weights:  46%|████▋     | 204/439 [00:00<00:00, 3034.36it/s, Materializing param=encoder.layer.11.attention.attention.key.bias]
Loading weights:  46%|████▋     | 204/439 [00:00<00:00, 3032.73it/s, Materializing param=encoder.layer.11.attention.attention.key.bias]
Loading weights:  47%|████▋     | 205/439 [00:00<00:00, 3044.97it/s, Materializing param=encoder.layer.11.attention.attention.key.weight]
Loading weights:  47%|████▋     | 205/439 [00:00<00:00, 3043.51it/s, Materializing param=encoder.layer.11.attention.attention.key.weight]
Loading weights:  47%|████▋     | 206/439 [00:00<00:00, 3055.96it/s, Materializing param=encoder.layer.11.attention.attention.query.bias]
Loading weights:  47%|████▋     | 206/439 [00:00<00:00, 3054.47it/s, Materializing param=encoder.layer.11.attention.attention.query.bias]
Loading weights:  47%|████▋     | 207/439 [00:00<00:00, 3066.98it/s, Materializing param=encoder.layer.11.attention.attention.query.weight]
Loading weights:  47%|████▋     | 207/439 [00:00<00:00, 3065.52it/s, Materializing param=encoder.layer.11.attention.attention.query.weight]
Loading weights:  47%|████▋     | 208/439 [00:00<00:00, 3078.05it/s, Materializing param=encoder.layer.11.attention.attention.value.bias]  
Loading weights:  47%|████▋     | 208/439 [00:00<00:00, 3076.60it/s, Materializing param=encoder.layer.11.attention.attention.value.bias]
Loading weights:  48%|████▊     | 209/439 [00:00<00:00, 3089.19it/s, Materializing param=encoder.layer.11.attention.attention.value.weight]
Loading weights:  48%|████▊     | 209/439 [00:00<00:00, 3087.77it/s, Materializing param=encoder.layer.11.attention.attention.value.weight]
Loading weights:  48%|████▊     | 210/439 [00:00<00:00, 3100.36it/s, Materializing param=encoder.layer.11.attention.output.dense.bias]     
Loading weights:  48%|████▊     | 210/439 [00:00<00:00, 3098.43it/s, Materializing param=encoder.layer.11.attention.output.dense.bias]
Loading weights:  48%|████▊     | 211/439 [00:00<00:00, 3110.67it/s, Materializing param=encoder.layer.11.attention.output.dense.weight]
Loading weights:  48%|████▊     | 211/439 [00:00<00:00, 3109.29it/s, Materializing param=encoder.layer.11.attention.output.dense.weight]
Loading weights:  48%|████▊     | 212/439 [00:00<00:00, 3120.99it/s, Materializing param=encoder.layer.11.layer_scale1.lambda1]         
Loading weights:  48%|████▊     | 212/439 [00:00<00:00, 3119.57it/s, Materializing param=encoder.layer.11.layer_scale1.lambda1]
Loading weights:  49%|████▊     | 213/439 [00:00<00:00, 3132.02it/s, Materializing param=encoder.layer.11.layer_scale2.lambda1]
Loading weights:  49%|████▊     | 213/439 [00:00<00:00, 3130.59it/s, Materializing param=encoder.layer.11.layer_scale2.lambda1]
Loading weights:  49%|████▊     | 214/439 [00:00<00:00, 3142.99it/s, Materializing param=encoder.layer.11.mlp.fc1.bias]        
Loading weights:  49%|████▊     | 214/439 [00:00<00:00, 3141.46it/s, Materializing param=encoder.layer.11.mlp.fc1.bias]
Loading weights:  49%|████▉     | 215/439 [00:00<00:00, 3153.83it/s, Materializing param=encoder.layer.11.mlp.fc1.weight]
Loading weights:  49%|████▉     | 215/439 [00:00<00:00, 3152.49it/s, Materializing param=encoder.layer.11.mlp.fc1.weight]
Loading weights:  49%|████▉     | 216/439 [00:00<00:00, 3164.68it/s, Materializing param=encoder.layer.11.mlp.fc2.bias]  
Loading weights:  49%|████▉     | 216/439 [00:00<00:00, 3163.29it/s, Materializing param=encoder.layer.11.mlp.fc2.bias]
Loading weights:  49%|████▉     | 217/439 [00:00<00:00, 3175.34it/s, Materializing param=encoder.layer.11.mlp.fc2.weight]
Loading weights:  49%|████▉     | 217/439 [00:00<00:00, 3173.89it/s, Materializing param=encoder.layer.11.mlp.fc2.weight]
Loading weights:  50%|████▉     | 218/439 [00:00<00:00, 3186.19it/s, Materializing param=encoder.layer.11.norm1.bias]    
Loading weights:  50%|████▉     | 218/439 [00:00<00:00, 3184.85it/s, Materializing param=encoder.layer.11.norm1.bias]
Loading weights:  50%|████▉     | 219/439 [00:00<00:00, 3197.21it/s, Materializing param=encoder.layer.11.norm1.weight]
Loading weights:  50%|████▉     | 219/439 [00:00<00:00, 3195.91it/s, Materializing param=encoder.layer.11.norm1.weight]
Loading weights:  50%|█████     | 220/439 [00:00<00:00, 3208.29it/s, Materializing param=encoder.layer.11.norm2.bias]  
Loading weights:  50%|█████     | 220/439 [00:00<00:00, 3206.89it/s, Materializing param=encoder.layer.11.norm2.bias]
Loading weights:  50%|█████     | 221/439 [00:00<00:00, 3219.23it/s, Materializing param=encoder.layer.11.norm2.weight]
Loading weights:  50%|█████     | 221/439 [00:00<00:00, 3217.85it/s, Materializing param=encoder.layer.11.norm2.weight]
Loading weights:  51%|█████     | 222/439 [00:00<00:00, 3230.11it/s, Materializing param=encoder.layer.12.attention.attention.key.bias]
Loading weights:  51%|█████     | 222/439 [00:00<00:00, 3228.71it/s, Materializing param=encoder.layer.12.attention.attention.key.bias]
Loading weights:  51%|█████     | 223/439 [00:00<00:00, 3240.61it/s, Materializing param=encoder.layer.12.attention.attention.key.weight]
Loading weights:  51%|█████     | 223/439 [00:00<00:00, 3239.10it/s, Materializing param=encoder.layer.12.attention.attention.key.weight]
Loading weights:  51%|█████     | 224/439 [00:00<00:00, 3251.27it/s, Materializing param=encoder.layer.12.attention.attention.query.bias]
Loading weights:  51%|█████     | 224/439 [00:00<00:00, 3249.86it/s, Materializing param=encoder.layer.12.attention.attention.query.bias]
Loading weights:  51%|█████▏    | 225/439 [00:00<00:00, 3261.89it/s, Materializing param=encoder.layer.12.attention.attention.query.weight]
Loading weights:  51%|█████▏    | 225/439 [00:00<00:00, 3260.43it/s, Materializing param=encoder.layer.12.attention.attention.query.weight]
Loading weights:  51%|█████▏    | 226/439 [00:00<00:00, 3272.37it/s, Materializing param=encoder.layer.12.attention.attention.value.bias]  
Loading weights:  51%|█████▏    | 226/439 [00:00<00:00, 3270.84it/s, Materializing param=encoder.layer.12.attention.attention.value.bias]
Loading weights:  52%|█████▏    | 227/439 [00:00<00:00, 3282.43it/s, Materializing param=encoder.layer.12.attention.attention.value.weight]
Loading weights:  52%|█████▏    | 227/439 [00:00<00:00, 3281.00it/s, Materializing param=encoder.layer.12.attention.attention.value.weight]
Loading weights:  52%|█████▏    | 228/439 [00:00<00:00, 3292.89it/s, Materializing param=encoder.layer.12.attention.output.dense.bias]     
Loading weights:  52%|█████▏    | 228/439 [00:00<00:00, 3291.36it/s, Materializing param=encoder.layer.12.attention.output.dense.bias]
Loading weights:  52%|█████▏    | 229/439 [00:00<00:00, 3303.12it/s, Materializing param=encoder.layer.12.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 229/439 [00:00<00:00, 3301.70it/s, Materializing param=encoder.layer.12.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 230/439 [00:00<00:00, 3313.79it/s, Materializing param=encoder.layer.12.layer_scale1.lambda1]         
Loading weights:  52%|█████▏    | 230/439 [00:00<00:00, 3312.31it/s, Materializing param=encoder.layer.12.layer_scale1.lambda1]
Loading weights:  53%|█████▎    | 231/439 [00:00<00:00, 3324.50it/s, Materializing param=encoder.layer.12.layer_scale2.lambda1]
Loading weights:  53%|█████▎    | 231/439 [00:00<00:00, 3323.02it/s, Materializing param=encoder.layer.12.layer_scale2.lambda1]
Loading weights:  53%|█████▎    | 232/439 [00:00<00:00, 3335.14it/s, Materializing param=encoder.layer.12.mlp.fc1.bias]        
Loading weights:  53%|█████▎    | 232/439 [00:00<00:00, 3333.66it/s, Materializing param=encoder.layer.12.mlp.fc1.bias]
Loading weights:  53%|█████▎    | 233/439 [00:00<00:00, 3345.68it/s, Materializing param=encoder.layer.12.mlp.fc1.weight]
Loading weights:  53%|█████▎    | 233/439 [00:00<00:00, 3344.19it/s, Materializing param=encoder.layer.12.mlp.fc1.weight]
Loading weights:  53%|█████▎    | 234/439 [00:00<00:00, 3356.27it/s, Materializing param=encoder.layer.12.mlp.fc2.bias]  
Loading weights:  53%|█████▎    | 234/439 [00:00<00:00, 3354.79it/s, Materializing param=encoder.layer.12.mlp.fc2.bias]
Loading weights:  54%|█████▎    | 235/439 [00:00<00:00, 3366.80it/s, Materializing param=encoder.layer.12.mlp.fc2.weight]
Loading weights:  54%|█████▎    | 235/439 [00:00<00:00, 3365.40it/s, Materializing param=encoder.layer.12.mlp.fc2.weight]
Loading weights:  54%|█████▍    | 236/439 [00:00<00:00, 3377.41it/s, Materializing param=encoder.layer.12.norm1.bias]    
Loading weights:  54%|█████▍    | 236/439 [00:00<00:00, 3375.95it/s, Materializing param=encoder.layer.12.norm1.bias]
Loading weights:  54%|█████▍    | 237/439 [00:00<00:00, 3387.89it/s, Materializing param=encoder.layer.12.norm1.weight]
Loading weights:  54%|█████▍    | 237/439 [00:00<00:00, 3386.39it/s, Materializing param=encoder.layer.12.norm1.weight]
Loading weights:  54%|█████▍    | 238/439 [00:00<00:00, 3398.53it/s, Materializing param=encoder.layer.12.norm2.bias]  
Loading weights:  54%|█████▍    | 238/439 [00:00<00:00, 3397.03it/s, Materializing param=encoder.layer.12.norm2.bias]
Loading weights:  54%|█████▍    | 239/439 [00:00<00:00, 3408.96it/s, Materializing param=encoder.layer.12.norm2.weight]
Loading weights:  54%|█████▍    | 239/439 [00:00<00:00, 3407.42it/s, Materializing param=encoder.layer.12.norm2.weight]
Loading weights:  55%|█████▍    | 240/439 [00:00<00:00, 3419.33it/s, Materializing param=encoder.layer.13.attention.attention.key.bias]
Loading weights:  55%|█████▍    | 240/439 [00:00<00:00, 3417.77it/s, Materializing param=encoder.layer.13.attention.attention.key.bias]
Loading weights:  55%|█████▍    | 241/439 [00:00<00:00, 3429.14it/s, Materializing param=encoder.layer.13.attention.attention.key.weight]
Loading weights:  55%|█████▍    | 241/439 [00:00<00:00, 3427.13it/s, Materializing param=encoder.layer.13.attention.attention.key.weight]
Loading weights:  55%|█████▌    | 242/439 [00:00<00:00, 3438.80it/s, Materializing param=encoder.layer.13.attention.attention.query.bias]
Loading weights:  55%|█████▌    | 242/439 [00:00<00:00, 3437.35it/s, Materializing param=encoder.layer.13.attention.attention.query.bias]
Loading weights:  55%|█████▌    | 243/439 [00:00<00:00, 3449.01it/s, Materializing param=encoder.layer.13.attention.attention.query.weight]
Loading weights:  55%|█████▌    | 243/439 [00:00<00:00, 3447.38it/s, Materializing param=encoder.layer.13.attention.attention.query.weight]
Loading weights:  56%|█████▌    | 244/439 [00:00<00:00, 3458.97it/s, Materializing param=encoder.layer.13.attention.attention.value.bias]  
Loading weights:  56%|█████▌    | 244/439 [00:00<00:00, 3457.41it/s, Materializing param=encoder.layer.13.attention.attention.value.bias]
Loading weights:  56%|█████▌    | 245/439 [00:00<00:00, 3469.18it/s, Materializing param=encoder.layer.13.attention.attention.value.weight]
Loading weights:  56%|█████▌    | 245/439 [00:00<00:00, 3467.65it/s, Materializing param=encoder.layer.13.attention.attention.value.weight]
Loading weights:  56%|█████▌    | 246/439 [00:00<00:00, 3479.39it/s, Materializing param=encoder.layer.13.attention.output.dense.bias]     
Loading weights:  56%|█████▌    | 246/439 [00:00<00:00, 3477.81it/s, Materializing param=encoder.layer.13.attention.output.dense.bias]
Loading weights:  56%|█████▋    | 247/439 [00:00<00:00, 3487.02it/s, Materializing param=encoder.layer.13.attention.output.dense.weight]
Loading weights:  56%|█████▋    | 247/439 [00:00<00:00, 3484.56it/s, Materializing param=encoder.layer.13.attention.output.dense.weight]
Loading weights:  56%|█████▋    | 248/439 [00:00<00:00, 3495.32it/s, Materializing param=encoder.layer.13.layer_scale1.lambda1]         
Loading weights:  56%|█████▋    | 248/439 [00:00<00:00, 3493.49it/s, Materializing param=encoder.layer.13.layer_scale1.lambda1]
Loading weights:  57%|█████▋    | 249/439 [00:00<00:00, 3504.71it/s, Materializing param=encoder.layer.13.layer_scale2.lambda1]
Loading weights:  57%|█████▋    | 249/439 [00:00<00:00, 3503.04it/s, Materializing param=encoder.layer.13.layer_scale2.lambda1]
Loading weights:  57%|█████▋    | 250/439 [00:00<00:00, 3512.91it/s, Materializing param=encoder.layer.13.mlp.fc1.bias]        
Loading weights:  57%|█████▋    | 250/439 [00:00<00:00, 3511.28it/s, Materializing param=encoder.layer.13.mlp.fc1.bias]
Loading weights:  57%|█████▋    | 251/439 [00:00<00:00, 3520.14it/s, Materializing param=encoder.layer.13.mlp.fc1.weight]
Loading weights:  57%|█████▋    | 251/439 [00:00<00:00, 3518.30it/s, Materializing param=encoder.layer.13.mlp.fc1.weight]
Loading weights:  57%|█████▋    | 252/439 [00:00<00:00, 3529.42it/s, Materializing param=encoder.layer.13.mlp.fc2.bias]  
Loading weights:  57%|█████▋    | 252/439 [00:00<00:00, 3527.67it/s, Materializing param=encoder.layer.13.mlp.fc2.bias]
Loading weights:  58%|█████▊    | 253/439 [00:00<00:00, 3539.00it/s, Materializing param=encoder.layer.13.mlp.fc2.weight]
Loading weights:  58%|█████▊    | 253/439 [00:00<00:00, 3537.42it/s, Materializing param=encoder.layer.13.mlp.fc2.weight]
Loading weights:  58%|█████▊    | 254/439 [00:00<00:00, 3548.87it/s, Materializing param=encoder.layer.13.norm1.bias]    
Loading weights:  58%|█████▊    | 254/439 [00:00<00:00, 3547.29it/s, Materializing param=encoder.layer.13.norm1.bias]
Loading weights:  58%|█████▊    | 255/439 [00:00<00:00, 3558.75it/s, Materializing param=encoder.layer.13.norm1.weight]
Loading weights:  58%|█████▊    | 255/439 [00:00<00:00, 3557.17it/s, Materializing param=encoder.layer.13.norm1.weight]
Loading weights:  58%|█████▊    | 256/439 [00:00<00:00, 3568.03it/s, Materializing param=encoder.layer.13.norm2.bias]  
Loading weights:  58%|█████▊    | 256/439 [00:00<00:00, 3566.44it/s, Materializing param=encoder.layer.13.norm2.bias]
Loading weights:  59%|█████▊    | 257/439 [00:00<00:00, 3577.99it/s, Materializing param=encoder.layer.13.norm2.weight]
Loading weights:  59%|█████▊    | 257/439 [00:00<00:00, 3576.44it/s, Materializing param=encoder.layer.13.norm2.weight]
Loading weights:  59%|█████▉    | 258/439 [00:00<00:00, 3587.75it/s, Materializing param=encoder.layer.14.attention.attention.key.bias]
Loading weights:  59%|█████▉    | 258/439 [00:00<00:00, 3586.08it/s, Materializing param=encoder.layer.14.attention.attention.key.bias]
Loading weights:  59%|█████▉    | 259/439 [00:00<00:00, 3597.02it/s, Materializing param=encoder.layer.14.attention.attention.key.weight]
Loading weights:  59%|█████▉    | 259/439 [00:00<00:00, 3595.32it/s, Materializing param=encoder.layer.14.attention.attention.key.weight]
Loading weights:  59%|█████▉    | 260/439 [00:00<00:00, 3605.55it/s, Materializing param=encoder.layer.14.attention.attention.query.bias]
Loading weights:  59%|█████▉    | 260/439 [00:00<00:00, 3602.89it/s, Materializing param=encoder.layer.14.attention.attention.query.bias]
Loading weights:  59%|█████▉    | 261/439 [00:00<00:00, 3613.36it/s, Materializing param=encoder.layer.14.attention.attention.query.weight]
Loading weights:  59%|█████▉    | 261/439 [00:00<00:00, 3611.70it/s, Materializing param=encoder.layer.14.attention.attention.query.weight]
Loading weights:  60%|█████▉    | 262/439 [00:00<00:00, 3622.84it/s, Materializing param=encoder.layer.14.attention.attention.value.bias]  
Loading weights:  60%|█████▉    | 262/439 [00:00<00:00, 3621.13it/s, Materializing param=encoder.layer.14.attention.attention.value.bias]
Loading weights:  60%|█████▉    | 263/439 [00:00<00:00, 3632.23it/s, Materializing param=encoder.layer.14.attention.attention.value.weight]
Loading weights:  60%|█████▉    | 263/439 [00:00<00:00, 3630.69it/s, Materializing param=encoder.layer.14.attention.attention.value.weight]
Loading weights:  60%|██████    | 264/439 [00:00<00:00, 3641.82it/s, Materializing param=encoder.layer.14.attention.output.dense.bias]     
Loading weights:  60%|██████    | 264/439 [00:00<00:00, 3640.22it/s, Materializing param=encoder.layer.14.attention.output.dense.bias]
Loading weights:  60%|██████    | 265/439 [00:00<00:00, 3651.53it/s, Materializing param=encoder.layer.14.attention.output.dense.weight]
Loading weights:  60%|██████    | 265/439 [00:00<00:00, 3649.87it/s, Materializing param=encoder.layer.14.attention.output.dense.weight]
Loading weights:  61%|██████    | 266/439 [00:00<00:00, 3661.24it/s, Materializing param=encoder.layer.14.layer_scale1.lambda1]         
Loading weights:  61%|██████    | 266/439 [00:00<00:00, 3659.68it/s, Materializing param=encoder.layer.14.layer_scale1.lambda1]
Loading weights:  61%|██████    | 267/439 [00:00<00:00, 3670.65it/s, Materializing param=encoder.layer.14.layer_scale2.lambda1]
Loading weights:  61%|██████    | 267/439 [00:00<00:00, 3669.14it/s, Materializing param=encoder.layer.14.layer_scale2.lambda1]
Loading weights:  61%|██████    | 268/439 [00:00<00:00, 3680.30it/s, Materializing param=encoder.layer.14.mlp.fc1.bias]        
Loading weights:  61%|██████    | 268/439 [00:00<00:00, 3678.74it/s, Materializing param=encoder.layer.14.mlp.fc1.bias]
Loading weights:  61%|██████▏   | 269/439 [00:00<00:00, 3689.48it/s, Materializing param=encoder.layer.14.mlp.fc1.weight]
Loading weights:  61%|██████▏   | 269/439 [00:00<00:00, 3687.91it/s, Materializing param=encoder.layer.14.mlp.fc1.weight]
Loading weights:  62%|██████▏   | 270/439 [00:00<00:00, 3699.18it/s, Materializing param=encoder.layer.14.mlp.fc2.bias]  
Loading weights:  62%|██████▏   | 270/439 [00:00<00:00, 3697.57it/s, Materializing param=encoder.layer.14.mlp.fc2.bias]
Loading weights:  62%|██████▏   | 271/439 [00:00<00:00, 3708.87it/s, Materializing param=encoder.layer.14.mlp.fc2.weight]
Loading weights:  62%|██████▏   | 271/439 [00:00<00:00, 3707.34it/s, Materializing param=encoder.layer.14.mlp.fc2.weight]
Loading weights:  62%|██████▏   | 272/439 [00:00<00:00, 3718.58it/s, Materializing param=encoder.layer.14.norm1.bias]    
Loading weights:  62%|██████▏   | 272/439 [00:00<00:00, 3717.06it/s, Materializing param=encoder.layer.14.norm1.bias]
Loading weights:  62%|██████▏   | 273/439 [00:00<00:00, 3728.38it/s, Materializing param=encoder.layer.14.norm1.weight]
Loading weights:  62%|██████▏   | 273/439 [00:00<00:00, 3726.85it/s, Materializing param=encoder.layer.14.norm1.weight]
Loading weights:  62%|██████▏   | 274/439 [00:00<00:00, 3738.12it/s, Materializing param=encoder.layer.14.norm2.bias]  
Loading weights:  62%|██████▏   | 274/439 [00:00<00:00, 3736.53it/s, Materializing param=encoder.layer.14.norm2.bias]
Loading weights:  63%|██████▎   | 275/439 [00:00<00:00, 3747.76it/s, Materializing param=encoder.layer.14.norm2.weight]
Loading weights:  63%|██████▎   | 275/439 [00:00<00:00, 3746.29it/s, Materializing param=encoder.layer.14.norm2.weight]
Loading weights:  63%|██████▎   | 276/439 [00:00<00:00, 3757.50it/s, Materializing param=encoder.layer.15.attention.attention.key.bias]
Loading weights:  63%|██████▎   | 276/439 [00:00<00:00, 3755.91it/s, Materializing param=encoder.layer.15.attention.attention.key.bias]
Loading weights:  63%|██████▎   | 277/439 [00:00<00:00, 3766.92it/s, Materializing param=encoder.layer.15.attention.attention.key.weight]
Loading weights:  63%|██████▎   | 277/439 [00:00<00:00, 3765.33it/s, Materializing param=encoder.layer.15.attention.attention.key.weight]
Loading weights:  63%|██████▎   | 278/439 [00:00<00:00, 3776.39it/s, Materializing param=encoder.layer.15.attention.attention.query.bias]
Loading weights:  63%|██████▎   | 278/439 [00:00<00:00, 3774.81it/s, Materializing param=encoder.layer.15.attention.attention.query.bias]
Loading weights:  64%|██████▎   | 279/439 [00:00<00:00, 3785.93it/s, Materializing param=encoder.layer.15.attention.attention.query.weight]
Loading weights:  64%|██████▎   | 279/439 [00:00<00:00, 3784.22it/s, Materializing param=encoder.layer.15.attention.attention.query.weight]
Loading weights:  64%|██████▍   | 280/439 [00:00<00:00, 3795.31it/s, Materializing param=encoder.layer.15.attention.attention.value.bias]  
Loading weights:  64%|██████▍   | 280/439 [00:00<00:00, 3793.71it/s, Materializing param=encoder.layer.15.attention.attention.value.bias]
Loading weights:  64%|██████▍   | 281/439 [00:00<00:00, 3804.74it/s, Materializing param=encoder.layer.15.attention.attention.value.weight]
Loading weights:  64%|██████▍   | 281/439 [00:00<00:00, 3803.15it/s, Materializing param=encoder.layer.15.attention.attention.value.weight]
Loading weights:  64%|██████▍   | 282/439 [00:00<00:00, 3814.10it/s, Materializing param=encoder.layer.15.attention.output.dense.bias]     
Loading weights:  64%|██████▍   | 282/439 [00:00<00:00, 3812.55it/s, Materializing param=encoder.layer.15.attention.output.dense.bias]
Loading weights:  64%|██████▍   | 283/439 [00:00<00:00, 3823.54it/s, Materializing param=encoder.layer.15.attention.output.dense.weight]
Loading weights:  64%|██████▍   | 283/439 [00:00<00:00, 3821.89it/s, Materializing param=encoder.layer.15.attention.output.dense.weight]
Loading weights:  65%|██████▍   | 284/439 [00:00<00:00, 3832.90it/s, Materializing param=encoder.layer.15.layer_scale1.lambda1]         
Loading weights:  65%|██████▍   | 284/439 [00:00<00:00, 3831.30it/s, Materializing param=encoder.layer.15.layer_scale1.lambda1]
Loading weights:  65%|██████▍   | 285/439 [00:00<00:00, 3842.37it/s, Materializing param=encoder.layer.15.layer_scale2.lambda1]
Loading weights:  65%|██████▍   | 285/439 [00:00<00:00, 3840.69it/s, Materializing param=encoder.layer.15.layer_scale2.lambda1]
Loading weights:  65%|██████▌   | 286/439 [00:00<00:00, 3851.42it/s, Materializing param=encoder.layer.15.mlp.fc1.bias]        
Loading weights:  65%|██████▌   | 286/439 [00:00<00:00, 3849.81it/s, Materializing param=encoder.layer.15.mlp.fc1.bias]
Loading weights:  65%|██████▌   | 287/439 [00:00<00:00, 3860.57it/s, Materializing param=encoder.layer.15.mlp.fc1.weight]
Loading weights:  65%|██████▌   | 287/439 [00:00<00:00, 3858.91it/s, Materializing param=encoder.layer.15.mlp.fc1.weight]
Loading weights:  66%|██████▌   | 288/439 [00:00<00:00, 3869.87it/s, Materializing param=encoder.layer.15.mlp.fc2.bias]  
Loading weights:  66%|██████▌   | 288/439 [00:00<00:00, 3868.00it/s, Materializing param=encoder.layer.15.mlp.fc2.bias]
Loading weights:  66%|██████▌   | 289/439 [00:00<00:00, 3878.67it/s, Materializing param=encoder.layer.15.mlp.fc2.weight]
Loading weights:  66%|██████▌   | 289/439 [00:00<00:00, 3877.06it/s, Materializing param=encoder.layer.15.mlp.fc2.weight]
Loading weights:  66%|██████▌   | 290/439 [00:00<00:00, 3887.96it/s, Materializing param=encoder.layer.15.norm1.bias]    
Loading weights:  66%|██████▌   | 290/439 [00:00<00:00, 3886.34it/s, Materializing param=encoder.layer.15.norm1.bias]
Loading weights:  66%|██████▋   | 291/439 [00:00<00:00, 3897.29it/s, Materializing param=encoder.layer.15.norm1.weight]
Loading weights:  66%|██████▋   | 291/439 [00:00<00:00, 3895.74it/s, Materializing param=encoder.layer.15.norm1.weight]
Loading weights:  67%|██████▋   | 292/439 [00:00<00:00, 3906.66it/s, Materializing param=encoder.layer.15.norm2.bias]  
Loading weights:  67%|██████▋   | 292/439 [00:00<00:00, 3905.10it/s, Materializing param=encoder.layer.15.norm2.bias]
Loading weights:  67%|██████▋   | 293/439 [00:00<00:00, 3916.05it/s, Materializing param=encoder.layer.15.norm2.weight]
Loading weights:  67%|██████▋   | 293/439 [00:00<00:00, 3914.43it/s, Materializing param=encoder.layer.15.norm2.weight]
Loading weights:  67%|██████▋   | 294/439 [00:00<00:00, 3925.38it/s, Materializing param=encoder.layer.16.attention.attention.key.bias]
Loading weights:  67%|██████▋   | 294/439 [00:00<00:00, 3923.75it/s, Materializing param=encoder.layer.16.attention.attention.key.bias]
Loading weights:  67%|██████▋   | 295/439 [00:00<00:00, 3933.43it/s, Materializing param=encoder.layer.16.attention.attention.key.weight]
Loading weights:  67%|██████▋   | 295/439 [00:00<00:00, 3931.71it/s, Materializing param=encoder.layer.16.attention.attention.key.weight]
Loading weights:  67%|██████▋   | 296/439 [00:00<00:00, 3942.47it/s, Materializing param=encoder.layer.16.attention.attention.query.bias]
Loading weights:  67%|██████▋   | 296/439 [00:00<00:00, 3940.73it/s, Materializing param=encoder.layer.16.attention.attention.query.bias]
Loading weights:  68%|██████▊   | 297/439 [00:00<00:00, 3951.46it/s, Materializing param=encoder.layer.16.attention.attention.query.weight]
Loading weights:  68%|██████▊   | 297/439 [00:00<00:00, 3949.83it/s, Materializing param=encoder.layer.16.attention.attention.query.weight]
Loading weights:  68%|██████▊   | 298/439 [00:00<00:00, 3960.39it/s, Materializing param=encoder.layer.16.attention.attention.value.bias]  
Loading weights:  68%|██████▊   | 298/439 [00:00<00:00, 3958.76it/s, Materializing param=encoder.layer.16.attention.attention.value.bias]
Loading weights:  68%|██████▊   | 299/439 [00:00<00:00, 3969.35it/s, Materializing param=encoder.layer.16.attention.attention.value.weight]
Loading weights:  68%|██████▊   | 299/439 [00:00<00:00, 3967.67it/s, Materializing param=encoder.layer.16.attention.attention.value.weight]
Loading weights:  68%|██████▊   | 300/439 [00:00<00:00, 3978.30it/s, Materializing param=encoder.layer.16.attention.output.dense.bias]     
Loading weights:  68%|██████▊   | 300/439 [00:00<00:00, 3976.60it/s, Materializing param=encoder.layer.16.attention.output.dense.bias]
Loading weights:  69%|██████▊   | 301/439 [00:00<00:00, 3986.97it/s, Materializing param=encoder.layer.16.attention.output.dense.weight]
Loading weights:  69%|██████▊   | 301/439 [00:00<00:00, 3985.27it/s, Materializing param=encoder.layer.16.attention.output.dense.weight]
Loading weights:  69%|██████▉   | 302/439 [00:00<00:00, 3995.86it/s, Materializing param=encoder.layer.16.layer_scale1.lambda1]         
Loading weights:  69%|██████▉   | 302/439 [00:00<00:00, 3994.22it/s, Materializing param=encoder.layer.16.layer_scale1.lambda1]
Loading weights:  69%|██████▉   | 303/439 [00:00<00:00, 4004.75it/s, Materializing param=encoder.layer.16.layer_scale2.lambda1]
Loading weights:  69%|██████▉   | 303/439 [00:00<00:00, 4003.17it/s, Materializing param=encoder.layer.16.layer_scale2.lambda1]
Loading weights:  69%|██████▉   | 304/439 [00:00<00:00, 4013.62it/s, Materializing param=encoder.layer.16.mlp.fc1.bias]        
Loading weights:  69%|██████▉   | 304/439 [00:00<00:00, 4011.98it/s, Materializing param=encoder.layer.16.mlp.fc1.bias]
Loading weights:  69%|██████▉   | 305/439 [00:00<00:00, 4021.68it/s, Materializing param=encoder.layer.16.mlp.fc1.weight]
Loading weights:  69%|██████▉   | 305/439 [00:00<00:00, 4019.39it/s, Materializing param=encoder.layer.16.mlp.fc1.weight]
Loading weights:  70%|██████▉   | 306/439 [00:00<00:00, 4029.12it/s, Materializing param=encoder.layer.16.mlp.fc2.bias]  
Loading weights:  70%|██████▉   | 306/439 [00:00<00:00, 4027.26it/s, Materializing param=encoder.layer.16.mlp.fc2.bias]
Loading weights:  70%|██████▉   | 307/439 [00:00<00:00, 4037.50it/s, Materializing param=encoder.layer.16.mlp.fc2.weight]
Loading weights:  70%|██████▉   | 307/439 [00:00<00:00, 4035.69it/s, Materializing param=encoder.layer.16.mlp.fc2.weight]
Loading weights:  70%|███████   | 308/439 [00:00<00:00, 4046.07it/s, Materializing param=encoder.layer.16.norm1.bias]    
Loading weights:  70%|███████   | 308/439 [00:00<00:00, 4044.31it/s, Materializing param=encoder.layer.16.norm1.bias]
Loading weights:  70%|███████   | 309/439 [00:00<00:00, 4054.74it/s, Materializing param=encoder.layer.16.norm1.weight]
Loading weights:  70%|███████   | 309/439 [00:00<00:00, 4052.97it/s, Materializing param=encoder.layer.16.norm1.weight]
Loading weights:  71%|███████   | 310/439 [00:00<00:00, 4063.55it/s, Materializing param=encoder.layer.16.norm2.bias]  
Loading weights:  71%|███████   | 310/439 [00:00<00:00, 4061.90it/s, Materializing param=encoder.layer.16.norm2.bias]
Loading weights:  71%|███████   | 311/439 [00:00<00:00, 4072.43it/s, Materializing param=encoder.layer.16.norm2.weight]
Loading weights:  71%|███████   | 311/439 [00:00<00:00, 4070.83it/s, Materializing param=encoder.layer.16.norm2.weight]
Loading weights:  71%|███████   | 312/439 [00:00<00:00, 4081.46it/s, Materializing param=encoder.layer.17.attention.attention.key.bias]
Loading weights:  71%|███████   | 312/439 [00:00<00:00, 4079.76it/s, Materializing param=encoder.layer.17.attention.attention.key.bias]
Loading weights:  71%|███████▏  | 313/439 [00:00<00:00, 4089.68it/s, Materializing param=encoder.layer.17.attention.attention.key.weight]
Loading weights:  71%|███████▏  | 313/439 [00:00<00:00, 4088.03it/s, Materializing param=encoder.layer.17.attention.attention.key.weight]
Loading weights:  72%|███████▏  | 314/439 [00:00<00:00, 4098.36it/s, Materializing param=encoder.layer.17.attention.attention.query.bias]
Loading weights:  72%|███████▏  | 314/439 [00:00<00:00, 4096.69it/s, Materializing param=encoder.layer.17.attention.attention.query.bias]
Loading weights:  72%|███████▏  | 315/439 [00:00<00:00, 4106.90it/s, Materializing param=encoder.layer.17.attention.attention.query.weight]
Loading weights:  72%|███████▏  | 315/439 [00:00<00:00, 4105.19it/s, Materializing param=encoder.layer.17.attention.attention.query.weight]
Loading weights:  72%|███████▏  | 316/439 [00:00<00:00, 4115.43it/s, Materializing param=encoder.layer.17.attention.attention.value.bias]  
Loading weights:  72%|███████▏  | 316/439 [00:00<00:00, 4113.77it/s, Materializing param=encoder.layer.17.attention.attention.value.bias]
Loading weights:  72%|███████▏  | 317/439 [00:00<00:00, 4124.22it/s, Materializing param=encoder.layer.17.attention.attention.value.weight]
Loading weights:  72%|███████▏  | 317/439 [00:00<00:00, 4122.50it/s, Materializing param=encoder.layer.17.attention.attention.value.weight]
Loading weights:  72%|███████▏  | 318/439 [00:00<00:00, 4132.81it/s, Materializing param=encoder.layer.17.attention.output.dense.bias]     
Loading weights:  72%|███████▏  | 318/439 [00:00<00:00, 4131.04it/s, Materializing param=encoder.layer.17.attention.output.dense.bias]
Loading weights:  73%|███████▎  | 319/439 [00:00<00:00, 4141.23it/s, Materializing param=encoder.layer.17.attention.output.dense.weight]
Loading weights:  73%|███████▎  | 319/439 [00:00<00:00, 4139.45it/s, Materializing param=encoder.layer.17.attention.output.dense.weight]
Loading weights:  73%|███████▎  | 320/439 [00:00<00:00, 4149.75it/s, Materializing param=encoder.layer.17.layer_scale1.lambda1]         
Loading weights:  73%|███████▎  | 320/439 [00:00<00:00, 4148.03it/s, Materializing param=encoder.layer.17.layer_scale1.lambda1]
Loading weights:  73%|███████▎  | 321/439 [00:00<00:00, 4158.36it/s, Materializing param=encoder.layer.17.layer_scale2.lambda1]
Loading weights:  73%|███████▎  | 321/439 [00:00<00:00, 4156.62it/s, Materializing param=encoder.layer.17.layer_scale2.lambda1]
Loading weights:  73%|███████▎  | 322/439 [00:00<00:00, 4166.45it/s, Materializing param=encoder.layer.17.mlp.fc1.bias]        
Loading weights:  73%|███████▎  | 322/439 [00:00<00:00, 4164.83it/s, Materializing param=encoder.layer.17.mlp.fc1.bias]
Loading weights:  74%|███████▎  | 323/439 [00:00<00:00, 4175.01it/s, Materializing param=encoder.layer.17.mlp.fc1.weight]
Loading weights:  74%|███████▎  | 323/439 [00:00<00:00, 4173.33it/s, Materializing param=encoder.layer.17.mlp.fc1.weight]
Loading weights:  74%|███████▍  | 324/439 [00:00<00:00, 4183.65it/s, Materializing param=encoder.layer.17.mlp.fc2.bias]  
Loading weights:  74%|███████▍  | 324/439 [00:00<00:00, 4182.09it/s, Materializing param=encoder.layer.17.mlp.fc2.bias]
Loading weights:  74%|███████▍  | 325/439 [00:00<00:00, 4192.34it/s, Materializing param=encoder.layer.17.mlp.fc2.weight]
Loading weights:  74%|███████▍  | 325/439 [00:00<00:00, 4190.72it/s, Materializing param=encoder.layer.17.mlp.fc2.weight]
Loading weights:  74%|███████▍  | 326/439 [00:00<00:00, 4200.80it/s, Materializing param=encoder.layer.17.norm1.bias]    
Loading weights:  74%|███████▍  | 326/439 [00:00<00:00, 4199.19it/s, Materializing param=encoder.layer.17.norm1.bias]
Loading weights:  74%|███████▍  | 327/439 [00:00<00:00, 4209.51it/s, Materializing param=encoder.layer.17.norm1.weight]
Loading weights:  74%|███████▍  | 327/439 [00:00<00:00, 4207.78it/s, Materializing param=encoder.layer.17.norm1.weight]
Loading weights:  75%|███████▍  | 328/439 [00:00<00:00, 4218.04it/s, Materializing param=encoder.layer.17.norm2.bias]  
Loading weights:  75%|███████▍  | 328/439 [00:00<00:00, 4216.48it/s, Materializing param=encoder.layer.17.norm2.bias]
Loading weights:  75%|███████▍  | 329/439 [00:00<00:00, 4226.72it/s, Materializing param=encoder.layer.17.norm2.weight]
Loading weights:  75%|███████▍  | 329/439 [00:00<00:00, 4225.04it/s, Materializing param=encoder.layer.17.norm2.weight]
Loading weights:  75%|███████▌  | 330/439 [00:00<00:00, 4235.43it/s, Materializing param=encoder.layer.18.attention.attention.key.bias]
Loading weights:  75%|███████▌  | 330/439 [00:00<00:00, 4233.69it/s, Materializing param=encoder.layer.18.attention.attention.key.bias]
Loading weights:  75%|███████▌  | 331/439 [00:00<00:00, 4243.52it/s, Materializing param=encoder.layer.18.attention.attention.key.weight]
Loading weights:  75%|███████▌  | 331/439 [00:00<00:00, 4241.84it/s, Materializing param=encoder.layer.18.attention.attention.key.weight]
Loading weights:  76%|███████▌  | 332/439 [00:00<00:00, 4251.94it/s, Materializing param=encoder.layer.18.attention.attention.query.bias]
Loading weights:  76%|███████▌  | 332/439 [00:00<00:00, 4250.08it/s, Materializing param=encoder.layer.18.attention.attention.query.bias]
Loading weights:  76%|███████▌  | 333/439 [00:00<00:00, 4260.10it/s, Materializing param=encoder.layer.18.attention.attention.query.weight]
Loading weights:  76%|███████▌  | 333/439 [00:00<00:00, 4258.31it/s, Materializing param=encoder.layer.18.attention.attention.query.weight]
Loading weights:  76%|███████▌  | 334/439 [00:00<00:00, 4268.36it/s, Materializing param=encoder.layer.18.attention.attention.value.bias]  
Loading weights:  76%|███████▌  | 334/439 [00:00<00:00, 4266.62it/s, Materializing param=encoder.layer.18.attention.attention.value.bias]
Loading weights:  76%|███████▋  | 335/439 [00:00<00:00, 4276.61it/s, Materializing param=encoder.layer.18.attention.attention.value.weight]
Loading weights:  76%|███████▋  | 335/439 [00:00<00:00, 4274.81it/s, Materializing param=encoder.layer.18.attention.attention.value.weight]
Loading weights:  77%|███████▋  | 336/439 [00:00<00:00, 4284.39it/s, Materializing param=encoder.layer.18.attention.output.dense.bias]     
Loading weights:  77%|███████▋  | 336/439 [00:00<00:00, 4282.70it/s, Materializing param=encoder.layer.18.attention.output.dense.bias]
Loading weights:  77%|███████▋  | 337/439 [00:00<00:00, 4292.72it/s, Materializing param=encoder.layer.18.attention.output.dense.weight]
Loading weights:  77%|███████▋  | 337/439 [00:00<00:00, 4290.91it/s, Materializing param=encoder.layer.18.attention.output.dense.weight]
Loading weights:  77%|███████▋  | 338/439 [00:00<00:00, 4300.95it/s, Materializing param=encoder.layer.18.layer_scale1.lambda1]         
Loading weights:  77%|███████▋  | 338/439 [00:00<00:00, 4299.20it/s, Materializing param=encoder.layer.18.layer_scale1.lambda1]
Loading weights:  77%|███████▋  | 339/439 [00:00<00:00, 4309.36it/s, Materializing param=encoder.layer.18.layer_scale2.lambda1]
Loading weights:  77%|███████▋  | 339/439 [00:00<00:00, 4307.60it/s, Materializing param=encoder.layer.18.layer_scale2.lambda1]
Loading weights:  77%|███████▋  | 340/439 [00:00<00:00, 4317.23it/s, Materializing param=encoder.layer.18.mlp.fc1.bias]        
Loading weights:  77%|███████▋  | 340/439 [00:00<00:00, 4315.53it/s, Materializing param=encoder.layer.18.mlp.fc1.bias]
Loading weights:  78%|███████▊  | 341/439 [00:00<00:00, 4325.37it/s, Materializing param=encoder.layer.18.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 341/439 [00:00<00:00, 4323.78it/s, Materializing param=encoder.layer.18.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 342/439 [00:00<00:00, 4333.94it/s, Materializing param=encoder.layer.18.mlp.fc2.bias]  
Loading weights:  78%|███████▊  | 342/439 [00:00<00:00, 4332.24it/s, Materializing param=encoder.layer.18.mlp.fc2.bias]
Loading weights:  78%|███████▊  | 343/439 [00:00<00:00, 4342.20it/s, Materializing param=encoder.layer.18.mlp.fc2.weight]
Loading weights:  78%|███████▊  | 343/439 [00:00<00:00, 4340.61it/s, Materializing param=encoder.layer.18.mlp.fc2.weight]
Loading weights:  78%|███████▊  | 344/439 [00:00<00:00, 4350.73it/s, Materializing param=encoder.layer.18.norm1.bias]    
Loading weights:  78%|███████▊  | 344/439 [00:00<00:00, 4349.08it/s, Materializing param=encoder.layer.18.norm1.bias]
Loading weights:  79%|███████▊  | 345/439 [00:00<00:00, 4359.14it/s, Materializing param=encoder.layer.18.norm1.weight]
Loading weights:  79%|███████▊  | 345/439 [00:00<00:00, 4357.42it/s, Materializing param=encoder.layer.18.norm1.weight]
Loading weights:  79%|███████▉  | 346/439 [00:00<00:00, 4367.58it/s, Materializing param=encoder.layer.18.norm2.bias]  
Loading weights:  79%|███████▉  | 346/439 [00:00<00:00, 4365.87it/s, Materializing param=encoder.layer.18.norm2.bias]
Loading weights:  79%|███████▉  | 347/439 [00:00<00:00, 4375.88it/s, Materializing param=encoder.layer.18.norm2.weight]
Loading weights:  79%|███████▉  | 347/439 [00:00<00:00, 4374.19it/s, Materializing param=encoder.layer.18.norm2.weight]
Loading weights:  79%|███████▉  | 348/439 [00:00<00:00, 4384.24it/s, Materializing param=encoder.layer.19.attention.attention.key.bias]
Loading weights:  79%|███████▉  | 348/439 [00:00<00:00, 4382.53it/s, Materializing param=encoder.layer.19.attention.attention.key.bias]
Loading weights:  79%|███████▉  | 349/439 [00:00<00:00, 4392.42it/s, Materializing param=encoder.layer.19.attention.attention.key.weight]
Loading weights:  79%|███████▉  | 349/439 [00:00<00:00, 4390.65it/s, Materializing param=encoder.layer.19.attention.attention.key.weight]
Loading weights:  80%|███████▉  | 350/439 [00:00<00:00, 4400.51it/s, Materializing param=encoder.layer.19.attention.attention.query.bias]
Loading weights:  80%|███████▉  | 350/439 [00:00<00:00, 4398.75it/s, Materializing param=encoder.layer.19.attention.attention.query.bias]
Loading weights:  80%|███████▉  | 351/439 [00:00<00:00, 4408.54it/s, Materializing param=encoder.layer.19.attention.attention.query.weight]
Loading weights:  80%|███████▉  | 351/439 [00:00<00:00, 4406.77it/s, Materializing param=encoder.layer.19.attention.attention.query.weight]
Loading weights:  80%|████████  | 352/439 [00:00<00:00, 4416.62it/s, Materializing param=encoder.layer.19.attention.attention.value.bias]  
Loading weights:  80%|████████  | 352/439 [00:00<00:00, 4414.78it/s, Materializing param=encoder.layer.19.attention.attention.value.bias]
Loading weights:  80%|████████  | 353/439 [00:00<00:00, 4424.33it/s, Materializing param=encoder.layer.19.attention.attention.value.weight]
Loading weights:  80%|████████  | 353/439 [00:00<00:00, 4421.78it/s, Materializing param=encoder.layer.19.attention.attention.value.weight]
Loading weights:  81%|████████  | 354/439 [00:00<00:00, 4430.70it/s, Materializing param=encoder.layer.19.attention.output.dense.bias]     
Loading weights:  81%|████████  | 354/439 [00:00<00:00, 4428.75it/s, Materializing param=encoder.layer.19.attention.output.dense.bias]
Loading weights:  81%|████████  | 355/439 [00:00<00:00, 4438.10it/s, Materializing param=encoder.layer.19.attention.output.dense.weight]
Loading weights:  81%|████████  | 355/439 [00:00<00:00, 4436.27it/s, Materializing param=encoder.layer.19.attention.output.dense.weight]
Loading weights:  81%|████████  | 356/439 [00:00<00:00, 4445.88it/s, Materializing param=encoder.layer.19.layer_scale1.lambda1]         
Loading weights:  81%|████████  | 356/439 [00:00<00:00, 4444.11it/s, Materializing param=encoder.layer.19.layer_scale1.lambda1]
Loading weights:  81%|████████▏ | 357/439 [00:00<00:00, 4453.86it/s, Materializing param=encoder.layer.19.layer_scale2.lambda1]
Loading weights:  81%|████████▏ | 357/439 [00:00<00:00, 4452.08it/s, Materializing param=encoder.layer.19.layer_scale2.lambda1]
Loading weights:  82%|████████▏ | 358/439 [00:00<00:00, 4461.89it/s, Materializing param=encoder.layer.19.mlp.fc1.bias]        
Loading weights:  82%|████████▏ | 358/439 [00:00<00:00, 4460.22it/s, Materializing param=encoder.layer.19.mlp.fc1.bias]
Loading weights:  82%|████████▏ | 359/439 [00:00<00:00, 4470.00it/s, Materializing param=encoder.layer.19.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 359/439 [00:00<00:00, 4468.33it/s, Materializing param=encoder.layer.19.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 360/439 [00:00<00:00, 4478.10it/s, Materializing param=encoder.layer.19.mlp.fc2.bias]  
Loading weights:  82%|████████▏ | 360/439 [00:00<00:00, 4475.49it/s, Materializing param=encoder.layer.19.mlp.fc2.bias]
Loading weights:  82%|████████▏ | 361/439 [00:00<00:00, 4484.68it/s, Materializing param=encoder.layer.19.mlp.fc2.weight]
Loading weights:  82%|████████▏ | 361/439 [00:00<00:00, 4482.95it/s, Materializing param=encoder.layer.19.mlp.fc2.weight]
Loading weights:  82%|████████▏ | 362/439 [00:00<00:00, 4492.76it/s, Materializing param=encoder.layer.19.norm1.bias]    
Loading weights:  82%|████████▏ | 362/439 [00:00<00:00, 4490.86it/s, Materializing param=encoder.layer.19.norm1.bias]
Loading weights:  83%|████████▎ | 363/439 [00:00<00:00, 4499.14it/s, Materializing param=encoder.layer.19.norm1.weight]
Loading weights:  83%|████████▎ | 363/439 [00:00<00:00, 4497.02it/s, Materializing param=encoder.layer.19.norm1.weight]
Loading weights:  83%|████████▎ | 364/439 [00:00<00:00, 4506.33it/s, Materializing param=encoder.layer.19.norm2.bias]  
Loading weights:  83%|████████▎ | 364/439 [00:00<00:00, 4504.51it/s, Materializing param=encoder.layer.19.norm2.bias]
Loading weights:  83%|████████▎ | 365/439 [00:00<00:00, 4513.63it/s, Materializing param=encoder.layer.19.norm2.weight]
Loading weights:  83%|████████▎ | 365/439 [00:00<00:00, 4511.61it/s, Materializing param=encoder.layer.19.norm2.weight]
Loading weights:  83%|████████▎ | 366/439 [00:00<00:00, 4520.86it/s, Materializing param=encoder.layer.20.attention.attention.key.bias]
Loading weights:  83%|████████▎ | 366/439 [00:00<00:00, 4518.85it/s, Materializing param=encoder.layer.20.attention.attention.key.bias]
Loading weights:  84%|████████▎ | 367/439 [00:00<00:00, 4527.73it/s, Materializing param=encoder.layer.20.attention.attention.key.weight]
Loading weights:  84%|████████▎ | 367/439 [00:00<00:00, 4525.60it/s, Materializing param=encoder.layer.20.attention.attention.key.weight]
Loading weights:  84%|████████▍ | 368/439 [00:00<00:00, 4534.76it/s, Materializing param=encoder.layer.20.attention.attention.query.bias]
Loading weights:  84%|████████▍ | 368/439 [00:00<00:00, 4532.73it/s, Materializing param=encoder.layer.20.attention.attention.query.bias]
Loading weights:  84%|████████▍ | 369/439 [00:00<00:00, 4541.53it/s, Materializing param=encoder.layer.20.attention.attention.query.weight]
Loading weights:  84%|████████▍ | 369/439 [00:00<00:00, 4539.52it/s, Materializing param=encoder.layer.20.attention.attention.query.weight]
Loading weights:  84%|████████▍ | 370/439 [00:00<00:00, 4548.58it/s, Materializing param=encoder.layer.20.attention.attention.value.bias]  
Loading weights:  84%|████████▍ | 370/439 [00:00<00:00, 4546.56it/s, Materializing param=encoder.layer.20.attention.attention.value.bias]
Loading weights:  85%|████████▍ | 371/439 [00:00<00:00, 4555.77it/s, Materializing param=encoder.layer.20.attention.attention.value.weight]
Loading weights:  85%|████████▍ | 371/439 [00:00<00:00, 4553.77it/s, Materializing param=encoder.layer.20.attention.attention.value.weight]
Loading weights:  85%|████████▍ | 372/439 [00:00<00:00, 4563.01it/s, Materializing param=encoder.layer.20.attention.output.dense.bias]     
Loading weights:  85%|████████▍ | 372/439 [00:00<00:00, 4561.11it/s, Materializing param=encoder.layer.20.attention.output.dense.bias]
Loading weights:  85%|████████▍ | 373/439 [00:00<00:00, 4569.89it/s, Materializing param=encoder.layer.20.attention.output.dense.weight]
Loading weights:  85%|████████▍ | 373/439 [00:00<00:00, 4568.00it/s, Materializing param=encoder.layer.20.attention.output.dense.weight]
Loading weights:  85%|████████▌ | 374/439 [00:00<00:00, 4577.16it/s, Materializing param=encoder.layer.20.layer_scale1.lambda1]         
Loading weights:  85%|████████▌ | 374/439 [00:00<00:00, 4575.20it/s, Materializing param=encoder.layer.20.layer_scale1.lambda1]
Loading weights:  85%|████████▌ | 375/439 [00:00<00:00, 4584.45it/s, Materializing param=encoder.layer.20.layer_scale2.lambda1]
Loading weights:  85%|████████▌ | 375/439 [00:00<00:00, 4582.44it/s, Materializing param=encoder.layer.20.layer_scale2.lambda1]
Loading weights:  86%|████████▌ | 376/439 [00:00<00:00, 4591.47it/s, Materializing param=encoder.layer.20.mlp.fc1.bias]        
Loading weights:  86%|████████▌ | 376/439 [00:00<00:00, 4589.56it/s, Materializing param=encoder.layer.20.mlp.fc1.bias]
Loading weights:  86%|████████▌ | 377/439 [00:00<00:00, 4598.51it/s, Materializing param=encoder.layer.20.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 377/439 [00:00<00:00, 4596.54it/s, Materializing param=encoder.layer.20.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 378/439 [00:00<00:00, 4605.59it/s, Materializing param=encoder.layer.20.mlp.fc2.bias]  
Loading weights:  86%|████████▌ | 378/439 [00:00<00:00, 4603.73it/s, Materializing param=encoder.layer.20.mlp.fc2.bias]
Loading weights:  86%|████████▋ | 379/439 [00:00<00:00, 4612.82it/s, Materializing param=encoder.layer.20.mlp.fc2.weight]
Loading weights:  86%|████████▋ | 379/439 [00:00<00:00, 4610.86it/s, Materializing param=encoder.layer.20.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 380/439 [00:00<00:00, 4620.05it/s, Materializing param=encoder.layer.20.norm1.bias]    
Loading weights:  87%|████████▋ | 380/439 [00:00<00:00, 4618.09it/s, Materializing param=encoder.layer.20.norm1.bias]
Loading weights:  87%|████████▋ | 381/439 [00:00<00:00, 4627.37it/s, Materializing param=encoder.layer.20.norm1.weight]
Loading weights:  87%|████████▋ | 381/439 [00:00<00:00, 4625.47it/s, Materializing param=encoder.layer.20.norm1.weight]
Loading weights:  87%|████████▋ | 382/439 [00:00<00:00, 4634.74it/s, Materializing param=encoder.layer.20.norm2.bias]  
Loading weights:  87%|████████▋ | 382/439 [00:00<00:00, 4632.70it/s, Materializing param=encoder.layer.20.norm2.bias]
Loading weights:  87%|████████▋ | 383/439 [00:00<00:00, 4641.79it/s, Materializing param=encoder.layer.20.norm2.weight]
Loading weights:  87%|████████▋ | 383/439 [00:00<00:00, 4639.88it/s, Materializing param=encoder.layer.20.norm2.weight]
Loading weights:  87%|████████▋ | 384/439 [00:00<00:00, 4649.06it/s, Materializing param=encoder.layer.21.attention.attention.key.bias]
Loading weights:  87%|████████▋ | 384/439 [00:00<00:00, 4647.11it/s, Materializing param=encoder.layer.21.attention.attention.key.bias]
Loading weights:  88%|████████▊ | 385/439 [00:00<00:00, 4655.76it/s, Materializing param=encoder.layer.21.attention.attention.key.weight]
Loading weights:  88%|████████▊ | 385/439 [00:00<00:00, 4653.74it/s, Materializing param=encoder.layer.21.attention.attention.key.weight]
Loading weights:  88%|████████▊ | 386/439 [00:00<00:00, 4662.79it/s, Materializing param=encoder.layer.21.attention.attention.query.bias]
Loading weights:  88%|████████▊ | 386/439 [00:00<00:00, 4660.75it/s, Materializing param=encoder.layer.21.attention.attention.query.bias]
Loading weights:  88%|████████▊ | 387/439 [00:00<00:00, 4669.62it/s, Materializing param=encoder.layer.21.attention.attention.query.weight]
Loading weights:  88%|████████▊ | 387/439 [00:00<00:00, 4667.71it/s, Materializing param=encoder.layer.21.attention.attention.query.weight]
Loading weights:  88%|████████▊ | 388/439 [00:00<00:00, 4676.49it/s, Materializing param=encoder.layer.21.attention.attention.value.bias]  
Loading weights:  88%|████████▊ | 388/439 [00:00<00:00, 4674.46it/s, Materializing param=encoder.layer.21.attention.attention.value.bias]
Loading weights:  89%|████████▊ | 389/439 [00:00<00:00, 4683.36it/s, Materializing param=encoder.layer.21.attention.attention.value.weight]
Loading weights:  89%|████████▊ | 389/439 [00:00<00:00, 4681.32it/s, Materializing param=encoder.layer.21.attention.attention.value.weight]
Loading weights:  89%|████████▉ | 390/439 [00:00<00:00, 4690.36it/s, Materializing param=encoder.layer.21.attention.output.dense.bias]     
Loading weights:  89%|████████▉ | 390/439 [00:00<00:00, 4688.39it/s, Materializing param=encoder.layer.21.attention.output.dense.bias]
Loading weights:  89%|████████▉ | 391/439 [00:00<00:00, 4697.08it/s, Materializing param=encoder.layer.21.attention.output.dense.weight]
Loading weights:  89%|████████▉ | 391/439 [00:00<00:00, 4695.11it/s, Materializing param=encoder.layer.21.attention.output.dense.weight]
Loading weights:  89%|████████▉ | 392/439 [00:00<00:00, 4704.00it/s, Materializing param=encoder.layer.21.layer_scale1.lambda1]         
Loading weights:  89%|████████▉ | 392/439 [00:00<00:00, 4701.99it/s, Materializing param=encoder.layer.21.layer_scale1.lambda1]
Loading weights:  90%|████████▉ | 393/439 [00:00<00:00, 4711.03it/s, Materializing param=encoder.layer.21.layer_scale2.lambda1]
Loading weights:  90%|████████▉ | 393/439 [00:00<00:00, 4709.12it/s, Materializing param=encoder.layer.21.layer_scale2.lambda1]
Loading weights:  90%|████████▉ | 394/439 [00:00<00:00, 4718.04it/s, Materializing param=encoder.layer.21.mlp.fc1.bias]        
Loading weights:  90%|████████▉ | 394/439 [00:00<00:00, 4716.06it/s, Materializing param=encoder.layer.21.mlp.fc1.bias]
Loading weights:  90%|████████▉ | 395/439 [00:00<00:00, 4724.76it/s, Materializing param=encoder.layer.21.mlp.fc1.weight]
Loading weights:  90%|████████▉ | 395/439 [00:00<00:00, 4722.84it/s, Materializing param=encoder.layer.21.mlp.fc1.weight]
Loading weights:  90%|█████████ | 396/439 [00:00<00:00, 4731.90it/s, Materializing param=encoder.layer.21.mlp.fc2.bias]  
Loading weights:  90%|█████████ | 396/439 [00:00<00:00, 4729.99it/s, Materializing param=encoder.layer.21.mlp.fc2.bias]
Loading weights:  90%|█████████ | 397/439 [00:00<00:00, 4739.10it/s, Materializing param=encoder.layer.21.mlp.fc2.weight]
Loading weights:  90%|█████████ | 397/439 [00:00<00:00, 4736.83it/s, Materializing param=encoder.layer.21.mlp.fc2.weight]
Loading weights:  91%|█████████ | 398/439 [00:00<00:00, 4745.71it/s, Materializing param=encoder.layer.21.norm1.bias]    
Loading weights:  91%|█████████ | 398/439 [00:00<00:00, 4743.73it/s, Materializing param=encoder.layer.21.norm1.bias]
Loading weights:  91%|█████████ | 399/439 [00:00<00:00, 4752.76it/s, Materializing param=encoder.layer.21.norm1.weight]
Loading weights:  91%|█████████ | 399/439 [00:00<00:00, 4750.90it/s, Materializing param=encoder.layer.21.norm1.weight]
Loading weights:  91%|█████████ | 400/439 [00:00<00:00, 4759.86it/s, Materializing param=encoder.layer.21.norm2.bias]  
Loading weights:  91%|█████████ | 400/439 [00:00<00:00, 4757.93it/s, Materializing param=encoder.layer.21.norm2.bias]
Loading weights:  91%|█████████▏| 401/439 [00:00<00:00, 4766.98it/s, Materializing param=encoder.layer.21.norm2.weight]
Loading weights:  91%|█████████▏| 401/439 [00:00<00:00, 4765.17it/s, Materializing param=encoder.layer.21.norm2.weight]
Loading weights:  92%|█████████▏| 402/439 [00:00<00:00, 4774.18it/s, Materializing param=encoder.layer.22.attention.attention.key.bias]
Loading weights:  92%|█████████▏| 402/439 [00:00<00:00, 4772.07it/s, Materializing param=encoder.layer.22.attention.attention.key.bias]
Loading weights:  92%|█████████▏| 403/439 [00:00<00:00, 4780.87it/s, Materializing param=encoder.layer.22.attention.attention.key.weight]
Loading weights:  92%|█████████▏| 403/439 [00:00<00:00, 4778.89it/s, Materializing param=encoder.layer.22.attention.attention.key.weight]
Loading weights:  92%|█████████▏| 404/439 [00:00<00:00, 4787.69it/s, Materializing param=encoder.layer.22.attention.attention.query.bias]
Loading weights:  92%|█████████▏| 404/439 [00:00<00:00, 4785.58it/s, Materializing param=encoder.layer.22.attention.attention.query.bias]
Loading weights:  92%|█████████▏| 405/439 [00:00<00:00, 4794.25it/s, Materializing param=encoder.layer.22.attention.attention.query.weight]
Loading weights:  92%|█████████▏| 405/439 [00:00<00:00, 4792.33it/s, Materializing param=encoder.layer.22.attention.attention.query.weight]
Loading weights:  92%|█████████▏| 406/439 [00:00<00:00, 4800.86it/s, Materializing param=encoder.layer.22.attention.attention.value.bias]  
Loading weights:  92%|█████████▏| 406/439 [00:00<00:00, 4798.81it/s, Materializing param=encoder.layer.22.attention.attention.value.bias]
Loading weights:  93%|█████████▎| 407/439 [00:00<00:00, 4807.68it/s, Materializing param=encoder.layer.22.attention.attention.value.weight]
Loading weights:  93%|█████████▎| 407/439 [00:00<00:00, 4805.70it/s, Materializing param=encoder.layer.22.attention.attention.value.weight]
Loading weights:  93%|█████████▎| 408/439 [00:00<00:00, 4814.33it/s, Materializing param=encoder.layer.22.attention.output.dense.bias]     
Loading weights:  93%|█████████▎| 408/439 [00:00<00:00, 4812.44it/s, Materializing param=encoder.layer.22.attention.output.dense.bias]
Loading weights:  93%|█████████▎| 409/439 [00:00<00:00, 4821.05it/s, Materializing param=encoder.layer.22.attention.output.dense.weight]
Loading weights:  93%|█████████▎| 409/439 [00:00<00:00, 4819.02it/s, Materializing param=encoder.layer.22.attention.output.dense.weight]
Loading weights:  93%|█████████▎| 410/439 [00:00<00:00, 4827.73it/s, Materializing param=encoder.layer.22.layer_scale1.lambda1]         
Loading weights:  93%|█████████▎| 410/439 [00:00<00:00, 4825.68it/s, Materializing param=encoder.layer.22.layer_scale1.lambda1]
Loading weights:  94%|█████████▎| 411/439 [00:00<00:00, 4834.44it/s, Materializing param=encoder.layer.22.layer_scale2.lambda1]
Loading weights:  94%|█████████▎| 411/439 [00:00<00:00, 4832.50it/s, Materializing param=encoder.layer.22.layer_scale2.lambda1]
Loading weights:  94%|█████████▍| 412/439 [00:00<00:00, 4841.41it/s, Materializing param=encoder.layer.22.mlp.fc1.bias]        
Loading weights:  94%|█████████▍| 412/439 [00:00<00:00, 4839.36it/s, Materializing param=encoder.layer.22.mlp.fc1.bias]
Loading weights:  94%|█████████▍| 413/439 [00:00<00:00, 4848.20it/s, Materializing param=encoder.layer.22.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 413/439 [00:00<00:00, 4846.33it/s, Materializing param=encoder.layer.22.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 414/439 [00:00<00:00, 4855.28it/s, Materializing param=encoder.layer.22.mlp.fc2.bias]  
Loading weights:  94%|█████████▍| 414/439 [00:00<00:00, 4853.34it/s, Materializing param=encoder.layer.22.mlp.fc2.bias]
Loading weights:  95%|█████████▍| 415/439 [00:00<00:00, 4862.15it/s, Materializing param=encoder.layer.22.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 415/439 [00:00<00:00, 4860.23it/s, Materializing param=encoder.layer.22.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 416/439 [00:00<00:00, 4869.03it/s, Materializing param=encoder.layer.22.norm1.bias]    
Loading weights:  95%|█████████▍| 416/439 [00:00<00:00, 4867.08it/s, Materializing param=encoder.layer.22.norm1.bias]
Loading weights:  95%|█████████▍| 417/439 [00:00<00:00, 4875.87it/s, Materializing param=encoder.layer.22.norm1.weight]
Loading weights:  95%|█████████▍| 417/439 [00:00<00:00, 4873.88it/s, Materializing param=encoder.layer.22.norm1.weight]
Loading weights:  95%|█████████▌| 418/439 [00:00<00:00, 4882.72it/s, Materializing param=encoder.layer.22.norm2.bias]  
Loading weights:  95%|█████████▌| 418/439 [00:00<00:00, 4880.83it/s, Materializing param=encoder.layer.22.norm2.bias]
Loading weights:  95%|█████████▌| 419/439 [00:00<00:00, 4889.65it/s, Materializing param=encoder.layer.22.norm2.weight]
Loading weights:  95%|█████████▌| 419/439 [00:00<00:00, 4887.61it/s, Materializing param=encoder.layer.22.norm2.weight]
Loading weights:  96%|█████████▌| 420/439 [00:00<00:00, 4896.40it/s, Materializing param=encoder.layer.23.attention.attention.key.bias]
Loading weights:  96%|█████████▌| 420/439 [00:00<00:00, 4894.47it/s, Materializing param=encoder.layer.23.attention.attention.key.bias]
Loading weights:  96%|█████████▌| 421/439 [00:00<00:00, 4902.92it/s, Materializing param=encoder.layer.23.attention.attention.key.weight]
Loading weights:  96%|█████████▌| 421/439 [00:00<00:00, 4900.92it/s, Materializing param=encoder.layer.23.attention.attention.key.weight]
Loading weights:  96%|█████████▌| 422/439 [00:00<00:00, 4909.41it/s, Materializing param=encoder.layer.23.attention.attention.query.bias]
Loading weights:  96%|█████████▌| 422/439 [00:00<00:00, 4907.43it/s, Materializing param=encoder.layer.23.attention.attention.query.bias]
Loading weights:  96%|█████████▋| 423/439 [00:00<00:00, 4915.98it/s, Materializing param=encoder.layer.23.attention.attention.query.weight]
Loading weights:  96%|█████████▋| 423/439 [00:00<00:00, 4913.91it/s, Materializing param=encoder.layer.23.attention.attention.query.weight]
Loading weights:  97%|█████████▋| 424/439 [00:00<00:00, 4922.44it/s, Materializing param=encoder.layer.23.attention.attention.value.bias]  
Loading weights:  97%|█████████▋| 424/439 [00:00<00:00, 4920.44it/s, Materializing param=encoder.layer.23.attention.attention.value.bias]
Loading weights:  97%|█████████▋| 425/439 [00:00<00:00, 4928.79it/s, Materializing param=encoder.layer.23.attention.attention.value.weight]
Loading weights:  97%|█████████▋| 425/439 [00:00<00:00, 4926.73it/s, Materializing param=encoder.layer.23.attention.attention.value.weight]
Loading weights:  97%|█████████▋| 426/439 [00:00<00:00, 4935.46it/s, Materializing param=encoder.layer.23.attention.output.dense.bias]     
Loading weights:  97%|█████████▋| 426/439 [00:00<00:00, 4933.45it/s, Materializing param=encoder.layer.23.attention.output.dense.bias]
Loading weights:  97%|█████████▋| 427/439 [00:00<00:00, 4942.06it/s, Materializing param=encoder.layer.23.attention.output.dense.weight]
Loading weights:  97%|█████████▋| 427/439 [00:00<00:00, 4940.12it/s, Materializing param=encoder.layer.23.attention.output.dense.weight]
Loading weights:  97%|█████████▋| 428/439 [00:00<00:00, 4948.59it/s, Materializing param=encoder.layer.23.layer_scale1.lambda1]         
Loading weights:  97%|█████████▋| 428/439 [00:00<00:00, 4946.82it/s, Materializing param=encoder.layer.23.layer_scale1.lambda1]
Loading weights:  98%|█████████▊| 429/439 [00:00<00:00, 4955.40it/s, Materializing param=encoder.layer.23.layer_scale2.lambda1]
Loading weights:  98%|█████████▊| 429/439 [00:00<00:00, 4953.52it/s, Materializing param=encoder.layer.23.layer_scale2.lambda1]
Loading weights:  98%|█████████▊| 430/439 [00:00<00:00, 4962.36it/s, Materializing param=encoder.layer.23.mlp.fc1.bias]        
Loading weights:  98%|█████████▊| 430/439 [00:00<00:00, 4960.71it/s, Materializing param=encoder.layer.23.mlp.fc1.bias]
Loading weights:  98%|█████████▊| 431/439 [00:00<00:00, 4969.09it/s, Materializing param=encoder.layer.23.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 431/439 [00:00<00:00, 4967.44it/s, Materializing param=encoder.layer.23.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 432/439 [00:00<00:00, 4976.26it/s, Materializing param=encoder.layer.23.mlp.fc2.bias]  
Loading weights:  98%|█████████▊| 432/439 [00:00<00:00, 4974.66it/s, Materializing param=encoder.layer.23.mlp.fc2.bias]
Loading weights:  99%|█████████▊| 433/439 [00:00<00:00, 4983.46it/s, Materializing param=encoder.layer.23.mlp.fc2.weight]
Loading weights:  99%|█████████▊| 433/439 [00:00<00:00, 4981.69it/s, Materializing param=encoder.layer.23.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 434/439 [00:00<00:00, 4990.63it/s, Materializing param=encoder.layer.23.norm1.bias]    
Loading weights:  99%|█████████▉| 434/439 [00:00<00:00, 4988.89it/s, Materializing param=encoder.layer.23.norm1.bias]
Loading weights:  99%|█████████▉| 435/439 [00:00<00:00, 4997.81it/s, Materializing param=encoder.layer.23.norm1.weight]
Loading weights:  99%|█████████▉| 435/439 [00:00<00:00, 4996.15it/s, Materializing param=encoder.layer.23.norm1.weight]
Loading weights:  99%|█████████▉| 436/439 [00:00<00:00, 5004.99it/s, Materializing param=encoder.layer.23.norm2.bias]  
Loading weights:  99%|█████████▉| 436/439 [00:00<00:00, 5003.31it/s, Materializing param=encoder.layer.23.norm2.bias]
Loading weights: 100%|█████████▉| 437/439 [00:00<00:00, 5012.16it/s, Materializing param=encoder.layer.23.norm2.weight]
Loading weights: 100%|█████████▉| 437/439 [00:00<00:00, 5010.32it/s, Materializing param=encoder.layer.23.norm2.weight]
Loading weights: 100%|█████████▉| 438/439 [00:00<00:00, 5019.19it/s, Materializing param=layernorm.bias]               
Loading weights: 100%|█████████▉| 438/439 [00:00<00:00, 5017.58it/s, Materializing param=layernorm.bias]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 5026.27it/s, Materializing param=layernorm.weight]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 5024.67it/s, Materializing param=layernorm.weight]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 5020.30it/s, Materializing param=layernorm.weight]
+  ✓ DINOv3: facebook/dinov2-large
+
Loading weights:   0%|          | 0/408 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/408 [00:00<00:00, 52428.80it/s, Materializing param=logit_bias]
Loading weights:   0%|          | 1/408 [00:00<00:00, 7570.95it/s, Materializing param=logit_bias] 
Loading weights:   0%|          | 2/408 [00:00<00:00, 3332.78it/s, Materializing param=logit_scale]
Loading weights:   0%|          | 2/408 [00:00<00:00, 2971.52it/s, Materializing param=logit_scale]
Loading weights:   1%|          | 3/408 [00:00<00:00, 2819.38it/s, Materializing param=text_model.embeddings.position_embedding.weight]
Loading weights:   1%|          | 3/408 [00:00<00:00, 2683.50it/s, Materializing param=text_model.embeddings.position_embedding.weight]
Loading weights:   1%|          | 4/408 [00:00<00:00, 1930.41it/s, Materializing param=text_model.embeddings.token_embedding.weight]   
Loading weights:   1%|          | 4/408 [00:00<00:00, 1869.95it/s, Materializing param=text_model.embeddings.token_embedding.weight]
Loading weights:   1%|          | 5/408 [00:00<00:00, 1968.42it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.bias]
Loading weights:   1%|          | 5/408 [00:00<00:00, 1917.13it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.bias]
Loading weights:   1%|▏         | 6/408 [00:00<00:00, 1651.52it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.weight]
Loading weights:   1%|▏         | 6/408 [00:00<00:00, 1490.25it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.weight]
Loading weights:   2%|▏         | 7/408 [00:00<00:00, 1499.88it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:   2%|▏         | 7/408 [00:00<00:00, 1451.03it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.bias]
Loading weights:   2%|▏         | 8/408 [00:00<00:00, 1618.72it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.weight]
Loading weights:   2%|▏         | 8/408 [00:00<00:00, 1497.56it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.weight]
Loading weights:   2%|▏         | 9/408 [00:00<00:00, 1497.25it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:   2%|▏         | 9/408 [00:00<00:00, 1421.37it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:   2%|▏         | 10/408 [00:00<00:00, 1494.50it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:   2%|▏         | 10/408 [00:00<00:00, 1484.97it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:   3%|▎         | 11/408 [00:00<00:00, 1533.72it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:   3%|▎         | 11/408 [00:00<00:00, 1522.23it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:   3%|▎         | 12/408 [00:00<00:00, 1524.00it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:   3%|▎         | 12/408 [00:00<00:00, 1359.18it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:   3%|▎         | 13/408 [00:00<00:00, 1393.92it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:   3%|▎         | 13/408 [00:00<00:00, 1383.42it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:   3%|▎         | 14/408 [00:00<00:00, 1427.40it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:   3%|▎         | 14/408 [00:00<00:00, 1421.73it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:   4%|▎         | 15/408 [00:00<00:00, 1392.22it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:   4%|▎         | 15/408 [00:00<00:00, 1348.68it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:   4%|▍         | 16/408 [00:00<00:00, 1392.88it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:   4%|▍         | 16/408 [00:00<00:00, 1380.84it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:   4%|▍         | 17/408 [00:00<00:00, 1454.84it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.bias]    
Loading weights:   4%|▍         | 17/408 [00:00<00:00, 1376.40it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.bias]
Loading weights:   4%|▍         | 18/408 [00:00<00:00, 1387.26it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|▍         | 18/408 [00:00<00:00, 1380.56it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:   5%|▍         | 19/408 [00:00<00:00, 1417.68it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.bias]  
Loading weights:   5%|▍         | 19/408 [00:00<00:00, 1410.95it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.bias]
Loading weights:   5%|▍         | 20/408 [00:00<00:00, 1435.64it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:   5%|▍         | 20/408 [00:00<00:00, 1431.53it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 21/408 [00:00<00:00, 1463.30it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.bias]       
Loading weights:   5%|▌         | 21/408 [00:00<00:00, 1459.44it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.bias]
Loading weights:   5%|▌         | 22/408 [00:00<00:00, 1434.15it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.weight]
Loading weights:   5%|▌         | 22/408 [00:00<00:00, 1378.10it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.weight]
Loading weights:   6%|▌         | 23/408 [00:00<00:00, 1410.26it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.bias]  
Loading weights:   6%|▌         | 23/408 [00:00<00:00, 1377.81it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.bias]
Loading weights:   6%|▌         | 24/408 [00:00<00:00, 1388.56it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.weight]
Loading weights:   6%|▌         | 24/408 [00:00<00:00, 1362.71it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.weight]
Loading weights:   6%|▌         | 25/408 [00:00<00:00, 1338.32it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:   6%|▌         | 25/408 [00:00<00:00, 1322.61it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:   6%|▋         | 26/408 [00:00<00:00, 1328.75it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:   6%|▋         | 26/408 [00:00<00:00, 1313.66it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:   7%|▋         | 27/408 [00:00<00:00, 1351.89it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:   7%|▋         | 27/408 [00:00<00:00, 1349.53it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:   7%|▋         | 28/408 [00:00<00:00, 1352.66it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:   7%|▋         | 28/408 [00:00<00:00, 1332.01it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:   7%|▋         | 29/408 [00:00<00:00, 1338.88it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:   7%|▋         | 29/408 [00:00<00:00, 1335.49it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:   7%|▋         | 30/408 [00:00<00:00, 1359.12it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 30/408 [00:00<00:00, 1354.15it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:   8%|▊         | 31/408 [00:00<00:00, 1390.82it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:   8%|▊         | 31/408 [00:00<00:00, 1366.96it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:   8%|▊         | 32/408 [00:00<00:00, 1336.90it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:   8%|▊         | 32/408 [00:00<00:00, 1333.17it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:   8%|▊         | 33/408 [00:00<00:00, 1354.07it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.bias]    
Loading weights:   8%|▊         | 33/408 [00:00<00:00, 1351.96it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.bias]
Loading weights:   8%|▊         | 34/408 [00:00<00:00, 1374.35it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/408 [00:00<00:00, 1370.13it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:   9%|▊         | 35/408 [00:00<00:00, 1377.23it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.bias]  
Loading weights:   9%|▊         | 35/408 [00:00<00:00, 1373.74it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.bias]
Loading weights:   9%|▉         | 36/408 [00:00<00:00, 1388.36it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:   9%|▉         | 36/408 [00:00<00:00, 1383.76it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:   9%|▉         | 37/408 [00:00<00:00, 1402.52it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.bias]       
Loading weights:   9%|▉         | 37/408 [00:00<00:00, 1389.09it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.bias]
Loading weights:   9%|▉         | 38/408 [00:00<00:00, 1409.70it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.weight]
Loading weights:   9%|▉         | 38/408 [00:00<00:00, 1382.52it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  10%|▉         | 39/408 [00:00<00:00, 1414.02it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  10%|▉         | 39/408 [00:00<00:00, 1411.10it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  10%|▉         | 40/408 [00:00<00:00, 1440.87it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  10%|▉         | 40/408 [00:00<00:00, 1409.09it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  10%|█         | 41/408 [00:00<00:00, 1412.72it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  10%|█         | 41/408 [00:00<00:00, 1399.36it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  10%|█         | 42/408 [00:00<00:00, 1427.44it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  10%|█         | 42/408 [00:00<00:00, 1425.76it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  11%|█         | 43/408 [00:00<00:00, 1445.33it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  11%|█         | 43/408 [00:00<00:00, 1441.35it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  11%|█         | 44/408 [00:00<00:00, 1425.01it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  11%|█         | 44/408 [00:00<00:00, 1423.07it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  11%|█         | 45/408 [00:00<00:00, 1370.70it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  11%|█         | 45/408 [00:00<00:00, 1356.89it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  11%|█▏        | 46/408 [00:00<00:00, 1363.49it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  11%|█▏        | 46/408 [00:00<00:00, 1360.06it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 47/408 [00:00<00:00, 1383.33it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  12%|█▏        | 47/408 [00:00<00:00, 1365.88it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  12%|█▏        | 48/408 [00:00<00:00, 1370.95it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  12%|█▏        | 48/408 [00:00<00:00, 1357.73it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  12%|█▏        | 49/408 [00:00<00:00, 1362.09it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.bias]    
Loading weights:  12%|█▏        | 49/408 [00:00<00:00, 1360.77it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.bias]
Loading weights:  12%|█▏        | 50/408 [00:00<00:00, 1351.21it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  12%|█▏        | 50/408 [00:00<00:00, 1349.60it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  12%|█▎        | 51/408 [00:00<00:00, 1336.62it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.bias]  
Loading weights:  12%|█▎        | 51/408 [00:00<00:00, 1327.50it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.bias]
Loading weights:  13%|█▎        | 52/408 [00:00<00:00, 1344.58it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 52/408 [00:00<00:00, 1333.26it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 53/408 [00:00<00:00, 1356.12it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.bias]       
Loading weights:  13%|█▎        | 53/408 [00:00<00:00, 1345.52it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  13%|█▎        | 54/408 [00:00<00:00, 1366.36it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  13%|█▎        | 54/408 [00:00<00:00, 1364.98it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  13%|█▎        | 55/408 [00:00<00:00, 1359.50it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  13%|█▎        | 55/408 [00:00<00:00, 1358.09it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  14%|█▎        | 56/408 [00:00<00:00, 1348.68it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  14%|█▎        | 56/408 [00:00<00:00, 1339.61it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  14%|█▍        | 57/408 [00:00<00:00, 1345.36it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  14%|█▍        | 57/408 [00:00<00:00, 1343.89it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  14%|█▍        | 58/408 [00:00<00:00, 1356.24it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  14%|█▍        | 58/408 [00:00<00:00, 1336.22it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  14%|█▍        | 59/408 [00:00<00:00, 1353.92it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  14%|█▍        | 59/408 [00:00<00:00, 1344.24it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  15%|█▍        | 60/408 [00:00<00:00, 1356.49it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  15%|█▍        | 60/408 [00:00<00:00, 1346.14it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  15%|█▍        | 61/408 [00:00<00:00, 1349.88it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  15%|█▍        | 61/408 [00:00<00:00, 1338.75it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  15%|█▌        | 62/408 [00:00<00:00, 1356.56it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  15%|█▌        | 62/408 [00:00<00:00, 1354.54it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  15%|█▌        | 63/408 [00:00<00:00, 1343.97it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  15%|█▌        | 63/408 [00:00<00:00, 1338.29it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  16%|█▌        | 64/408 [00:00<00:00, 1350.58it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  16%|█▌        | 64/408 [00:00<00:00, 1348.90it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  16%|█▌        | 65/408 [00:00<00:00, 1366.35it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.bias]    
Loading weights:  16%|█▌        | 65/408 [00:00<00:00, 1359.15it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.bias]
Loading weights:  16%|█▌        | 66/408 [00:00<00:00, 1353.26it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  16%|█▌        | 66/408 [00:00<00:00, 1352.16it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  16%|█▋        | 67/408 [00:00<00:00, 1348.77it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.bias]  
Loading weights:  16%|█▋        | 67/408 [00:00<00:00, 1337.56it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.bias]
Loading weights:  17%|█▋        | 68/408 [00:00<00:00, 1347.55it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  17%|█▋        | 68/408 [00:00<00:00, 1340.66it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  17%|█▋        | 69/408 [00:00<00:00, 1357.41it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.bias]       
Loading weights:  17%|█▋        | 69/408 [00:00<00:00, 1355.73it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  17%|█▋        | 70/408 [00:00<00:00, 1372.84it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  17%|█▋        | 70/408 [00:00<00:00, 1371.77it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  17%|█▋        | 71/408 [00:00<00:00, 1389.95it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  17%|█▋        | 71/408 [00:00<00:00, 1389.13it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  18%|█▊        | 72/408 [00:00<00:00, 1407.32it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  18%|█▊        | 72/408 [00:00<00:00, 1406.52it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  18%|█▊        | 73/408 [00:00<00:00, 1424.73it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  18%|█▊        | 73/408 [00:00<00:00, 1423.83it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  18%|█▊        | 74/408 [00:00<00:00, 1441.96it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  18%|█▊        | 74/408 [00:00<00:00, 1441.09it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  18%|█▊        | 75/408 [00:00<00:00, 1459.15it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  18%|█▊        | 75/408 [00:00<00:00, 1458.29it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  19%|█▊        | 76/408 [00:00<00:00, 1476.33it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  19%|█▊        | 76/408 [00:00<00:00, 1475.47it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  19%|█▉        | 77/408 [00:00<00:00, 1493.49it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  19%|█▉        | 77/408 [00:00<00:00, 1492.62it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  19%|█▉        | 78/408 [00:00<00:00, 1510.55it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 78/408 [00:00<00:00, 1509.67it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 79/408 [00:00<00:00, 1527.57it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  19%|█▉        | 79/408 [00:00<00:00, 1526.66it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  20%|█▉        | 80/408 [00:00<00:00, 1544.55it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  20%|█▉        | 80/408 [00:00<00:00, 1543.60it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  20%|█▉        | 81/408 [00:00<00:00, 1561.41it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.bias]    
Loading weights:  20%|█▉        | 81/408 [00:00<00:00, 1560.33it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.bias]
Loading weights:  20%|██        | 82/408 [00:00<00:00, 1578.07it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  20%|██        | 82/408 [00:00<00:00, 1577.07it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  20%|██        | 83/408 [00:00<00:00, 1594.80it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.bias]  
Loading weights:  20%|██        | 83/408 [00:00<00:00, 1593.79it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.bias]
Loading weights:  21%|██        | 84/408 [00:00<00:00, 1611.42it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  21%|██        | 84/408 [00:00<00:00, 1610.36it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  21%|██        | 85/408 [00:00<00:00, 1627.91it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.bias]       
Loading weights:  21%|██        | 85/408 [00:00<00:00, 1626.92it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  21%|██        | 86/408 [00:00<00:00, 1644.55it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  21%|██        | 86/408 [00:00<00:00, 1643.54it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  21%|██▏       | 87/408 [00:00<00:00, 1661.13it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  21%|██▏       | 87/408 [00:00<00:00, 1660.14it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  22%|██▏       | 88/408 [00:00<00:00, 1677.68it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  22%|██▏       | 88/408 [00:00<00:00, 1676.66it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  22%|██▏       | 89/408 [00:00<00:00, 1694.17it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  22%|██▏       | 89/408 [00:00<00:00, 1693.18it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  22%|██▏       | 90/408 [00:00<00:00, 1710.63it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  22%|██▏       | 90/408 [00:00<00:00, 1709.63it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  22%|██▏       | 91/408 [00:00<00:00, 1726.98it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  22%|██▏       | 91/408 [00:00<00:00, 1725.97it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  23%|██▎       | 92/408 [00:00<00:00, 1743.31it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  23%|██▎       | 92/408 [00:00<00:00, 1742.28it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  23%|██▎       | 93/408 [00:00<00:00, 1759.59it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  23%|██▎       | 93/408 [00:00<00:00, 1758.56it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  23%|██▎       | 94/408 [00:00<00:00, 1775.85it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  23%|██▎       | 94/408 [00:00<00:00, 1774.82it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  23%|██▎       | 95/408 [00:00<00:00, 1792.08it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  23%|██▎       | 95/408 [00:00<00:00, 1791.04it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  24%|██▎       | 96/408 [00:00<00:00, 1808.25it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  24%|██▎       | 96/408 [00:00<00:00, 1807.22it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  24%|██▍       | 97/408 [00:00<00:00, 1824.37it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.bias]    
Loading weights:  24%|██▍       | 97/408 [00:00<00:00, 1823.30it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.bias]
Loading weights:  24%|██▍       | 98/408 [00:00<00:00, 1840.45it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  24%|██▍       | 98/408 [00:00<00:00, 1839.41it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  24%|██▍       | 99/408 [00:00<00:00, 1856.43it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.bias]  
Loading weights:  24%|██▍       | 99/408 [00:00<00:00, 1855.35it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.bias]
Loading weights:  25%|██▍       | 100/408 [00:00<00:00, 1872.42it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  25%|██▍       | 100/408 [00:00<00:00, 1871.33it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  25%|██▍       | 101/408 [00:00<00:00, 1888.31it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.bias]       
Loading weights:  25%|██▍       | 101/408 [00:00<00:00, 1887.21it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  25%|██▌       | 102/408 [00:00<00:00, 1904.23it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  25%|██▌       | 102/408 [00:00<00:00, 1903.09it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  25%|██▌       | 103/408 [00:00<00:00, 1920.10it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  25%|██▌       | 103/408 [00:00<00:00, 1919.03it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  25%|██▌       | 104/408 [00:00<00:00, 1935.92it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  25%|██▌       | 104/408 [00:00<00:00, 1934.85it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  26%|██▌       | 105/408 [00:00<00:00, 1951.71it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  26%|██▌       | 105/408 [00:00<00:00, 1950.61it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  26%|██▌       | 106/408 [00:00<00:00, 1967.41it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  26%|██▌       | 106/408 [00:00<00:00, 1966.35it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  26%|██▌       | 107/408 [00:00<00:00, 1983.13it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  26%|██▌       | 107/408 [00:00<00:00, 1982.03it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  26%|���█▋       | 108/408 [00:00<00:00, 1998.78it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  26%|██▋       | 108/408 [00:00<00:00, 1997.70it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  27%|██▋       | 109/408 [00:00<00:00, 2014.41it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  27%|██▋       | 109/408 [00:00<00:00, 2013.22it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  27%|██▋       | 110/408 [00:00<00:00, 2029.85it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 110/408 [00:00<00:00, 2028.69it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 111/408 [00:00<00:00, 2045.29it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  27%|██▋       | 111/408 [00:00<00:00, 2044.09it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  27%|██▋       | 112/408 [00:00<00:00, 2060.49it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  27%|██▋       | 112/408 [00:00<00:00, 2059.31it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  28%|██▊       | 113/408 [00:00<00:00, 2075.87it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.bias]    
Loading weights:  28%|██▊       | 113/408 [00:00<00:00, 2074.72it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.bias]
Loading weights:  28%|██▊       | 114/408 [00:00<00:00, 2091.25it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 114/408 [00:00<00:00, 2090.02it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 115/408 [00:00<00:00, 2106.49it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.bias]  
Loading weights:  28%|██▊       | 115/408 [00:00<00:00, 2105.26it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.bias]
Loading weights:  28%|██▊       | 116/408 [00:00<00:00, 2121.71it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 116/408 [00:00<00:00, 2120.53it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  29%|██▊       | 117/408 [00:00<00:00, 2136.99it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.bias]       
Loading weights:  29%|██▊       | 117/408 [00:00<00:00, 2135.78it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  29%|██▉       | 118/408 [00:00<00:00, 2152.14it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  29%|██▉       | 118/408 [00:00<00:00, 2150.96it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  29%|██▉       | 119/408 [00:00<00:00, 2167.34it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  29%|██▉       | 119/408 [00:00<00:00, 2166.15it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  29%|██▉       | 120/408 [00:00<00:00, 2182.50it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  29%|██▉       | 120/408 [00:00<00:00, 2181.30it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  30%|██▉       | 121/408 [00:00<00:00, 2197.61it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  30%|██▉       | 121/408 [00:00<00:00, 2196.41it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  30%|██▉       | 122/408 [00:00<00:00, 2212.66it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  30%|██▉       | 122/408 [00:00<00:00, 2211.47it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  30%|███       | 123/408 [00:00<00:00, 2227.65it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  30%|███       | 123/408 [00:00<00:00, 2226.36it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  30%|███       | 124/408 [00:00<00:00, 2242.48it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  30%|███       | 124/408 [00:00<00:00, 2241.22it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  31%|███       | 125/408 [00:00<00:00, 2257.37it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  31%|███       | 125/408 [00:00<00:00, 2256.15it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  31%|███       | 126/408 [00:00<00:00, 2272.24it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  31%|███       | 126/408 [00:00<00:00, 2271.00it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  31%|███       | 127/408 [00:00<00:00, 2287.05it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  31%|███       | 127/408 [00:00<00:00, 2285.77it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  31%|███▏      | 128/408 [00:00<00:00, 2301.74it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  31%|███▏      | 128/408 [00:00<00:00, 2300.45it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  32%|███▏      | 129/408 [00:00<00:00, 2316.39it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.bias]    
Loading weights:  32%|███▏      | 129/408 [00:00<00:00, 2315.10it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.bias]
Loading weights:  32%|███▏      | 130/408 [00:00<00:00, 2331.00it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 130/408 [00:00<00:00, 2329.70it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 131/408 [00:00<00:00, 2345.57it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.bias]  
Loading weights:  32%|███▏      | 131/408 [00:00<00:00, 2344.27it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.bias]
Loading weights:  32%|███▏      | 132/408 [00:00<00:00, 2360.09it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  32%|███▏      | 132/408 [00:00<00:00, 2358.78it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 133/408 [00:00<00:00, 2374.58it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.bias]       
Loading weights:  33%|███▎      | 133/408 [00:00<00:00, 2373.26it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  33%|███▎      | 134/408 [00:00<00:00, 2389.05it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  33%|███▎      | 134/408 [00:00<00:00, 2387.73it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  33%|███▎      | 135/408 [00:00<00:00, 2403.55it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  33%|███▎      | 135/408 [00:00<00:00, 2402.23it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  33%|███▎      | 136/408 [00:00<00:00, 2417.99it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  33%|███▎      | 136/408 [00:00<00:00, 2416.66it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  34%|███▎      | 137/408 [00:00<00:00, 2432.40it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  34%|███▎      | 137/408 [00:00<00:00, 2431.06it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  34%|███▍      | 138/408 [00:00<00:00, 2446.76it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  34%|███▍      | 138/408 [00:00<00:00, 2445.41it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  34%|███▍      | 139/408 [00:00<00:00, 2461.05it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  34%|███▍      | 139/408 [00:00<00:00, 2459.74it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  34%|███▍      | 140/408 [00:00<00:00, 2475.33it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  34%|███▍      | 140/408 [00:00<00:00, 2474.02it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  35%|███▍      | 141/408 [00:00<00:00, 2489.63it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  35%|███▍      | 141/408 [00:00<00:00, 2488.22it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  35%|███▍      | 142/408 [00:00<00:00, 2503.79it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 142/408 [00:00<00:00, 2502.38it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  35%|███▌      | 143/408 [00:00<00:00, 2517.86it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  35%|███▌      | 143/408 [00:00<00:00, 2516.45it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  35%|███▌      | 144/408 [00:00<00:00, 2531.91it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  35%|███▌      | 144/408 [00:00<00:00, 2530.53it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  36%|███▌      | 145/408 [00:00<00:00, 2545.95it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.bias]    
Loading weights:  36%|███▌      | 145/408 [00:00<00:00, 2544.57it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.bias]
Loading weights:  36%|███▌      | 146/408 [00:00<00:00, 2560.01it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  36%|███▌      | 146/408 [00:00<00:00, 2558.62it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  36%|███▌      | 147/408 [00:00<00:00, 2574.02it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.bias]  
Loading weights:  36%|███▌      | 147/408 [00:00<00:00, 2572.68it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.bias]
Loading weights:  36%|███▋      | 148/408 [00:00<00:00, 2588.00it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  36%|███▋      | 148/408 [00:00<00:00, 2586.60it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  37%|███▋      | 149/408 [00:00<00:00, 2601.89it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.bias]       
Loading weights:  37%|███▋      | 149/408 [00:00<00:00, 2600.52it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  37%|███▋      | 150/408 [00:00<00:00, 2615.79it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  37%|███▋      | 150/408 [00:00<00:00, 2614.38it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  37%|███▋      | 151/408 [00:00<00:00, 2629.65it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  37%|███▋      | 151/408 [00:00<00:00, 2628.28it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  37%|███▋      | 152/408 [00:00<00:00, 2643.52it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  37%|███▋      | 152/408 [00:00<00:00, 2642.09it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  38%|███▊      | 153/408 [00:00<00:00, 2657.36it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  38%|███▊      | 153/408 [00:00<00:00, 2655.93it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  38%|███▊      | 154/408 [00:00<00:00, 2671.01it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  38%|███▊      | 154/408 [00:00<00:00, 2669.67it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  38%|███▊      | 155/408 [00:00<00:00, 2684.77it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  38%|███▊      | 155/408 [00:00<00:00, 2683.42it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  38%|███▊      | 156/408 [00:00<00:00, 2698.55it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  38%|███▊      | 156/408 [00:00<00:00, 2697.09it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  38%|███▊      | 157/408 [00:00<00:00, 2712.13it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  38%|███▊      | 157/408 [00:00<00:00, 2710.68it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  39%|███▊      | 158/408 [00:00<00:00, 2725.78it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  39%|███▊      | 158/408 [00:00<00:00, 2724.32it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 159/408 [00:00<00:00, 2739.30it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  39%|███▉      | 159/408 [00:00<00:00, 2737.80it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  39%|███▉      | 160/408 [00:00<00:00, 2752.73it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  39%|███▉      | 160/408 [00:00<00:00, 2751.22it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  39%|███▉      | 161/408 [00:00<00:00, 2766.14it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.bias]    
Loading weights:  39%|███▉      | 161/408 [00:00<00:00, 2764.62it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.bias]
Loading weights:  40%|███▉      | 162/408 [00:00<00:00, 2779.49it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 162/408 [00:00<00:00, 2777.97it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 163/408 [00:00<00:00, 2792.82it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.bias]  
Loading weights:  40%|███▉      | 163/408 [00:00<00:00, 2791.33it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.bias]
Loading weights:  40%|████      | 164/408 [00:00<00:00, 2806.15it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  40%|████      | 164/408 [00:00<00:00, 2804.62it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  40%|████      | 165/408 [00:00<00:00, 2819.36it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.bias]      
Loading weights:  40%|████      | 165/408 [00:00<00:00, 2817.90it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  41%|████      | 166/408 [00:00<00:00, 2832.72it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  41%|████      | 166/408 [00:00<00:00, 2831.27it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  41%|████      | 167/408 [00:00<00:00, 2846.03it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  41%|████      | 167/408 [00:00<00:00, 2844.53it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  41%|████      | 168/408 [00:00<00:00, 2859.32it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  41%|████      | 168/408 [00:00<00:00, 2857.87it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  41%|████▏     | 169/408 [00:00<00:00, 2872.54it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  41%|████▏     | 169/408 [00:00<00:00, 2871.07it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  42%|████▏     | 170/408 [00:00<00:00, 2885.70it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  42%|████▏     | 170/408 [00:00<00:00, 2884.19it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  42%|████▏     | 171/408 [00:00<00:00, 2898.84it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  42%|████▏     | 171/408 [00:00<00:00, 2897.41it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  42%|████▏     | 172/408 [00:00<00:00, 2912.09it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  42%|████▏     | 172/408 [00:00<00:00, 2910.62it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  42%|████▏     | 173/408 [00:00<00:00, 2925.17it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  42%|████▏     | 173/408 [00:00<00:00, 2923.63it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  43%|████▎     | 174/408 [00:00<00:00, 2938.10it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  43%|████▎     | 174/408 [00:00<00:00, 2936.50it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  43%|████▎     | 175/408 [00:00<00:00, 2951.00it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  43%|████▎     | 175/408 [00:00<00:00, 2949.40it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  43%|████▎     | 176/408 [00:00<00:00, 2963.76it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  43%|████▎     | 176/408 [00:00<00:00, 2962.21it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  43%|████▎     | 177/408 [00:00<00:00, 2976.54it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.bias]    
Loading weights:  43%|████▎     | 177/408 [00:00<00:00, 2974.99it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.bias]
Loading weights:  44%|████▎     | 178/408 [00:00<00:00, 2989.27it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  44%|████▎     | 178/408 [00:00<00:00, 2987.67it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  44%|████▍     | 179/408 [00:00<00:00, 3001.99it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.bias]  
Loading weights:  44%|████▍     | 179/408 [00:00<00:00, 3000.43it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.bias]
Loading weights:  44%|████▍     | 180/408 [00:00<00:00, 3014.73it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  44%|████▍     | 180/408 [00:00<00:00, 3013.10it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  44%|████▍     | 181/408 [00:00<00:00, 3027.36it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.bias]       
Loading weights:  44%|████▍     | 181/408 [00:00<00:00, 3025.80it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  45%|████▍     | 182/408 [00:00<00:00, 3040.06it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  45%|████▍     | 182/408 [00:00<00:00, 3038.49it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  45%|████▍     | 183/408 [00:00<00:00, 3052.74it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  45%|████▍     | 183/408 [00:00<00:00, 3051.22it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  45%|████▌     | 184/408 [00:00<00:00, 3065.44it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  45%|████▌     | 184/408 [00:00<00:00, 3063.95it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  45%|████▌     | 185/408 [00:00<00:00, 3078.14it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  45%|████▌     | 185/408 [00:00<00:00, 3076.66it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  46%|████▌     | 186/408 [00:00<00:00, 3090.89it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  46%|████▌     | 186/408 [00:00<00:00, 3089.38it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  46%|████▌     | 187/408 [00:00<00:00, 3103.58it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  46%|████▌     | 187/408 [00:00<00:00, 3102.08it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  46%|████▌     | 188/408 [00:00<00:00, 3116.30it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  46%|████▌     | 188/408 [00:00<00:00, 3114.70it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  46%|████▋     | 189/408 [00:00<00:00, 3128.77it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  46%|████▋     | 189/408 [00:00<00:00, 3127.11it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  47%|████▋     | 190/408 [00:00<00:00, 3141.12it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 190/408 [00:00<00:00, 3139.56it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 191/408 [00:00<00:00, 3153.59it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  47%|████▋     | 191/408 [00:00<00:00, 3152.02it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  47%|████▋     | 192/408 [00:00<00:00, 3166.01it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  47%|████▋     | 192/408 [00:00<00:00, 3164.44it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  47%|████▋     | 193/408 [00:00<00:00, 3178.43it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.bias]    
Loading weights:  47%|████▋     | 193/408 [00:00<00:00, 3176.80it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.bias]
Loading weights:  48%|████▊     | 194/408 [00:00<00:00, 3190.74it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 194/408 [00:00<00:00, 3189.21it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  48%|���███▊     | 195/408 [00:00<00:00, 3203.06it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.bias]  
Loading weights:  48%|████▊     | 195/408 [00:00<00:00, 3201.55it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.bias]
Loading weights:  48%|████▊     | 196/408 [00:00<00:00, 3215.47it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 196/408 [00:00<00:00, 3213.90it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 197/408 [00:00<00:00, 3227.60it/s, Materializing param=text_model.final_layer_norm.bias]                    
Loading weights:  48%|████▊     | 197/408 [00:00<00:00, 3226.02it/s, Materializing param=text_model.final_layer_norm.bias]
Loading weights:  49%|████▊     | 198/408 [00:00<00:00, 3240.11it/s, Materializing param=text_model.final_layer_norm.weight]
Loading weights:  49%|████▊     | 198/408 [00:00<00:00, 3238.58it/s, Materializing param=text_model.final_layer_norm.weight]
Loading weights:  49%|████▉     | 199/408 [00:00<00:00, 3252.59it/s, Materializing param=text_model.head.bias]              
Loading weights:  49%|████▉     | 199/408 [00:00<00:00, 3251.04it/s, Materializing param=text_model.head.bias]
Loading weights:  49%|████▉     | 200/408 [00:00<00:00, 3265.14it/s, Materializing param=text_model.head.weight]
Loading weights:  49%|████▉     | 200/408 [00:00<00:00, 3263.71it/s, Materializing param=text_model.head.weight]
Loading weights:  49%|████▉     | 201/408 [00:00<00:00, 3277.78it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  49%|████▉     | 201/408 [00:00<00:00, 3276.13it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  50%|████▉     | 202/408 [00:00<00:00, 3289.73it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  50%|████▉     | 202/408 [00:00<00:00, 3288.13it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  50%|████▉     | 203/408 [00:00<00:00, 3302.00it/s, Materializing param=vision_model.embeddings.position_embedding.weight]
Loading weights:  50%|████▉     | 203/408 [00:00<00:00, 3300.32it/s, Materializing param=vision_model.embeddings.position_embedding.weight]
Loading weights:  50%|█████     | 204/408 [00:00<00:00, 3314.11it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]   
Loading weights:  50%|█████     | 204/408 [00:00<00:00, 3312.45it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  50%|█████     | 205/408 [00:00<00:00, 3326.08it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  50%|█████     | 205/408 [00:00<00:00, 3324.36it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  50%|█████     | 206/408 [00:00<00:00, 3338.07it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:  50%|█████     | 206/408 [00:00<00:00, 3336.40it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]
Loading weights:  51%|█████     | 207/408 [00:00<00:00, 3349.89it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  51%|█████     | 207/408 [00:00<00:00, 3348.33it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  51%|█████     | 208/408 [00:00<00:00, 3361.93it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:  51%|█████     | 208/408 [00:00<00:00, 3360.20it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:  51%|█████     | 209/408 [00:00<00:00, 3373.69it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  51%|█████     | 209/408 [00:00<00:00, 3372.06it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  51%|█████▏    | 210/408 [00:00<00:00, 3385.56it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:  51%|█████▏    | 210/408 [00:00<00:00, 3383.93it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:  52%|█████▏    | 211/408 [00:00<00:00, 3397.08it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  52%|█████▏    | 211/408 [00:00<00:00, 3395.39it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  52%|█████▏    | 212/408 [00:00<00:00, 3408.53it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:  52%|█████▏    | 212/408 [00:00<00:00, 3406.66it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:  52%|█████▏    | 213/408 [00:00<00:00, 3419.87it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 213/408 [00:00<00:00, 3418.05it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 214/408 [00:00<00:00, 3431.30it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:  52%|█████▏    | 214/408 [00:00<00:00, 3429.43it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:  53%|█████▎    | 215/408 [00:00<00:00, 3442.71it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 215/408 [00:00<00:00, 3440.54it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 216/408 [00:00<00:00, 3451.69it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.bias]    
Loading weights:  53%|█████▎    | 216/408 [00:00<00:00, 3449.33it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.bias]
Loading weights:  53%|█████▎    | 217/408 [00:00<00:00, 3462.08it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 217/408 [00:00<00:00, 3460.04it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 218/408 [00:00<00:00, 3472.77it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.bias]  
Loading weights:  53%|█████▎    | 218/408 [00:00<00:00, 3470.72it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.bias]
Loading weights:  54%|█████▎    | 219/408 [00:00<00:00, 3483.48it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:  54%|█████▎    | 219/408 [00:00<00:00, 3481.44it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:  54%|█████▍    | 220/408 [00:00<00:00, 3494.33it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]       
Loading weights:  54%|█████▍    | 220/408 [00:00<00:00, 3492.29it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]
Loading weights:  54%|█████▍    | 221/408 [00:00<00:00, 3504.98it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  54%|█████▍    | 221/408 [00:00<00:00, 3503.03it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  54%|█████▍    | 222/408 [00:00<00:00, 3515.78it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]  
Loading weights:  54%|█████▍    | 222/408 [00:00<00:00, 3514.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]
Loading weights:  55%|█████▍    | 223/408 [00:00<00:00, 3526.80it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  55%|█████▍    | 223/408 [00:00<00:00, 3524.80it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  55%|█████▍    | 224/408 [00:00<00:00, 3537.58it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:  55%|█████▍    | 224/408 [00:00<00:00, 3535.63it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:  55%|█████▌    | 225/408 [00:00<00:00, 3548.23it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  55%|█████▌    | 225/408 [00:00<00:00, 3546.32it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  55%|█████▌    | 226/408 [00:00<00:00, 3559.22it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:  55%|█████▌    | 226/408 [00:00<00:00, 3557.31it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:  56%|█████▌    | 227/408 [00:00<00:00, 3570.08it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  56%|█████▌    | 227/408 [00:00<00:00, 3568.11it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  56%|█████▌    | 228/408 [00:00<00:00, 3580.79it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:  56%|█████▌    | 228/408 [00:00<00:00, 3578.77it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:  56%|█████▌    | 229/408 [00:00<00:00, 3591.31it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 229/408 [00:00<00:00, 3589.39it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 230/408 [00:00<00:00, 3601.95it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:  56%|█████▋    | 230/408 [00:00<00:00, 3600.05it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:  57%|█████▋    | 231/408 [00:00<00:00, 3612.65it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:  57%|█████▋    | 231/408 [00:00<00:00, 3610.55it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:  57%|█████▋    | 232/408 [00:00<00:00, 3623.13it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.bias]    
Loading weights:  57%|█████▋    | 232/408 [00:00<00:00, 3621.15it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.bias]
Loading weights:  57%|█████▋    | 233/408 [00:00<00:00, 3633.52it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 233/408 [00:00<00:00, 3631.49it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 234/408 [00:00<00:00, 3644.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.bias]  
Loading weights:  57%|█████▋    | 234/408 [00:00<00:00, 3641.92it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.bias]
Loading weights:  58%|█████▊    | 235/408 [00:00<00:00, 3654.29it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 235/408 [00:00<00:00, 3652.23it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 236/408 [00:00<00:00, 3664.71it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]       
Loading weights:  58%|█████▊    | 236/408 [00:00<00:00, 3662.66it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]
Loading weights:  58%|█████▊    | 237/408 [00:00<00:00, 3675.11it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  58%|█████▊    | 237/408 [00:00<00:00, 3673.11it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  58%|█████▊    | 238/408 [00:00<00:00, 3685.69it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  58%|█████▊    | 238/408 [00:00<00:00, 3683.69it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  59%|█████▊    | 239/408 [00:00<00:00, 3696.20it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  59%|█████▊    | 239/408 [00:00<00:00, 3694.20it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  59%|█████▉    | 240/408 [00:00<00:00, 3706.55it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  59%|█████▉    | 240/408 [00:00<00:00, 3704.67it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  59%|█████▉    | 241/408 [00:00<00:00, 3716.96it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  59%|█████▉    | 241/408 [00:00<00:00, 3714.90it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  59%|█████▉    | 242/408 [00:00<00:00, 3727.48it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  59%|█████▉    | 242/408 [00:00<00:00, 3725.60it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  60%|█████▉    | 243/408 [00:00<00:00, 3737.99it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  60%|█████▉    | 243/408 [00:00<00:00, 3736.16it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  60%|█████▉    | 244/408 [00:00<00:00, 3748.77it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  60%|█████▉    | 244/408 [00:00<00:00, 3746.93it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  60%|██████    | 245/408 [00:00<00:00, 3759.34it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  60%|██████    | 245/408 [00:00<00:00, 3757.44it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  60%|██████    | 246/408 [00:00<00:00, 3769.83it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  60%|██████    | 246/408 [00:00<00:00, 3767.92it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  61%|██████    | 247/408 [00:00<00:00, 3780.33it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  61%|██████    | 247/408 [00:00<00:00, 3778.54it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  61%|██████    | 248/408 [00:00<00:00, 3790.88it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.bias]    
Loading weights:  61%|██████    | 248/408 [00:00<00:00, 3788.91it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.bias]
Loading weights:  61%|██████    | 249/408 [00:00<00:00, 3801.23it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  61%|██████    | 249/408 [00:00<00:00, 3799.26it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  61%|██████▏   | 250/408 [00:00<00:00, 3811.55it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.bias]  
Loading weights:  61%|██████▏   | 250/408 [00:00<00:00, 3809.75it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.bias]
Loading weights:  62%|██████▏   | 251/408 [00:00<00:00, 3821.96it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 251/408 [00:00<00:00, 3819.74it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 252/408 [00:00<00:00, 3832.05it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]       
Loading weights:  62%|██████▏   | 252/408 [00:00<00:00, 3830.07it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  62%|██████▏   | 253/408 [00:00<00:00, 3842.30it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  62%|██████▏   | 253/408 [00:00<00:00, 3840.49it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  62%|██████▏   | 254/408 [00:00<00:00, 3852.74it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  62%|██████▏   | 254/408 [00:00<00:00, 3850.82it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  62%|██████▎   | 255/408 [00:00<00:00, 3862.94it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  62%|██████▎   | 255/408 [00:00<00:00, 3861.07it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  63%|██████▎   | 256/408 [00:00<00:00, 3873.32it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  63%|██████▎   | 256/408 [00:00<00:00, 3871.33it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  63%|██████▎   | 257/408 [00:00<00:00, 3883.46it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  63%|██████▎   | 257/408 [00:00<00:00, 3881.46it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  63%|██████▎   | 258/408 [00:00<00:00, 3893.56it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  63%|██████▎   | 258/408 [00:00<00:00, 3891.69it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  63%|██████▎   | 259/408 [00:00<00:00, 3903.84it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  63%|██████▎   | 259/408 [00:00<00:00, 3901.96it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  64%|██████▎   | 260/408 [00:00<00:00, 3914.11it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  64%|██████▎   | 260/408 [00:00<00:00, 3912.18it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  64%|██████▍   | 261/408 [00:00<00:00, 3924.15it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 261/408 [00:00<00:00, 3922.27it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 262/408 [00:00<00:00, 3934.41it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  64%|██████▍   | 262/408 [00:00<00:00, 3932.45it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  64%|██████▍   | 263/408 [00:00<00:00, 3944.45it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  64%|██████▍   | 263/408 [00:00<00:00, 3942.55it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  65%|██████▍   | 264/408 [00:00<00:00, 3954.53it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.bias]    
Loading weights:  65%|██████▍   | 264/408 [00:00<00:00, 3952.57it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.bias]
Loading weights:  65%|██████▍   | 265/408 [00:00<00:00, 3964.63it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 265/408 [00:00<00:00, 3962.73it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 266/408 [00:00<00:00, 3974.60it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.bias]  
Loading weights:  65%|██████▌   | 266/408 [00:00<00:00, 3971.81it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.bias]
Loading weights:  65%|██████▌   | 267/408 [00:00<00:00, 3983.63it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 267/408 [00:00<00:00, 3981.69it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  66%|██████▌   | 268/408 [00:00<00:00, 3993.68it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]       
Loading weights:  66%|██████▌   | 268/408 [00:00<00:00, 3991.77it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  66%|██████▌   | 269/408 [00:00<00:00, 4003.63it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  66%|██████▌   | 269/408 [00:00<00:00, 4001.79it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  66%|██████▌   | 270/408 [00:00<00:00, 4013.84it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  66%|██████▌   | 270/408 [00:00<00:00, 4011.88it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  66%|██████▋   | 271/408 [00:00<00:00, 4023.82it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  66%|██████▋   | 271/408 [00:00<00:00, 4021.90it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  67%|██████▋   | 272/408 [00:00<00:00, 4031.79it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  67%|██████▋   | 272/408 [00:00<00:00, 4029.62it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  67%|██████▋   | 273/408 [00:00<00:00, 4041.14it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  67%|██████▋   | 273/408 [00:00<00:00, 4039.06it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  67%|██████▋   | 274/408 [00:00<00:00, 4050.63it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  67%|██████▋   | 274/408 [00:00<00:00, 4048.53it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  67%|██████▋   | 275/408 [00:00<00:00, 4060.19it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  67%|██████▋   | 275/408 [00:00<00:00, 4058.33it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  68%|██████▊   | 276/408 [00:00<00:00, 4069.89it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  68%|██████▊   | 276/408 [00:00<00:00, 4067.92it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  68%|██████▊   | 277/408 [00:00<00:00, 4079.47it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  68%|██████▊   | 277/408 [00:00<00:00, 4077.61it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  68%|██████▊   | 278/408 [00:00<00:00, 4089.19it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  68%|██████▊   | 278/408 [00:00<00:00, 4087.21it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  68%|██████▊   | 279/408 [00:00<00:00, 4098.84it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  68%|██████▊   | 279/408 [00:00<00:00, 4096.80it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  69%|██████▊   | 280/408 [00:00<00:00, 4108.39it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.bias]    
Loading weights:  69%|██████▊   | 280/408 [00:00<00:00, 4106.53it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.bias]
Loading weights:  69%|██████▉   | 281/408 [00:00<00:00, 4118.18it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  69%|██████▉   | 281/408 [00:00<00:00, 4116.13it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  69%|██████▉   | 282/408 [00:00<00:00, 4127.76it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.bias]  
Loading weights:  69%|██████▉   | 282/408 [00:00<00:00, 4125.95it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.bias]
Loading weights:  69%|██████▉   | 283/408 [00:00<00:00, 4137.61it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  69%|██████▉   | 283/408 [00:00<00:00, 4135.67it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 284/408 [00:00<00:00, 4147.31it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]       
Loading weights:  70%|██████▉   | 284/408 [00:00<00:00, 4145.38it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  70%|██████▉   | 285/408 [00:00<00:00, 4156.94it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  70%|██████▉   | 285/408 [00:00<00:00, 4155.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  70%|███████   | 286/408 [00:00<00:00, 4166.54it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  70%|███████   | 286/408 [00:00<00:00, 4164.66it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  70%|███████   | 287/408 [00:00<00:00, 4176.30it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  70%|███████   | 287/408 [00:00<00:00, 4174.41it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  71%|███████   | 288/408 [00:00<00:00, 4186.05it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  71%|███████   | 288/408 [00:00<00:00, 4184.03it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  71%|███████   | 289/408 [00:00<00:00, 4195.52it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  71%|███████   | 289/408 [00:00<00:00, 4193.09it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  71%|███████   | 290/408 [00:00<00:00, 4204.54it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  71%|███████   | 290/408 [00:00<00:00, 4202.65it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  71%|███████▏  | 291/408 [00:00<00:00, 4214.03it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  71%|███████▏  | 291/408 [00:00<00:00, 4212.02it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  72%|███████▏  | 292/408 [00:00<00:00, 4223.43it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  72%|███████▏  | 292/408 [00:00<00:00, 4221.48it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  72%|███████▏  | 293/408 [00:00<00:00, 4232.76it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 293/408 [00:00<00:00, 4230.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 294/408 [00:00<00:00, 4241.57it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  72%|███████▏  | 294/408 [00:00<00:00, 4239.37it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  72%|███████▏  | 295/408 [00:00<00:00, 4250.41it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  72%|███████▏  | 295/408 [00:00<00:00, 4248.07it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  73%|███████▎  | 296/408 [00:00<00:00, 4259.17it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.bias]    
Loading weights:  73%|███████▎  | 296/408 [00:00<00:00, 4257.08it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.bias]
Loading weights:  73%|███████▎  | 297/408 [00:00<00:00, 4268.04it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  73%|███████▎  | 297/408 [00:00<00:00, 4265.89it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  73%|███████▎  | 298/408 [00:00<00:00, 4277.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.bias]  
Loading weights:  73%|███████▎  | 298/408 [00:00<00:00, 4274.85it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.bias]
Loading weights:  73%|███████▎  | 299/408 [00:00<00:00, 4285.93it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  73%|███████▎  | 299/408 [00:00<00:00, 4283.83it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  74%|███████▎  | 300/408 [00:00<00:00, 4294.80it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]       
Loading weights:  74%|███████▎  | 300/408 [00:00<00:00, 4292.71it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  74%|███████▍  | 301/408 [00:00<00:00, 4303.62it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  74%|███████▍  | 301/408 [00:00<00:00, 4301.41it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  74%|███████▍  | 302/408 [00:00<00:00, 4312.50it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  74%|███████▍  | 302/408 [00:00<00:00, 4310.59it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  74%|███████▍  | 303/408 [00:00<00:00, 4317.40it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  74%|███████▍  | 303/408 [00:00<00:00, 4314.64it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  75%|███████▍  | 304/408 [00:00<00:00, 4324.88it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  75%|███████▍  | 304/408 [00:00<00:00, 4322.47it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  75%|███████▍  | 305/408 [00:00<00:00, 4333.06it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  75%|███████▍  | 305/408 [00:00<00:00, 4330.77it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  75%|███████▌  | 306/408 [00:00<00:00, 4341.28it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  75%|███████▌  | 306/408 [00:00<00:00, 4339.13it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  75%|███████▌  | 307/408 [00:00<00:00, 4349.86it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  75%|███████▌  | 307/408 [00:00<00:00, 4347.57it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  75%|███████▌  | 308/408 [00:00<00:00, 4358.28it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  75%|███████▌  | 308/408 [00:00<00:00, 4356.07it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  76%|███████▌  | 309/408 [00:00<00:00, 4366.76it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 309/408 [00:00<00:00, 4364.40it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 310/408 [00:00<00:00, 4375.06it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  76%|███████▌  | 310/408 [00:00<00:00, 4372.78it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  76%|███████▌  | 311/408 [00:00<00:00, 4383.31it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  76%|███████▌  | 311/408 [00:00<00:00, 4381.33it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  76%|███████▋  | 312/408 [00:00<00:00, 4391.09it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.bias]    
Loading weights:  76%|███████▋  | 312/408 [00:00<00:00, 4388.30it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.bias]
Loading weights:  77%|███████▋  | 313/408 [00:00<00:00, 4398.30it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 313/408 [00:00<00:00, 4395.94it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 314/408 [00:00<00:00, 4406.34it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.bias]  
Loading weights:  77%|███████▋  | 314/408 [00:00<00:00, 4403.93it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.bias]
Loading weights:  77%|███████▋  | 315/408 [00:00<00:00, 4414.17it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 315/408 [00:00<00:00, 4411.89it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 316/408 [00:00<00:00, 4422.42it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]       
Loading weights:  77%|███████▋  | 316/408 [00:00<00:00, 4420.20it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  78%|███████▊  | 317/408 [00:00<00:00, 4430.77it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  78%|███████▊  | 317/408 [00:00<00:00, 4428.85it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  78%|███████▊  | 318/408 [00:00<00:00, 4439.30it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  78%|███████▊  | 318/408 [00:00<00:00, 4437.06it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  78%|███████▊  | 319/408 [00:00<00:00, 4447.53it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  78%|███████▊  | 319/408 [00:00<00:00, 4445.24it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  78%|███████▊  | 320/408 [00:00<00:00, 4455.89it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  78%|███████▊  | 320/408 [00:00<00:00, 4453.60it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  79%|███████▊  | 321/408 [00:00<00:00, 4464.16it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  79%|███████▊  | 321/408 [00:00<00:00, 4461.86it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  79%|███████▉  | 322/408 [00:00<00:00, 4472.47it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  79%|███████▉  | 322/408 [00:00<00:00, 4470.43it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  79%|███████▉  | 323/408 [00:00<00:00, 4480.75it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  79%|███████▉  | 323/408 [00:00<00:00, 4478.58it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  79%|███████▉  | 324/408 [00:00<00:00, 4489.01it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  79%|███████▉  | 324/408 [00:00<00:00, 4486.73it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  80%|███████▉  | 325/408 [00:00<00:00, 4497.02it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  80%|███████▉  | 325/408 [00:00<00:00, 4494.89it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  80%|███████▉  | 326/408 [00:00<00:00, 4505.32it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  80%|███████▉  | 326/408 [00:00<00:00, 4503.13it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  80%|████████  | 327/408 [00:00<00:00, 4513.33it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  80%|████████  | 327/408 [00:00<00:00, 4511.16it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  80%|████████  | 328/408 [00:00<00:00, 4521.14it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.bias]    
Loading weights:  80%|████████  | 328/408 [00:00<00:00, 4519.03it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.bias]
Loading weights:  81%|████████  | 329/408 [00:00<00:00, 4529.37it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  81%|████████  | 329/408 [00:00<00:00, 4526.87it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  81%|████████  | 330/408 [00:00<00:00, 4537.07it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.bias]  
Loading weights:  81%|████████  | 330/408 [00:00<00:00, 4535.02it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.bias]
Loading weights:  81%|████████  | 331/408 [00:00<00:00, 4545.20it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  81%|████████  | 331/408 [00:00<00:00, 4542.96it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  81%|████████▏ | 332/408 [00:00<00:00, 4553.42it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]       
Loading weights:  81%|████████▏ | 332/408 [00:00<00:00, 4551.05it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  82%|████████▏ | 333/408 [00:00<00:00, 4561.32it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  82%|████████▏ | 333/408 [00:00<00:00, 4559.13it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  82%|████████▏ | 334/408 [00:00<00:00, 4569.57it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  82%|████████▏ | 334/408 [00:00<00:00, 4567.51it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  82%|████████▏ | 335/408 [00:00<00:00, 4577.76it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  82%|████████▏ | 335/408 [00:00<00:00, 4575.62it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  82%|████████▏ | 336/408 [00:00<00:00, 4586.03it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  82%|████████▏ | 336/408 [00:00<00:00, 4583.89it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  83%|████████▎ | 337/408 [00:00<00:00, 4594.28it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 337/408 [00:00<00:00, 4592.22it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 338/408 [00:00<00:00, 4602.59it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  83%|████████▎ | 338/408 [00:00<00:00, 4600.39it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  83%|████████▎ | 339/408 [00:00<00:00, 4610.74it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 339/408 [00:00<00:00, 4608.60it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 340/408 [00:00<00:00, 4618.99it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  83%|████████▎ | 340/408 [00:00<00:00, 4616.80it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  84%|████████▎ | 341/408 [00:00<00:00, 4626.91it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  84%|████████▎ | 341/408 [00:00<00:00, 4624.73it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 342/408 [00:00<00:00, 4634.90it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  84%|████████▍ | 342/408 [00:00<00:00, 4632.64it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  84%|████████▍ | 343/408 [00:00<00:00, 4642.98it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  84%|████████▍ | 343/408 [00:00<00:00, 4640.84it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  84%|████████▍ | 344/408 [00:00<00:00, 4651.16it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.bias]    
Loading weights:  84%|████████▍ | 344/408 [00:00<00:00, 4648.96it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.bias]
Loading weights:  85%|████████▍ | 345/408 [00:00<00:00, 4659.20it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 345/408 [00:00<00:00, 4657.05it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 346/408 [00:00<00:00, 4667.16it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.bias]  
Loading weights:  85%|████████▍ | 346/408 [00:00<00:00, 4664.95it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.bias]
Loading weights:  85%|████████▌ | 347/408 [00:00<00:00, 4675.35it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 347/408 [00:00<00:00, 4673.15it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 348/408 [00:00<00:00, 4683.14it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]       
Loading weights:  85%|████████▌ | 348/408 [00:00<00:00, 4681.13it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  86%|████████▌ | 349/408 [00:00<00:00, 4691.30it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  86%|████████▌ | 349/408 [00:00<00:00, 4689.11it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  86%|████████▌ | 350/408 [00:00<00:00, 4699.30it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  86%|████████▌ | 350/408 [00:00<00:00, 4697.11it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  86%|████████▌ | 351/408 [00:00<00:00, 4707.11it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  86%|████████▌ | 351/408 [00:00<00:00, 4705.03it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  86%|████████▋ | 352/408 [00:00<00:00, 4715.02it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  86%|████████▋ | 352/408 [00:00<00:00, 4713.06it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  87%|████████▋ | 353/408 [00:00<00:00, 4723.48it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 353/408 [00:00<00:00, 4721.33it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 354/408 [00:00<00:00, 4731.17it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  87%|████████▋ | 354/408 [00:00<00:00, 4729.15it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  87%|████████▋ | 355/408 [00:00<00:00, 4739.15it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 355/408 [00:00<00:00, 4736.99it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 356/408 [00:00<00:00, 4747.12it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  87%|████████▋ | 356/408 [00:00<00:00, 4744.90it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  88%|████████▊ | 357/408 [00:00<00:00, 4754.60it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  88%|████████▊ | 357/408 [00:00<00:00, 4752.44it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  88%|████████▊ | 358/408 [00:00<00:00, 4762.22it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  88%|████████▊ | 358/408 [00:00<00:00, 4759.94it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  88%|████████▊ | 359/408 [00:00<00:00, 4770.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  88%|████████▊ | 359/408 [00:00<00:00, 4767.72it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  88%|████████▊ | 360/408 [00:00<00:00, 4777.50it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.bias]    
Loading weights:  88%|████████▊ | 360/408 [00:00<00:00, 4775.36it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.bias]
Loading weights:  88%|████████▊ | 361/408 [00:00<00:00, 4785.20it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  88%|████████▊ | 361/408 [00:00<00:00, 4783.10it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  89%|████████▊ | 362/408 [00:00<00:00, 4792.93it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.bias]  
Loading weights:  89%|████████▊ | 362/408 [00:00<00:00, 4790.77it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.bias]
Loading weights:  89%|████████▉ | 363/408 [00:00<00:00, 4800.70it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 363/408 [00:00<00:00, 4798.48it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 364/408 [00:00<00:00, 4808.25it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]      
Loading weights:  89%|████████▉ | 364/408 [00:00<00:00, 4806.03it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  89%|████████▉ | 365/408 [00:00<00:00, 4815.75it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  89%|████████▉ | 365/408 [00:00<00:00, 4813.52it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  90%|████████▉ | 366/408 [00:00<00:00, 4823.34it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  90%|████████▉ | 366/408 [00:00<00:00, 4821.24it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  90%|████████▉ | 367/408 [00:00<00:00, 4831.04it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  90%|████████▉ | 367/408 [00:00<00:00, 4828.87it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  90%|█████████ | 368/408 [00:00<00:00, 4838.72it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  90%|█████████ | 368/408 [00:00<00:00, 4836.57it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  90%|█████████ | 369/408 [00:00<00:00, 4846.20it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  90%|█████████ | 369/408 [00:00<00:00, 4844.03it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  91%|█████████ | 370/408 [00:00<00:00, 4853.85it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  91%|█████████ | 370/408 [00:00<00:00, 4851.70it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  91%|█████████ | 371/408 [00:00<00:00, 4861.37it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  91%|█████████ | 371/408 [00:00<00:00, 4859.20it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  91%|█████████ | 372/408 [00:00<00:00, 4868.85it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  91%|█████████ | 372/408 [00:00<00:00, 4866.75it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  91%|█████████▏| 373/408 [00:00<00:00, 4876.32it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 373/408 [00:00<00:00, 4873.97it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 374/408 [00:00<00:00, 4883.78it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  92%|█████████▏| 374/408 [00:00<00:00, 4881.73it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  92%|█████████▏| 375/408 [00:00<00:00, 4891.40it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  92%|█████████▏| 375/408 [00:00<00:00, 4888.98it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  92%|█████████▏| 376/408 [00:00<00:00, 4898.70it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.bias]    
Loading weights:  92%|█████████▏| 376/408 [00:00<00:00, 4896.40it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.bias]
Loading weights:  92%|█████████▏| 377/408 [00:00<00:00, 4905.91it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 377/408 [00:00<00:00, 4903.79it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  93%|█████████▎| 378/408 [00:00<00:00, 4913.24it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.bias]  
Loading weights:  93%|█████████▎| 378/408 [00:00<00:00, 4910.94it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.bias]
Loading weights:  93%|█████████▎| 379/408 [00:00<00:00, 4920.35it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 379/408 [00:00<00:00, 4918.12it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 380/408 [00:00<00:00, 4927.44it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]       
Loading weights:  93%|█████████▎| 380/408 [00:00<00:00, 4925.20it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  93%|█████████▎| 381/408 [00:00<00:00, 4934.72it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  93%|█████████▎| 381/408 [00:00<00:00, 4932.50it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  94%|█████████▎| 382/408 [00:00<00:00, 4941.98it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  94%|█████████▎| 382/408 [00:00<00:00, 4939.80it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  94%|█████████▍| 383/408 [00:00<00:00, 4949.48it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  94%|█████████▍| 383/408 [00:00<00:00, 4947.16it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  94%|█████████▍| 384/408 [00:00<00:00, 4956.83it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  94%|█████████▍| 384/408 [00:00<00:00, 4954.71it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  94%|█████████▍| 385/408 [00:00<00:00, 4964.09it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 385/408 [00:00<00:00, 4961.78it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  95%|█████████▍| 386/408 [00:00<00:00, 4971.28it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  95%|█████████▍| 386/408 [00:00<00:00, 4968.91it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  95%|█████████▍| 387/408 [00:00<00:00, 4978.12it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 387/408 [00:00<00:00, 4975.88it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  95%|█████████▌| 388/408 [00:00<00:00, 4985.28it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  95%|█████████▌| 388/408 [00:00<00:00, 4982.91it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  95%|█████████▌| 389/408 [00:00<00:00, 4992.30it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  95%|█████████▌| 389/408 [00:00<00:00, 4990.12it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  96%|█████████▌| 390/408 [00:00<00:00, 4999.35it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  96%|█████████▌| 390/408 [00:00<00:00, 4997.05it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  96%|█████████▌| 391/408 [00:00<00:00, 5006.14it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  96%|█████████▌| 391/408 [00:00<00:00, 5003.78it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  96%|█████████▌| 392/408 [00:00<00:00, 5013.10it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.bias]    
Loading weights:  96%|█████████▌| 392/408 [00:00<00:00, 5010.86it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.bias]
Loading weights:  96%|█████████▋| 393/408 [00:00<00:00, 5020.04it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  96%|█████████▋| 393/408 [00:00<00:00, 5017.75it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 394/408 [00:00<00:00, 5027.12it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.bias]  
Loading weights:  97%|█████████▋| 394/408 [00:00<00:00, 5024.86it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.bias]
Loading weights:  97%|█████████▋| 395/408 [00:00<00:00, 5033.95it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 395/408 [00:00<00:00, 5031.77it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 396/408 [00:00<00:00, 5041.37it/s, Materializing param=vision_model.head.attention.in_proj_bias]              
Loading weights:  97%|█████████▋| 396/408 [00:00<00:00, 5039.32it/s, Materializing param=vision_model.head.attention.in_proj_bias]
Loading weights:  97%|█████████▋| 397/408 [00:00<00:00, 5048.64it/s, Materializing param=vision_model.head.attention.in_proj_weight]
Loading weights:  97%|█████████▋| 397/408 [00:00<00:00, 5046.21it/s, Materializing param=vision_model.head.attention.in_proj_weight]
Loading weights:  98%|█████████▊| 398/408 [00:00<00:00, 5055.69it/s, Materializing param=vision_model.head.attention.out_proj.bias] 
Loading weights:  98%|█████████▊| 398/408 [00:00<00:00, 5053.39it/s, Materializing param=vision_model.head.attention.out_proj.bias]
Loading weights:  98%|█████████▊| 399/408 [00:00<00:00, 5062.49it/s, Materializing param=vision_model.head.attention.out_proj.weight]
Loading weights:  98%|█████████▊| 399/408 [00:00<00:00, 5060.30it/s, Materializing param=vision_model.head.attention.out_proj.weight]
Loading weights:  98%|█████████▊| 400/408 [00:00<00:00, 5069.64it/s, Materializing param=vision_model.head.layernorm.bias]           
Loading weights:  98%|█████████▊| 400/408 [00:00<00:00, 5067.15it/s, Materializing param=vision_model.head.layernorm.bias]
Loading weights:  98%|█████████▊| 401/408 [00:00<00:00, 5076.65it/s, Materializing param=vision_model.head.layernorm.weight]
Loading weights:  98%|█████████▊| 401/408 [00:00<00:00, 5074.46it/s, Materializing param=vision_model.head.layernorm.weight]
Loading weights:  99%|█████████▊| 402/408 [00:00<00:00, 5084.10it/s, Materializing param=vision_model.head.mlp.fc1.bias]    
Loading weights:  99%|█████████▊| 402/408 [00:00<00:00, 5081.84it/s, Materializing param=vision_model.head.mlp.fc1.bias]
Loading weights:  99%|█████████▉| 403/408 [00:00<00:00, 5091.52it/s, Materializing param=vision_model.head.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 403/408 [00:00<00:00, 5089.59it/s, Materializing param=vision_model.head.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 404/408 [00:00<00:00, 5099.27it/s, Materializing param=vision_model.head.mlp.fc2.bias]  
Loading weights:  99%|█████████▉| 404/408 [00:00<00:00, 5095.35it/s, Materializing param=vision_model.head.mlp.fc2.bias]
Loading weights:  99%|█████████▉| 405/408 [00:00<00:00, 5104.09it/s, Materializing param=vision_model.head.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 405/408 [00:00<00:00, 5101.98it/s, Materializing param=vision_model.head.mlp.fc2.weight]
Loading weights: 100%|█████████▉| 406/408 [00:00<00:00, 5110.83it/s, Materializing param=vision_model.head.probe]         
Loading weights: 100%|█████████▉| 406/408 [00:00<00:00, 5108.53it/s, Materializing param=vision_model.head.probe]
Loading weights: 100%|█████████▉| 407/408 [00:00<00:00, 5117.83it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|█████████▉| 407/408 [00:00<00:00, 5115.57it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 5124.73it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 5122.54it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 5117.65it/s, Materializing param=vision_model.post_layernorm.weight]
+  ✓ SigLIP: google/siglip-base-patch16-224
+  ✓ Trainable: 52,988,543 / 770,333,825 parameters
+
+[Loading Dataset]
+  Loading annotations from data/coco/annotations/instances_train2017.json...
+  Loaded 4,956 images with detections
+
+[Optimizer]
+  ✓ AdamW (lr=0.0003) + scheduler
+
+============================================================
+🚀 STARTING EXTENDED TRAINING
+============================================================
+  Dataset: 4,956 samples
+  Epochs: 4
+  Learning rate: 0.0003
+
+📚 Epoch 1/4
+----------------------------------------
+  Step    50 | Loss: 4.7794 | Avg: 6.7309 | Cls: 3.239 | Box: 0.006 | IoU: 0.754 | LR: 0.000075 | 16s
+  Step   100 | Loss: 7.5686 | Avg: 6.0360 | Cls: 5.760 | Box: 0.012 | IoU: 0.874 | LR: 0.000150 | 32s
+  Step   150 | Loss: 4.3834 | Avg: 5.7159 | Cls: 3.084 | Box: 0.013 | IoU: 0.616 | LR: 0.000225 | 48s
+  Step   200 | Loss: 5.0462 | Avg: 5.5182 | Cls: 3.799 | Box: 0.023 | IoU: 0.567 | LR: 0.000300 | 64s
+  Step   250 | Loss: 4.8114 | Avg: 5.4028 | Cls: 3.179 | Box: 0.008 | IoU: 0.796 | LR: 0.000299 | 80s
+  Step   300 | Loss: 5.5303 | Avg: 5.3463 | Cls: 3.705 | Box: 0.047 | IoU: 0.796 | LR: 0.000298 | 96s
+  Step   350 | Loss: 8.8445 | Avg: 5.3213 | Cls: 7.178 | Box: 0.004 | IoU: 0.824 | LR: 0.000298 | 110s
+  Step   400 | Loss: 5.7433 | Avg: 5.2748 | Cls: 3.803 | Box: 0.021 | IoU: 0.916 | LR: 0.000297 | 129s
+  Step   450 | Loss: 3.2379 | Avg: 5.2723 | Cls: 1.582 | Box: 0.023 | IoU: 0.771 | LR: 0.000296 | 145s
+  Step   500 | Loss: 4.8529 | Avg: 5.2387 | Cls: 2.939 | Box: 0.010 | IoU: 0.932 | LR: 0.000295 | 159s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_000500
+  Step   550 | Loss: 5.5408 | Avg: 5.2050 | Cls: 3.588 | Box: 0.022 | IoU: 0.921 | LR: 0.000295 | 173s
+  Step   600 | Loss: 4.9596 | Avg: 5.1682 | Cls: 3.224 | Box: 0.006 | IoU: 0.854 | LR: 0.000294 | 187s
+  Step   650 | Loss: 5.8401 | Avg: 5.1453 | Cls: 4.221 | Box: 0.034 | IoU: 0.725 | LR: 0.000293 | 207s
+  Step   700 | Loss: 4.3935 | Avg: 5.1307 | Cls: 2.995 | Box: 0.004 | IoU: 0.690 | LR: 0.000292 | 224s
+  Step   750 | Loss: 4.4548 | Avg: 5.1094 | Cls: 2.745 | Box: 0.005 | IoU: 0.843 | LR: 0.000292 | 243s
+  Step   800 | Loss: 5.8853 | Avg: 5.0931 | Cls: 4.128 | Box: 0.011 | IoU: 0.850 | LR: 0.000291 | 270s
+  Step   850 | Loss: 4.2114 | Avg: 5.0616 | Cls: 2.920 | Box: 0.003 | IoU: 0.640 | LR: 0.000290 | 288s
+  Step   900 | Loss: 5.1706 | Avg: 5.0505 | Cls: 3.469 | Box: 0.012 | IoU: 0.822 | LR: 0.000289 | 302s
+  Step   950 | Loss: 4.2003 | Avg: 5.0401 | Cls: 2.726 | Box: 0.012 | IoU: 0.708 | LR: 0.000289 | 317s
+  Step  1000 | Loss: 3.5962 | Avg: 5.0336 | Cls: 1.867 | Box: 0.013 | IoU: 0.832 | LR: 0.000288 | 329s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_001000
+  Step  1050 | Loss: 5.4276 | Avg: 5.0226 | Cls: 3.600 | Box: 0.016 | IoU: 0.873 | LR: 0.000287 | 340s
+  Step  1100 | Loss: 5.9580 | Avg: 5.0018 | Cls: 4.097 | Box: 0.008 | IoU: 0.911 | LR: 0.000286 | 352s
+  Step  1150 | Loss: 3.4186 | Avg: 4.9847 | Cls: 2.649 | Box: 0.001 | IoU: 0.382 | LR: 0.000285 | 363s
+  Step  1200 | Loss: 4.2480 | Avg: 4.9823 | Cls: 3.132 | Box: 0.004 | IoU: 0.548 | LR: 0.000285 | 374s
+  Step  1250 | Loss: 3.7895 | Avg: 4.9674 | Cls: 2.987 | Box: 0.002 | IoU: 0.396 | LR: 0.000284 | 386s
+  Step  1300 | Loss: 5.6692 | Avg: 4.9568 | Cls: 4.081 | Box: 0.025 | IoU: 0.732 | LR: 0.000283 | 397s
+  Step  1350 | Loss: 4.5752 | Avg: 4.9448 | Cls: 3.096 | Box: 0.003 | IoU: 0.733 | LR: 0.000282 | 408s
+  Step  1400 | Loss: 5.3403 | Avg: 4.9357 | Cls: 4.250 | Box: 0.014 | IoU: 0.511 | LR: 0.000282 | 419s
+  Step  1450 | Loss: 4.7927 | Avg: 4.9252 | Cls: 3.711 | Box: 0.006 | IoU: 0.526 | LR: 0.000281 | 431s
+  Step  1500 | Loss: 5.2880 | Avg: 4.9090 | Cls: 3.284 | Box: 0.010 | IoU: 0.977 | LR: 0.000280 | 443s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_001500
+  Step  1550 | Loss: 6.1750 | Avg: 4.9020 | Cls: 3.978 | Box: 0.086 | IoU: 0.885 | LR: 0.000279 | 455s
+  Step  1600 | Loss: 3.5699 | Avg: 4.8950 | Cls: 2.968 | Box: 0.004 | IoU: 0.292 | LR: 0.000279 | 469s
+  Step  1650 | Loss: 6.3770 | Avg: 4.8861 | Cls: 5.316 | Box: 0.002 | IoU: 0.524 | LR: 0.000278 | 482s
+  Step  1700 | Loss: 5.0151 | Avg: 4.8883 | Cls: 3.824 | Box: 0.004 | IoU: 0.587 | LR: 0.000277 | 493s
+  Step  1750 | Loss: 4.9665 | Avg: 4.8865 | Cls: 3.796 | Box: 0.003 | IoU: 0.577 | LR: 0.000276 | 506s
+  Step  1800 | Loss: 3.8892 | Avg: 4.8824 | Cls: 2.365 | Box: 0.006 | IoU: 0.748 | LR: 0.000276 | 519s
+  Step  1850 | Loss: 4.1724 | Avg: 4.8793 | Cls: 2.733 | Box: 0.010 | IoU: 0.694 | LR: 0.000275 | 531s
+  Step  1900 | Loss: 5.8375 | Avg: 4.8751 | Cls: 3.926 | Box: 0.015 | IoU: 0.918 | LR: 0.000274 | 542s
+  Step  1950 | Loss: 3.3329 | Avg: 4.8708 | Cls: 1.538 | Box: 0.019 | IoU: 0.849 | LR: 0.000273 | 554s
+  Step  2000 | Loss: 3.7076 | Avg: 4.8686 | Cls: 2.367 | Box: 0.005 | IoU: 0.658 | LR: 0.000272 | 566s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_002000
+  Step  2050 | Loss: 4.6514 | Avg: 4.8620 | Cls: 3.206 | Box: 0.002 | IoU: 0.718 | LR: 0.000272 | 577s
+  Step  2100 | Loss: 6.1296 | Avg: 4.8537 | Cls: 4.834 | Box: 0.013 | IoU: 0.615 | LR: 0.000271 | 588s
+  Step  2150 | Loss: 3.8532 | Avg: 4.8444 | Cls: 2.777 | Box: 0.009 | IoU: 0.514 | LR: 0.000270 | 600s
+  Step  2200 | Loss: 4.8790 | Avg: 4.8399 | Cls: 3.684 | Box: 0.003 | IoU: 0.590 | LR: 0.000269 | 614s
+  Step  2250 | Loss: 4.4040 | Avg: 4.8340 | Cls: 3.155 | Box: 0.007 | IoU: 0.608 | LR: 0.000269 | 627s
+  Step  2300 | Loss: 3.5779 | Avg: 4.8297 | Cls: 2.996 | Box: 0.002 | IoU: 0.285 | LR: 0.000268 | 641s
+  Step  2350 | Loss: 4.9822 | Avg: 4.8253 | Cls: 3.749 | Box: 0.007 | IoU: 0.599 | LR: 0.000267 | 655s
+  Step  2400 | Loss: 4.7291 | Avg: 4.8238 | Cls: 3.250 | Box: 0.005 | IoU: 0.727 | LR: 0.000266 | 668s
+  Step  2450 | Loss: 2.6337 | Avg: 4.8195 | Cls: 2.184 | Box: 0.002 | IoU: 0.220 | LR: 0.000266 | 682s
+  Step  2500 | Loss: 4.3926 | Avg: 4.8160 | Cls: 3.347 | Box: 0.002 | IoU: 0.517 | LR: 0.000265 | 696s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_002500
+  Step  2550 | Loss: 6.5339 | Avg: 4.8134 | Cls: 5.009 | Box: 0.008 | IoU: 0.743 | LR: 0.000264 | 710s
+  Step  2600 | Loss: 4.5375 | Avg: 4.8115 | Cls: 2.626 | Box: 0.006 | IoU: 0.940 | LR: 0.000263 | 724s
+  Step  2650 | Loss: 5.4583 | Avg: 4.8083 | Cls: 3.768 | Box: 0.011 | IoU: 0.818 | LR: 0.000263 | 736s
+  Step  2700 | Loss: 4.2948 | Avg: 4.8048 | Cls: 2.628 | Box: 0.006 | IoU: 0.819 | LR: 0.000262 | 750s
+  Step  2750 | Loss: 4.7360 | Avg: 4.8026 | Cls: 3.002 | Box: 0.005 | IoU: 0.855 | LR: 0.000261 | 763s
+  Step  2800 | Loss: 5.7760 | Avg: 4.7983 | Cls: 4.062 | Box: 0.004 | IoU: 0.847 | LR: 0.000260 | 776s
+  Step  2850 | Loss: 6.3319 | Avg: 4.7974 | Cls: 4.907 | Box: 0.006 | IoU: 0.696 | LR: 0.000259 | 789s
+  Step  2900 | Loss: 4.5990 | Avg: 4.7944 | Cls: 3.204 | Box: 0.007 | IoU: 0.679 | LR: 0.000259 | 802s
+  Step  2950 | Loss: 2.6158 | Avg: 4.7917 | Cls: 1.847 | Box: 0.003 | IoU: 0.377 | LR: 0.000258 | 813s
+  Step  3000 | Loss: 6.3429 | Avg: 4.7858 | Cls: 5.393 | Box: 0.004 | IoU: 0.465 | LR: 0.000257 | 825s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_003000
+  Step  3050 | Loss: 4.5723 | Avg: 4.7827 | Cls: 3.786 | Box: 0.002 | IoU: 0.389 | LR: 0.000256 | 836s
+  Step  3100 | Loss: 4.8228 | Avg: 4.7766 | Cls: 3.268 | Box: 0.006 | IoU: 0.763 | LR: 0.000256 | 847s
+  Step  3150 | Loss: 5.2093 | Avg: 4.7722 | Cls: 3.232 | Box: 0.008 | IoU: 0.969 | LR: 0.000255 | 858s
+  Step  3200 | Loss: 3.6358 | Avg: 4.7660 | Cls: 3.245 | Box: 0.002 | IoU: 0.191 | LR: 0.000254 | 869s
+  Step  3250 | Loss: 2.5897 | Avg: 4.7649 | Cls: 2.070 | Box: 0.003 | IoU: 0.252 | LR: 0.000253 | 880s
+  Step  3300 | Loss: 6.1269 | Avg: 4.7581 | Cls: 4.740 | Box: 0.004 | IoU: 0.683 | LR: 0.000253 | 891s
+  Step  3350 | Loss: 2.0351 | Avg: 4.7539 | Cls: 1.273 | Box: 0.004 | IoU: 0.371 | LR: 0.000252 | 902s
+  Step  3400 | Loss: 4.0720 | Avg: 4.7533 | Cls: 2.603 | Box: 0.002 | IoU: 0.728 | LR: 0.000251 | 913s
+  Step  3450 | Loss: 4.5604 | Avg: 4.7502 | Cls: 2.989 | Box: 0.005 | IoU: 0.772 | LR: 0.000250 | 924s
+  Step  3500 | Loss: 4.0913 | Avg: 4.7447 | Cls: 3.011 | Box: 0.022 | IoU: 0.484 | LR: 0.000250 | 935s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_003500
+  Step  3550 | Loss: 4.2024 | Avg: 4.7366 | Cls: 2.791 | Box: 0.009 | IoU: 0.683 | LR: 0.000249 | 948s
+  Step  3600 | Loss: 4.2964 | Avg: 4.7311 | Cls: 2.550 | Box: 0.008 | IoU: 0.854 | LR: 0.000248 | 959s
+  Step  3650 | Loss: 4.2631 | Avg: 4.7268 | Cls: 2.381 | Box: 0.008 | IoU: 0.922 | LR: 0.000247 | 972s
+  Step  3700 | Loss: 4.9256 | Avg: 4.7259 | Cls: 3.847 | Box: 0.008 | IoU: 0.519 | LR: 0.000246 | 987s
+  Step  3750 | Loss: 4.9787 | Avg: 4.7235 | Cls: 3.371 | Box: 0.004 | IoU: 0.794 | LR: 0.000246 | 1001s
+  Step  3800 | Loss: 3.8837 | Avg: 4.7195 | Cls: 2.962 | Box: 0.005 | IoU: 0.450 | LR: 0.000245 | 1013s
+  Step  3850 | Loss: 5.5769 | Avg: 4.7208 | Cls: 4.725 | Box: 0.005 | IoU: 0.414 | LR: 0.000244 | 1025s
+  Step  3900 | Loss: 4.1240 | Avg: 4.7173 | Cls: 3.169 | Box: 0.004 | IoU: 0.469 | LR: 0.000243 | 1036s
+  Step  3950 | Loss: 4.9652 | Avg: 4.7133 | Cls: 3.551 | Box: 0.008 | IoU: 0.688 | LR: 0.000243 | 1048s
+  Step  4000 | Loss: 4.8884 | Avg: 4.7106 | Cls: 2.894 | Box: 0.012 | IoU: 0.968 | LR: 0.000242 | 1059s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_004000
+  Step  4050 | Loss: 3.9766 | Avg: 4.7080 | Cls: 3.258 | Box: 0.003 | IoU: 0.353 | LR: 0.000241 | 1071s
+  Step  4100 | Loss: 2.6530 | Avg: 4.7028 | Cls: 1.928 | Box: 0.005 | IoU: 0.351 | LR: 0.000240 | 1082s
+  Step  4150 | Loss: 5.8748 | Avg: 4.7022 | Cls: 4.122 | Box: 0.016 | IoU: 0.836 | LR: 0.000240 | 1093s
+  Step  4200 | Loss: 5.8855 | Avg: 4.7012 | Cls: 4.127 | Box: 0.008 | IoU: 0.859 | LR: 0.000239 | 1104s
+  Step  4250 | Loss: 4.7114 | Avg: 4.6976 | Cls: 3.239 | Box: 0.005 | IoU: 0.723 | LR: 0.000238 | 1116s
+  Step  4300 | Loss: 3.3976 | Avg: 4.6947 | Cls: 2.538 | Box: 0.002 | IoU: 0.425 | LR: 0.000237 | 1126s
+  Step  4350 | Loss: 4.8382 | Avg: 4.6914 | Cls: 3.372 | Box: 0.002 | IoU: 0.728 | LR: 0.000237 | 1137s
+  Step  4400 | Loss: 4.5373 | Avg: 4.6895 | Cls: 4.193 | Box: 0.002 | IoU: 0.168 | LR: 0.000236 | 1149s
+  Step  4450 | Loss: 4.1439 | Avg: 4.6858 | Cls: 2.850 | Box: 0.003 | IoU: 0.640 | LR: 0.000235 | 1160s
+  Step  4500 | Loss: 5.0621 | Avg: 4.6842 | Cls: 3.768 | Box: 0.004 | IoU: 0.638 | LR: 0.000234 | 1171s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_004500
+  Step  4550 | Loss: 4.6452 | Avg: 4.6811 | Cls: 3.302 | Box: 0.004 | IoU: 0.662 | LR: 0.000233 | 1183s
+  Step  4600 | Loss: 4.6811 | Avg: 4.6787 | Cls: 2.793 | Box: 0.008 | IoU: 0.924 | LR: 0.000233 | 1195s
+  Step  4650 | Loss: 4.9769 | Avg: 4.6770 | Cls: 3.318 | Box: 0.005 | IoU: 0.817 | LR: 0.000232 | 1206s
+  Step  4700 | Loss: 3.5122 | Avg: 4.6736 | Cls: 2.213 | Box: 0.002 | IoU: 0.644 | LR: 0.000231 | 1217s
+  Step  4750 | Loss: 4.2810 | Avg: 4.6704 | Cls: 2.706 | Box: 0.008 | IoU: 0.768 | LR: 0.000230 | 1228s
+  Step  4800 | Loss: 4.1430 | Avg: 4.6689 | Cls: 2.387 | Box: 0.010 | IoU: 0.854 | LR: 0.000230 | 1240s
+  Step  4850 | Loss: 3.1144 | Avg: 4.6664 | Cls: 2.406 | Box: 0.002 | IoU: 0.348 | LR: 0.000229 | 1254s
+  Step  4900 | Loss: 4.6232 | Avg: 4.6628 | Cls: 2.778 | Box: 0.007 | IoU: 0.904 | LR: 0.000228 | 1267s
+  Step  4950 | Loss: 3.9855 | Avg: 4.6612 | Cls: 2.272 | Box: 0.007 | IoU: 0.839 | LR: 0.000227 | 1280s
+
+  ✓ Epoch 1 | Avg: 4.6604 | Cls: 3.322 | Box: 0.008 | GIoU: 0.000
+
+📚 Epoch 2/4
+----------------------------------------
+  Step  5000 | Loss: 5.0539 | Avg: 4.2985 | Cls: 3.216 | Box: 0.005 | IoU: 0.906 | LR: 0.000227 | 1294s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_005000
+  Step  5050 | Loss: 3.9226 | Avg: 4.3233 | Cls: 3.337 | Box: 0.003 | IoU: 0.285 | LR: 0.000226 | 1317s
+  Step  5100 | Loss: 4.9060 | Avg: 4.4664 | Cls: 3.210 | Box: 0.006 | IoU: 0.832 | LR: 0.000225 | 1331s
+  Step  5150 | Loss: 4.6801 | Avg: 4.4174 | Cls: 3.821 | Box: 0.002 | IoU: 0.425 | LR: 0.000224 | 1347s
+  Step  5200 | Loss: 4.0333 | Avg: 4.3850 | Cls: 3.187 | Box: 0.004 | IoU: 0.412 | LR: 0.000224 | 1370s
+  Step  5250 | Loss: 3.4722 | Avg: 4.3642 | Cls: 2.337 | Box: 0.006 | IoU: 0.552 | LR: 0.000223 | 1385s
+  Step  5300 | Loss: 3.4154 | Avg: 4.3651 | Cls: 2.493 | Box: 0.006 | IoU: 0.448 | LR: 0.000222 | 1402s
+  Step  5350 | Loss: 5.2650 | Avg: 4.3729 | Cls: 3.651 | Box: 0.006 | IoU: 0.791 | LR: 0.000221 | 1422s
+  Step  5400 | Loss: 5.3575 | Avg: 4.3607 | Cls: 3.541 | Box: 0.009 | IoU: 0.886 | LR: 0.000221 | 1433s
+  Step  5450 | Loss: 4.8486 | Avg: 4.3732 | Cls: 3.114 | Box: 0.004 | IoU: 0.857 | LR: 0.000220 | 1445s
+  Step  5500 | Loss: 5.1874 | Avg: 4.3561 | Cls: 3.770 | Box: 0.003 | IoU: 0.701 | LR: 0.000219 | 1457s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_005500
+  Step  5550 | Loss: 3.1143 | Avg: 4.3704 | Cls: 2.182 | Box: 0.002 | IoU: 0.462 | LR: 0.000218 | 1469s
+  Step  5600 | Loss: 4.8955 | Avg: 4.3728 | Cls: 3.615 | Box: 0.010 | IoU: 0.615 | LR: 0.000217 | 1486s
+  Step  5650 | Loss: 3.8253 | Avg: 4.3859 | Cls: 2.542 | Box: 0.009 | IoU: 0.620 | LR: 0.000217 | 1502s
+  Step  5700 | Loss: 4.4772 | Avg: 4.3809 | Cls: 3.113 | Box: 0.003 | IoU: 0.676 | LR: 0.000216 | 1517s
+  Step  5750 | Loss: 4.3074 | Avg: 4.3775 | Cls: 3.370 | Box: 0.001 | IoU: 0.467 | LR: 0.000215 | 1532s
+  Step  5800 | Loss: 2.5549 | Avg: 4.3716 | Cls: 1.984 | Box: 0.010 | IoU: 0.261 | LR: 0.000214 | 1547s
+  Step  5850 | Loss: 3.4101 | Avg: 4.3657 | Cls: 2.187 | Box: 0.001 | IoU: 0.608 | LR: 0.000214 | 1562s
+  Step  5900 | Loss: 3.8190 | Avg: 4.3715 | Cls: 2.835 | Box: 0.003 | IoU: 0.483 | LR: 0.000213 | 1576s
+  Step  5950 | Loss: 4.2915 | Avg: 4.3615 | Cls: 3.286 | Box: 0.002 | IoU: 0.497 | LR: 0.000212 | 1592s
+  Step  6000 | Loss: 4.6421 | Avg: 4.3535 | Cls: 2.756 | Box: 0.012 | IoU: 0.914 | LR: 0.000211 | 1609s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_006000
+  Step  6050 | Loss: 3.3087 | Avg: 4.3605 | Cls: 2.043 | Box: 0.008 | IoU: 0.614 | LR: 0.000211 | 1625s
+  Step  6100 | Loss: 4.8896 | Avg: 4.3546 | Cls: 2.846 | Box: 0.014 | IoU: 0.986 | LR: 0.000210 | 1638s
+  Step  6150 | Loss: 2.1397 | Avg: 4.3550 | Cls: 1.840 | Box: 0.001 | IoU: 0.147 | LR: 0.000209 | 1652s
+  Step  6200 | Loss: 3.5667 | Avg: 4.3552 | Cls: 2.173 | Box: 0.007 | IoU: 0.680 | LR: 0.000208 | 1665s
+  Step  6250 | Loss: 3.4581 | Avg: 4.3502 | Cls: 2.674 | Box: 0.002 | IoU: 0.387 | LR: 0.000208 | 1679s
+  Step  6300 | Loss: 2.9494 | Avg: 4.3469 | Cls: 2.515 | Box: 0.003 | IoU: 0.210 | LR: 0.000207 | 1693s
+  Step  6350 | Loss: 5.0083 | Avg: 4.3479 | Cls: 3.325 | Box: 0.005 | IoU: 0.830 | LR: 0.000206 | 1707s
+  Step  6400 | Loss: 4.0776 | Avg: 4.3448 | Cls: 2.316 | Box: 0.006 | IoU: 0.865 | LR: 0.000205 | 1721s
+  Step  6450 | Loss: 4.3630 | Avg: 4.3467 | Cls: 2.430 | Box: 0.010 | IoU: 0.941 | LR: 0.000204 | 1736s
+  Step  6500 | Loss: 7.0030 | Avg: 4.3452 | Cls: 4.938 | Box: 0.042 | IoU: 0.927 | LR: 0.000204 | 1752s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_006500
+  Step  6550 | Loss: 4.5333 | Avg: 4.3462 | Cls: 3.215 | Box: 0.004 | IoU: 0.649 | LR: 0.000203 | 1767s
+  Step  6600 | Loss: 4.6084 | Avg: 4.3496 | Cls: 3.549 | Box: 0.009 | IoU: 0.507 | LR: 0.000202 | 1781s
+  Step  6650 | Loss: 4.8796 | Avg: 4.3482 | Cls: 3.394 | Box: 0.003 | IoU: 0.735 | LR: 0.000201 | 1794s
+  Step  6700 | Loss: 3.3116 | Avg: 4.3420 | Cls: 1.661 | Box: 0.014 | IoU: 0.790 | LR: 0.000201 | 1806s
+  Step  6750 | Loss: 6.7550 | Avg: 4.3408 | Cls: 4.914 | Box: 0.001 | IoU: 0.917 | LR: 0.000200 | 1819s
+  Step  6800 | Loss: 4.8760 | Avg: 4.3458 | Cls: 3.505 | Box: 0.007 | IoU: 0.668 | LR: 0.000199 | 1832s
+  Step  6850 | Loss: 4.1454 | Avg: 4.3467 | Cls: 2.550 | Box: 0.004 | IoU: 0.788 | LR: 0.000198 | 1845s
+  Step  6900 | Loss: 5.3003 | Avg: 4.3477 | Cls: 3.937 | Box: 0.011 | IoU: 0.654 | LR: 0.000198 | 1858s
+  Step  6950 | Loss: 4.4748 | Avg: 4.3512 | Cls: 3.215 | Box: 0.004 | IoU: 0.620 | LR: 0.000197 | 1871s
+  Step  7000 | Loss: 3.9507 | Avg: 4.3531 | Cls: 2.850 | Box: 0.004 | IoU: 0.541 | LR: 0.000196 | 1883s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_007000
+  Step  7050 | Loss: 4.5142 | Avg: 4.3514 | Cls: 2.936 | Box: 0.005 | IoU: 0.777 | LR: 0.000195 | 1897s
+  Step  7100 | Loss: 3.6353 | Avg: 4.3478 | Cls: 2.565 | Box: 0.008 | IoU: 0.516 | LR: 0.000195 | 1911s
+  Step  7150 | Loss: 3.0568 | Avg: 4.3462 | Cls: 2.342 | Box: 0.005 | IoU: 0.346 | LR: 0.000194 | 1927s
+  Step  7200 | Loss: 4.9521 | Avg: 4.3422 | Cls: 3.165 | Box: 0.005 | IoU: 0.882 | LR: 0.000193 | 1941s
+  Step  7250 | Loss: 5.6284 | Avg: 4.3445 | Cls: 4.119 | Box: 0.011 | IoU: 0.727 | LR: 0.000192 | 1956s
+  Step  7300 | Loss: 5.8328 | Avg: 4.3401 | Cls: 4.130 | Box: 0.007 | IoU: 0.835 | LR: 0.000191 | 1971s
+  Step  7350 | Loss: 3.8180 | Avg: 4.3405 | Cls: 2.234 | Box: 0.003 | IoU: 0.784 | LR: 0.000191 | 1991s
+  Step  7400 | Loss: 5.7180 | Avg: 4.3400 | Cls: 4.854 | Box: 0.006 | IoU: 0.417 | LR: 0.000190 | 2010s
+  Step  7450 | Loss: 3.8389 | Avg: 4.3373 | Cls: 3.047 | Box: 0.004 | IoU: 0.385 | LR: 0.000189 | 2051s
+  Step  7500 | Loss: 4.9167 | Avg: 4.3378 | Cls: 3.427 | Box: 0.003 | IoU: 0.737 | LR: 0.000188 | 2102s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_007500
+  Step  7550 | Loss: 4.8499 | Avg: 4.3412 | Cls: 3.574 | Box: 0.008 | IoU: 0.618 | LR: 0.000188 | 2138s
+  Step  7600 | Loss: 3.4087 | Avg: 4.3359 | Cls: 1.762 | Box: 0.003 | IoU: 0.816 | LR: 0.000187 | 2154s
+  Step  7650 | Loss: 4.4514 | Avg: 4.3325 | Cls: 3.006 | Box: 0.008 | IoU: 0.703 | LR: 0.000186 | 2169s
+  Step  7700 | Loss: 4.1127 | Avg: 4.3308 | Cls: 3.324 | Box: 0.001 | IoU: 0.391 | LR: 0.000185 | 2195s
+  Step  7750 | Loss: 4.6739 | Avg: 4.3287 | Cls: 3.782 | Box: 0.005 | IoU: 0.434 | LR: 0.000185 | 2213s
+  Step  7800 | Loss: 4.6908 | Avg: 4.3304 | Cls: 3.823 | Box: 0.003 | IoU: 0.426 | LR: 0.000184 | 2228s
+  Step  7850 | Loss: 6.8976 | Avg: 4.3297 | Cls: 6.135 | Box: 0.002 | IoU: 0.375 | LR: 0.000183 | 2243s
+  Step  7900 | Loss: 5.5392 | Avg: 4.3300 | Cls: 3.816 | Box: 0.025 | IoU: 0.800 | LR: 0.000182 | 2258s
+  Step  7950 | Loss: 5.1809 | Avg: 4.3292 | Cls: 3.377 | Box: 0.010 | IoU: 0.876 | LR: 0.000182 | 2274s
+  Step  8000 | Loss: 3.5965 | Avg: 4.3289 | Cls: 1.928 | Box: 0.001 | IoU: 0.831 | LR: 0.000181 | 2289s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_008000
+  Step  8050 | Loss: 4.5274 | Avg: 4.3298 | Cls: 2.820 | Box: 0.003 | IoU: 0.846 | LR: 0.000180 | 2309s
+  Step  8100 | Loss: 3.7055 | Avg: 4.3289 | Cls: 2.341 | Box: 0.004 | IoU: 0.673 | LR: 0.000179 | 2328s
+  Step  8150 | Loss: 4.4139 | Avg: 4.3297 | Cls: 3.742 | Box: 0.005 | IoU: 0.325 | LR: 0.000178 | 2345s
+  Step  8200 | Loss: 3.9615 | Avg: 4.3275 | Cls: 2.612 | Box: 0.003 | IoU: 0.666 | LR: 0.000178 | 2360s
+  Step  8250 | Loss: 3.7958 | Avg: 4.3239 | Cls: 2.528 | Box: 0.002 | IoU: 0.629 | LR: 0.000177 | 2378s
+  Step  8300 | Loss: 4.1673 | Avg: 4.3231 | Cls: 2.417 | Box: 0.008 | IoU: 0.856 | LR: 0.000176 | 2393s
+  Step  8350 | Loss: 4.9637 | Avg: 4.3253 | Cls: 3.253 | Box: 0.007 | IoU: 0.838 | LR: 0.000175 | 2408s
+  Step  8400 | Loss: 4.7408 | Avg: 4.3228 | Cls: 3.316 | Box: 0.005 | IoU: 0.701 | LR: 0.000175 | 2428s
+  Step  8450 | Loss: 3.4286 | Avg: 4.3228 | Cls: 1.679 | Box: 0.007 | IoU: 0.857 | LR: 0.000174 | 2445s
+  Step  8500 | Loss: 4.0403 | Avg: 4.3182 | Cls: 2.969 | Box: 0.006 | IoU: 0.520 | LR: 0.000173 | 2460s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_008500
+  Step  8550 | Loss: 4.0215 | Avg: 4.3161 | Cls: 2.554 | Box: 0.003 | IoU: 0.725 | LR: 0.000172 | 2475s
+  Step  8600 | Loss: 2.5064 | Avg: 4.3127 | Cls: 1.827 | Box: 0.004 | IoU: 0.329 | LR: 0.000172 | 2489s
+  Step  8650 | Loss: 4.7215 | Avg: 4.3107 | Cls: 2.876 | Box: 0.011 | IoU: 0.896 | LR: 0.000171 | 2503s
+  Step  8700 | Loss: 4.8289 | Avg: 4.3085 | Cls: 3.603 | Box: 0.004 | IoU: 0.603 | LR: 0.000170 | 2517s
+  Step  8750 | Loss: 3.6953 | Avg: 4.3093 | Cls: 2.316 | Box: 0.006 | IoU: 0.675 | LR: 0.000169 | 2531s
+  Step  8800 | Loss: 5.2642 | Avg: 4.3042 | Cls: 4.626 | Box: 0.003 | IoU: 0.311 | LR: 0.000169 | 2547s
+  Step  8850 | Loss: 5.8125 | Avg: 4.3064 | Cls: 4.324 | Box: 0.002 | IoU: 0.739 | LR: 0.000168 | 2565s
+  Step  8900 | Loss: 3.9351 | Avg: 4.3047 | Cls: 2.546 | Box: 0.026 | IoU: 0.630 | LR: 0.000167 | 2582s
+  Step  8950 | Loss: 4.0633 | Avg: 4.3021 | Cls: 2.751 | Box: 0.004 | IoU: 0.646 | LR: 0.000166 | 2599s
+  Step  9000 | Loss: 3.3548 | Avg: 4.3018 | Cls: 1.621 | Box: 0.013 | IoU: 0.835 | LR: 0.000165 | 2622s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_009000
+  Step  9050 | Loss: 4.2775 | Avg: 4.3024 | Cls: 3.134 | Box: 0.011 | IoU: 0.545 | LR: 0.000165 | 2641s
+  Step  9100 | Loss: 3.8573 | Avg: 4.3000 | Cls: 2.007 | Box: 0.012 | IoU: 0.896 | LR: 0.000164 | 2660s
+  Step  9150 | Loss: 2.4288 | Avg: 4.2987 | Cls: 1.670 | Box: 0.004 | IoU: 0.369 | LR: 0.000163 | 2675s
+  Step  9200 | Loss: 4.4381 | Avg: 4.2995 | Cls: 3.030 | Box: 0.004 | IoU: 0.693 | LR: 0.000162 | 2690s
+  Step  9250 | Loss: 3.0559 | Avg: 4.2972 | Cls: 1.547 | Box: 0.008 | IoU: 0.735 | LR: 0.000162 | 2705s
+  Step  9300 | Loss: 1.7001 | Avg: 4.2951 | Cls: 1.320 | Box: 0.002 | IoU: 0.186 | LR: 0.000161 | 2720s
+  Step  9350 | Loss: 3.9287 | Avg: 4.2950 | Cls: 2.585 | Box: 0.003 | IoU: 0.665 | LR: 0.000160 | 2734s
+  Step  9400 | Loss: 3.4947 | Avg: 4.2926 | Cls: 1.691 | Box: 0.005 | IoU: 0.890 | LR: 0.000159 | 2747s
+  Step  9450 | Loss: 3.8370 | Avg: 4.2924 | Cls: 2.746 | Box: 0.003 | IoU: 0.538 | LR: 0.000159 | 2760s
+  Step  9500 | Loss: 3.6664 | Avg: 4.2931 | Cls: 1.968 | Box: 0.005 | IoU: 0.836 | LR: 0.000158 | 2776s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_009500
+  Step  9550 | Loss: 3.5950 | Avg: 4.2950 | Cls: 1.961 | Box: 0.008 | IoU: 0.798 | LR: 0.000157 | 2797s
+  Step  9600 | Loss: 3.1260 | Avg: 4.2912 | Cls: 1.504 | Box: 0.006 | IoU: 0.796 | LR: 0.000156 | 2813s
+  Step  9650 | Loss: 4.4436 | Avg: 4.2911 | Cls: 4.024 | Box: 0.004 | IoU: 0.200 | LR: 0.000156 | 2845s
+  Step  9700 | Loss: 4.0555 | Avg: 4.2892 | Cls: 3.496 | Box: 0.002 | IoU: 0.274 | LR: 0.000155 | 2869s
+  Step  9750 | Loss: 3.7894 | Avg: 4.2887 | Cls: 2.394 | Box: 0.007 | IoU: 0.681 | LR: 0.000154 | 2888s
+  Step  9800 | Loss: 3.5666 | Avg: 4.2860 | Cls: 2.669 | Box: 0.004 | IoU: 0.439 | LR: 0.000153 | 2904s
+  Step  9850 | Loss: 3.7456 | Avg: 4.2845 | Cls: 3.245 | Box: 0.003 | IoU: 0.243 | LR: 0.000152 | 2922s
+  Step  9900 | Loss: 4.2973 | Avg: 4.2831 | Cls: 2.783 | Box: 0.011 | IoU: 0.730 | LR: 0.000152 | 2939s
+
+  ✓ Epoch 2 | Avg: 4.2835 | Cls: 2.993 | Box: 0.006 | GIoU: 0.000
+
+📚 Epoch 3/4
+----------------------------------------
+  Step  9950 | Loss: 4.5782 | Avg: 4.1283 | Cls: 3.134 | Box: 0.005 | IoU: 0.710 | LR: 0.000151 | 2956s
+  Step 10000 | Loss: 4.0603 | Avg: 4.1108 | Cls: 2.652 | Box: 0.002 | IoU: 0.700 | LR: 0.000150 | 2974s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_010000
+  Step 10050 | Loss: 5.5440 | Avg: 4.0645 | Cls: 3.850 | Box: 0.004 | IoU: 0.838 | LR: 0.000149 | 3006s
+  Step 10100 | Loss: 3.5158 | Avg: 4.0590 | Cls: 1.835 | Box: 0.003 | IoU: 0.833 | LR: 0.000149 | 3037s
+  Step 10150 | Loss: 5.7047 | Avg: 4.0548 | Cls: 4.277 | Box: 0.006 | IoU: 0.699 | LR: 0.000148 | 3065s
+  Step 10200 | Loss: 3.8182 | Avg: 4.0501 | Cls: 2.428 | Box: 0.003 | IoU: 0.688 | LR: 0.000147 | 3088s
+  Step 10250 | Loss: 2.8558 | Avg: 4.0596 | Cls: 2.208 | Box: 0.002 | IoU: 0.320 | LR: 0.000146 | 3109s
+  Step 10300 | Loss: 3.9293 | Avg: 4.0403 | Cls: 3.317 | Box: 0.001 | IoU: 0.302 | LR: 0.000146 | 3129s
+  Step 10350 | Loss: 3.9264 | Avg: 4.0739 | Cls: 2.766 | Box: 0.004 | IoU: 0.571 | LR: 0.000145 | 3148s
+  Step 10400 | Loss: 3.0491 | Avg: 4.0721 | Cls: 2.257 | Box: 0.005 | IoU: 0.383 | LR: 0.000144 | 3165s
+  Step 10450 | Loss: 3.3019 | Avg: 4.0570 | Cls: 1.776 | Box: 0.004 | IoU: 0.753 | LR: 0.000143 | 3183s
+  Step 10500 | Loss: 4.4051 | Avg: 4.0630 | Cls: 3.067 | Box: 0.010 | IoU: 0.645 | LR: 0.000143 | 3201s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_010500
+  Step 10550 | Loss: 5.1095 | Avg: 4.0721 | Cls: 4.477 | Box: 0.001 | IoU: 0.315 | LR: 0.000142 | 3222s
+  Step 10600 | Loss: 3.6369 | Avg: 4.0700 | Cls: 1.642 | Box: 0.009 | IoU: 0.975 | LR: 0.000141 | 3239s
+  Step 10650 | Loss: 4.2782 | Avg: 4.0851 | Cls: 3.491 | Box: 0.003 | IoU: 0.386 | LR: 0.000140 | 3257s
+  Step 10700 | Loss: 3.3251 | Avg: 4.0887 | Cls: 1.758 | Box: 0.012 | IoU: 0.753 | LR: 0.000139 | 3276s
+  Step 10750 | Loss: 3.6943 | Avg: 4.0969 | Cls: 2.092 | Box: 0.008 | IoU: 0.781 | LR: 0.000139 | 3295s
+  Step 10800 | Loss: 5.1162 | Avg: 4.0919 | Cls: 4.124 | Box: 0.003 | IoU: 0.489 | LR: 0.000138 | 3318s
+  Step 10850 | Loss: 4.3682 | Avg: 4.0938 | Cls: 2.797 | Box: 0.006 | IoU: 0.770 | LR: 0.000137 | 3337s
+  Step 10900 | Loss: 3.4519 | Avg: 4.1104 | Cls: 1.698 | Box: 0.007 | IoU: 0.860 | LR: 0.000136 | 3357s
+  Step 10950 | Loss: 3.9568 | Avg: 4.1198 | Cls: 3.140 | Box: 0.005 | IoU: 0.396 | LR: 0.000136 | 3375s
+  Step 11000 | Loss: 1.5071 | Avg: 4.1141 | Cls: 1.094 | Box: 0.002 | IoU: 0.201 | LR: 0.000135 | 3393s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_011000
+  Step 11050 | Loss: 4.3438 | Avg: 4.1175 | Cls: 3.191 | Box: 0.004 | IoU: 0.566 | LR: 0.000134 | 3417s
+  Step 11100 | Loss: 2.8914 | Avg: 4.1218 | Cls: 1.811 | Box: 0.006 | IoU: 0.526 | LR: 0.000133 | 3435s
+  Step 11150 | Loss: 4.1742 | Avg: 4.1275 | Cls: 2.366 | Box: 0.013 | IoU: 0.872 | LR: 0.000133 | 3452s
+  Step 11200 | Loss: 2.5832 | Avg: 4.1232 | Cls: 1.397 | Box: 0.004 | IoU: 0.583 | LR: 0.000132 | 3472s
+  Step 11250 | Loss: 3.8303 | Avg: 4.1280 | Cls: 2.790 | Box: 0.004 | IoU: 0.509 | LR: 0.000131 | 3488s
+  Step 11300 | Loss: 3.4737 | Avg: 4.1250 | Cls: 2.911 | Box: 0.005 | IoU: 0.269 | LR: 0.000130 | 3504s
+  Step 11350 | Loss: 3.6346 | Avg: 4.1211 | Cls: 2.474 | Box: 0.006 | IoU: 0.565 | LR: 0.000130 | 3525s
+  Step 11400 | Loss: 4.5717 | Avg: 4.1144 | Cls: 3.064 | Box: 0.003 | IoU: 0.746 | LR: 0.000129 | 3546s
+  Step 11450 | Loss: 2.5528 | Avg: 4.1091 | Cls: 1.717 | Box: 0.003 | IoU: 0.410 | LR: 0.000128 | 3564s
+  Step 11500 | Loss: 3.6193 | Avg: 4.1050 | Cls: 2.350 | Box: 0.004 | IoU: 0.624 | LR: 0.000127 | 3582s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_011500
+  Step 11550 | Loss: 3.8957 | Avg: 4.1080 | Cls: 2.238 | Box: 0.007 | IoU: 0.811 | LR: 0.000126 | 3606s
+  Step 11600 | Loss: 3.7695 | Avg: 4.1065 | Cls: 2.346 | Box: 0.005 | IoU: 0.700 | LR: 0.000126 | 3625s
+  Step 11650 | Loss: 3.0616 | Avg: 4.1110 | Cls: 1.766 | Box: 0.005 | IoU: 0.636 | LR: 0.000125 | 3645s
+  Step 11700 | Loss: 4.5842 | Avg: 4.1162 | Cls: 3.544 | Box: 0.003 | IoU: 0.513 | LR: 0.000124 | 3662s
+  Step 11750 | Loss: 4.7773 | Avg: 4.1188 | Cls: 3.030 | Box: 0.019 | IoU: 0.825 | LR: 0.000123 | 3678s
+  Step 11800 | Loss: 3.7201 | Avg: 4.1157 | Cls: 3.031 | Box: 0.003 | IoU: 0.336 | LR: 0.000123 | 3696s
+  Step 11850 | Loss: 4.0243 | Avg: 4.1196 | Cls: 2.738 | Box: 0.006 | IoU: 0.628 | LR: 0.000122 | 3713s
+  Step 11900 | Loss: 5.0339 | Avg: 4.1207 | Cls: 3.304 | Box: 0.013 | IoU: 0.833 | LR: 0.000121 | 3729s
+  Step 11950 | Loss: 4.2858 | Avg: 4.1234 | Cls: 3.336 | Box: 0.007 | IoU: 0.457 | LR: 0.000120 | 3745s
+  Step 12000 | Loss: 4.7264 | Avg: 4.1254 | Cls: 3.465 | Box: 0.003 | IoU: 0.624 | LR: 0.000120 | 3762s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_012000
+  Step 12050 | Loss: 3.5034 | Avg: 4.1220 | Cls: 2.736 | Box: 0.008 | IoU: 0.364 | LR: 0.000119 | 3783s
+  Step 12100 | Loss: 4.3429 | Avg: 4.1207 | Cls: 3.332 | Box: 0.007 | IoU: 0.488 | LR: 0.000118 | 3801s
+  Step 12150 | Loss: 3.7524 | Avg: 4.1196 | Cls: 2.725 | Box: 0.005 | IoU: 0.502 | LR: 0.000117 | 3817s
+  Step 12200 | Loss: 3.8066 | Avg: 4.1182 | Cls: 2.899 | Box: 0.008 | IoU: 0.434 | LR: 0.000117 | 3834s
+  Step 12250 | Loss: 2.0318 | Avg: 4.1115 | Cls: 1.350 | Box: 0.006 | IoU: 0.327 | LR: 0.000116 | 3851s
+  Step 12300 | Loss: 4.3078 | Avg: 4.1126 | Cls: 2.510 | Box: 0.006 | IoU: 0.884 | LR: 0.000115 | 3867s
+  Step 12350 | Loss: 4.3446 | Avg: 4.1188 | Cls: 2.939 | Box: 0.007 | IoU: 0.686 | LR: 0.000114 | 3883s
+  Step 12400 | Loss: 2.7838 | Avg: 4.1226 | Cls: 1.557 | Box: 0.005 | IoU: 0.601 | LR: 0.000113 | 3904s
+  Step 12450 | Loss: 4.4469 | Avg: 4.1220 | Cls: 3.362 | Box: 0.004 | IoU: 0.531 | LR: 0.000113 | 3930s
+  Step 12500 | Loss: 4.0711 | Avg: 4.1215 | Cls: 2.871 | Box: 0.004 | IoU: 0.591 | LR: 0.000112 | 3949s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_012500
+  Step 12550 | Loss: 4.1921 | Avg: 4.1198 | Cls: 3.132 | Box: 0.004 | IoU: 0.520 | LR: 0.000111 | 3969s
+  Step 12600 | Loss: 3.1628 | Avg: 4.1250 | Cls: 1.839 | Box: 0.003 | IoU: 0.655 | LR: 0.000110 | 3991s
+  Step 12650 | Loss: 5.1025 | Avg: 4.1245 | Cls: 3.502 | Box: 0.049 | IoU: 0.678 | LR: 0.000110 | 4014s
+  Step 12700 | Loss: 1.6129 | Avg: 4.1263 | Cls: 0.949 | Box: 0.003 | IoU: 0.324 | LR: 0.000109 | 4031s
+  Step 12750 | Loss: 3.8987 | Avg: 4.1290 | Cls: 2.503 | Box: 0.003 | IoU: 0.689 | LR: 0.000108 | 4049s
+  Step 12800 | Loss: 5.0270 | Avg: 4.1289 | Cls: 3.766 | Box: 0.002 | IoU: 0.625 | LR: 0.000107 | 4065s
+  Step 12850 | Loss: 3.8911 | Avg: 4.1281 | Cls: 2.197 | Box: 0.007 | IoU: 0.831 | LR: 0.000107 | 4081s
+  Step 12900 | Loss: 3.1207 | Avg: 4.1245 | Cls: 2.830 | Box: 0.002 | IoU: 0.141 | LR: 0.000106 | 4097s
+  Step 12950 | Loss: 5.8716 | Avg: 4.1242 | Cls: 4.392 | Box: 0.014 | IoU: 0.705 | LR: 0.000105 | 4113s
+  Step 13000 | Loss: 3.0947 | Avg: 4.1220 | Cls: 1.845 | Box: 0.005 | IoU: 0.613 | LR: 0.000104 | 4129s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_013000
+  Step 13050 | Loss: 4.2788 | Avg: 4.1184 | Cls: 3.068 | Box: 0.014 | IoU: 0.570 | LR: 0.000104 | 4146s
+  Step 13100 | Loss: 2.3110 | Avg: 4.1169 | Cls: 1.437 | Box: 0.006 | IoU: 0.423 | LR: 0.000103 | 4163s
+  Step 13150 | Loss: 5.2590 | Avg: 4.1188 | Cls: 3.628 | Box: 0.004 | IoU: 0.805 | LR: 0.000102 | 4179s
+  Step 13200 | Loss: 4.8153 | Avg: 4.1181 | Cls: 4.115 | Box: 0.007 | IoU: 0.333 | LR: 0.000101 | 4195s
+  Step 13250 | Loss: 3.6782 | Avg: 4.1163 | Cls: 1.996 | Box: 0.005 | IoU: 0.828 | LR: 0.000100 | 4216s
+  Step 13300 | Loss: 4.9097 | Avg: 4.1181 | Cls: 3.680 | Box: 0.005 | IoU: 0.601 | LR: 0.000100 | 4235s
+  Step 13350 | Loss: 4.3514 | Avg: 4.1188 | Cls: 3.282 | Box: 0.003 | IoU: 0.527 | LR: 0.000099 | 4256s
+  Step 13400 | Loss: 4.3934 | Avg: 4.1147 | Cls: 2.473 | Box: 0.026 | IoU: 0.895 | LR: 0.000098 | 4279s
+  Step 13450 | Loss: 6.1594 | Avg: 4.1145 | Cls: 4.151 | Box: 0.004 | IoU: 0.994 | LR: 0.000097 | 4304s
+  Step 13500 | Loss: 3.9104 | Avg: 4.1128 | Cls: 2.252 | Box: 0.003 | IoU: 0.821 | LR: 0.000097 | 4323s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_013500
+  Step 13550 | Loss: 4.8469 | Avg: 4.1101 | Cls: 3.277 | Box: 0.003 | IoU: 0.777 | LR: 0.000096 | 4341s
+  Step 13600 | Loss: 3.1789 | Avg: 4.1103 | Cls: 2.162 | Box: 0.002 | IoU: 0.502 | LR: 0.000095 | 4357s
+  Step 13650 | Loss: 3.3460 | Avg: 4.1102 | Cls: 2.262 | Box: 0.005 | IoU: 0.528 | LR: 0.000094 | 4372s
+  Step 13700 | Loss: 1.9550 | Avg: 4.1097 | Cls: 1.757 | Box: 0.001 | IoU: 0.096 | LR: 0.000094 | 4388s
+  Step 13750 | Loss: 2.7629 | Avg: 4.1107 | Cls: 2.521 | Box: 0.001 | IoU: 0.120 | LR: 0.000093 | 4403s
+  Step 13800 | Loss: 3.6602 | Avg: 4.1096 | Cls: 2.391 | Box: 0.008 | IoU: 0.615 | LR: 0.000092 | 4419s
+  Step 13850 | Loss: 1.7925 | Avg: 4.1072 | Cls: 1.350 | Box: 0.003 | IoU: 0.213 | LR: 0.000091 | 4435s
+  Step 13900 | Loss: 4.4599 | Avg: 4.1059 | Cls: 2.732 | Box: 0.015 | IoU: 0.827 | LR: 0.000091 | 4456s
+  Step 13950 | Loss: 3.7545 | Avg: 4.1021 | Cls: 2.530 | Box: 0.003 | IoU: 0.604 | LR: 0.000090 | 4475s
+  Step 14000 | Loss: 4.8568 | Avg: 4.0996 | Cls: 2.987 | Box: 0.025 | IoU: 0.873 | LR: 0.000089 | 4494s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_014000
+  Step 14050 | Loss: 4.1743 | Avg: 4.0976 | Cls: 2.678 | Box: 0.003 | IoU: 0.740 | LR: 0.000088 | 4513s
+  Step 14100 | Loss: 4.0739 | Avg: 4.0971 | Cls: 3.163 | Box: 0.003 | IoU: 0.449 | LR: 0.000088 | 4531s
+  Step 14150 | Loss: 2.8144 | Avg: 4.0986 | Cls: 1.891 | Box: 0.008 | IoU: 0.443 | LR: 0.000087 | 4547s
+  Step 14200 | Loss: 4.3598 | Avg: 4.0966 | Cls: 2.977 | Box: 0.008 | IoU: 0.671 | LR: 0.000086 | 4563s
+  Step 14250 | Loss: 3.8560 | Avg: 4.0975 | Cls: 3.395 | Box: 0.001 | IoU: 0.228 | LR: 0.000085 | 4581s
+  Step 14300 | Loss: 2.1786 | Avg: 4.0941 | Cls: 1.317 | Box: 0.005 | IoU: 0.417 | LR: 0.000084 | 4604s
+  Step 14350 | Loss: 3.6056 | Avg: 4.0913 | Cls: 2.349 | Box: 0.006 | IoU: 0.613 | LR: 0.000084 | 4622s
+  Step 14400 | Loss: 3.9531 | Avg: 4.0913 | Cls: 2.630 | Box: 0.008 | IoU: 0.642 | LR: 0.000083 | 4639s
+  Step 14450 | Loss: 3.6811 | Avg: 4.0918 | Cls: 2.300 | Box: 0.002 | IoU: 0.684 | LR: 0.000082 | 4654s
+  Step 14500 | Loss: 4.3039 | Avg: 4.0914 | Cls: 3.091 | Box: 0.017 | IoU: 0.564 | LR: 0.000081 | 4670s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_014500
+  Step 14550 | Loss: 3.2830 | Avg: 4.0913 | Cls: 2.037 | Box: 0.003 | IoU: 0.616 | LR: 0.000081 | 4686s
+  Step 14600 | Loss: 3.6496 | Avg: 4.0911 | Cls: 2.301 | Box: 0.004 | IoU: 0.664 | LR: 0.000080 | 4702s
+  Step 14650 | Loss: 2.6748 | Avg: 4.0920 | Cls: 1.662 | Box: 0.005 | IoU: 0.495 | LR: 0.000079 | 4717s
+  Step 14700 | Loss: 5.6039 | Avg: 4.0935 | Cls: 3.919 | Box: 0.004 | IoU: 0.834 | LR: 0.000078 | 4736s
+  Step 14750 | Loss: 3.2732 | Avg: 4.0911 | Cls: 2.977 | Box: 0.002 | IoU: 0.143 | LR: 0.000078 | 4755s
+  Step 14800 | Loss: 3.1603 | Avg: 4.0906 | Cls: 2.117 | Box: 0.004 | IoU: 0.511 | LR: 0.000077 | 4772s
+  Step 14850 | Loss: 5.3626 | Avg: 4.0909 | Cls: 3.683 | Box: 0.004 | IoU: 0.829 | LR: 0.000076 | 4790s
+
+  ✓ Epoch 3 | Avg: 4.0902 | Cls: 2.825 | Box: 0.005 | GIoU: 0.000
+
+📚 Epoch 4/4
+----------------------------------------
+  Step 14900 | Loss: 3.8323 | Avg: 3.9449 | Cls: 2.455 | Box: 0.004 | IoU: 0.680 | LR: 0.000075 | 4811s
+  Step 14950 | Loss: 3.6878 | Avg: 4.0264 | Cls: 1.893 | Box: 0.004 | IoU: 0.887 | LR: 0.000075 | 4828s
+  Step 15000 | Loss: 2.7908 | Avg: 3.9773 | Cls: 1.532 | Box: 0.003 | IoU: 0.623 | LR: 0.000074 | 4845s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_015000
+  Step 15050 | Loss: 4.5940 | Avg: 3.9076 | Cls: 3.064 | Box: 0.006 | IoU: 0.751 | LR: 0.000073 | 4866s
+  Step 15100 | Loss: 4.6320 | Avg: 3.9291 | Cls: 3.385 | Box: 0.002 | IoU: 0.617 | LR: 0.000072 | 4884s
+  Step 15150 | Loss: 4.5491 | Avg: 3.9472 | Cls: 2.656 | Box: 0.005 | IoU: 0.934 | LR: 0.000071 | 4902s
+  Step 15200 | Loss: 5.4190 | Avg: 3.9674 | Cls: 4.169 | Box: 0.001 | IoU: 0.622 | LR: 0.000071 | 4918s
+  Step 15250 | Loss: 5.5441 | Avg: 3.9783 | Cls: 3.699 | Box: 0.007 | IoU: 0.904 | LR: 0.000070 | 4934s
+  Step 15300 | Loss: 5.0006 | Avg: 3.9702 | Cls: 3.107 | Box: 0.007 | IoU: 0.931 | LR: 0.000069 | 4950s
+  Step 15350 | Loss: 3.2771 | Avg: 3.9640 | Cls: 2.135 | Box: 0.001 | IoU: 0.570 | LR: 0.000068 | 4970s
+  Step 15400 | Loss: 2.5933 | Avg: 3.9565 | Cls: 1.942 | Box: 0.002 | IoU: 0.319 | LR: 0.000068 | 4988s
+  Step 15450 | Loss: 2.0165 | Avg: 3.9532 | Cls: 1.173 | Box: 0.005 | IoU: 0.409 | LR: 0.000067 | 5005s
+  Step 15500 | Loss: 3.3212 | Avg: 3.9453 | Cls: 1.479 | Box: 0.011 | IoU: 0.894 | LR: 0.000066 | 5023s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_015500
+  Step 15550 | Loss: 4.5284 | Avg: 3.9432 | Cls: 2.768 | Box: 0.007 | IoU: 0.863 | LR: 0.000065 | 5053s
+  Step 15600 | Loss: 5.7495 | Avg: 3.9409 | Cls: 4.190 | Box: 0.012 | IoU: 0.750 | LR: 0.000065 | 5074s
+  Step 15650 | Loss: 3.7157 | Avg: 3.9433 | Cls: 1.856 | Box: 0.005 | IoU: 0.917 | LR: 0.000064 | 5095s
+  Step 15700 | Loss: 2.6367 | Avg: 3.9689 | Cls: 1.784 | Box: 0.005 | IoU: 0.413 | LR: 0.000063 | 5117s
+  Step 15750 | Loss: 3.6207 | Avg: 3.9675 | Cls: 1.951 | Box: 0.007 | IoU: 0.818 | LR: 0.000062 | 5138s
+  Step 15800 | Loss: 5.2881 | Avg: 3.9721 | Cls: 4.504 | Box: 0.005 | IoU: 0.379 | LR: 0.000062 | 5160s
+  Step 15850 | Loss: 3.4537 | Avg: 3.9686 | Cls: 2.248 | Box: 0.007 | IoU: 0.584 | LR: 0.000061 | 5178s
+  Step 15900 | Loss: 2.5064 | Avg: 3.9791 | Cls: 1.591 | Box: 0.004 | IoU: 0.448 | LR: 0.000060 | 5197s
+  Step 15950 | Loss: 3.5474 | Avg: 3.9805 | Cls: 2.079 | Box: 0.006 | IoU: 0.719 | LR: 0.000059 | 5216s
+  Step 16000 | Loss: 2.9473 | Avg: 3.9779 | Cls: 1.702 | Box: 0.004 | IoU: 0.614 | LR: 0.000058 | 5233s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_016000
+  Step 16050 | Loss: 5.0140 | Avg: 3.9836 | Cls: 3.938 | Box: 0.002 | IoU: 0.532 | LR: 0.000058 | 5256s
+  Step 16100 | Loss: 3.4717 | Avg: 3.9776 | Cls: 1.898 | Box: 0.004 | IoU: 0.777 | LR: 0.000057 | 5275s
+  Step 16150 | Loss: 4.1289 | Avg: 3.9727 | Cls: 2.197 | Box: 0.014 | IoU: 0.932 | LR: 0.000056 | 5292s
+  Step 16200 | Loss: 4.9936 | Avg: 3.9781 | Cls: 3.439 | Box: 0.005 | IoU: 0.766 | LR: 0.000055 | 5312s
+  Step 16250 | Loss: 5.0111 | Avg: 3.9739 | Cls: 3.029 | Box: 0.009 | IoU: 0.969 | LR: 0.000055 | 5333s
+  Step 16300 | Loss: 5.1470 | Avg: 3.9767 | Cls: 3.376 | Box: 0.009 | IoU: 0.863 | LR: 0.000054 | 5353s
+  Step 16350 | Loss: 4.0152 | Avg: 3.9768 | Cls: 2.421 | Box: 0.003 | IoU: 0.790 | LR: 0.000053 | 5371s
+  Step 16400 | Loss: 4.0697 | Avg: 3.9723 | Cls: 2.620 | Box: 0.009 | IoU: 0.703 | LR: 0.000052 | 5387s
+  Step 16450 | Loss: 3.9710 | Avg: 3.9716 | Cls: 3.134 | Box: 0.005 | IoU: 0.405 | LR: 0.000052 | 5403s
+  Step 16500 | Loss: 4.2573 | Avg: 3.9751 | Cls: 2.665 | Box: 0.003 | IoU: 0.788 | LR: 0.000051 | 5421s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_016500
+  Step 16550 | Loss: 3.7407 | Avg: 3.9737 | Cls: 2.355 | Box: 0.008 | IoU: 0.673 | LR: 0.000050 | 5442s
+  Step 16600 | Loss: 5.2276 | Avg: 3.9779 | Cls: 3.625 | Box: 0.005 | IoU: 0.789 | LR: 0.000049 | 5459s
+  Step 16650 | Loss: 5.0983 | Avg: 3.9760 | Cls: 4.226 | Box: 0.003 | IoU: 0.429 | LR: 0.000049 | 5477s
+  Step 16700 | Loss: 3.5571 | Avg: 3.9751 | Cls: 2.614 | Box: 0.008 | IoU: 0.452 | LR: 0.000048 | 5501s
+  Step 16750 | Loss: 4.3611 | Avg: 3.9734 | Cls: 3.300 | Box: 0.003 | IoU: 0.523 | LR: 0.000047 | 5526s
+  Step 16800 | Loss: 4.0940 | Avg: 3.9683 | Cls: 2.422 | Box: 0.007 | IoU: 0.818 | LR: 0.000046 | 5547s
+  Step 16850 | Loss: 4.1014 | Avg: 3.9677 | Cls: 2.391 | Box: 0.004 | IoU: 0.844 | LR: 0.000045 | 5566s
+  Step 16900 | Loss: 3.9911 | Avg: 3.9710 | Cls: 2.757 | Box: 0.001 | IoU: 0.615 | LR: 0.000045 | 5585s
+  Step 16950 | Loss: 3.0806 | Avg: 3.9724 | Cls: 1.573 | Box: 0.003 | IoU: 0.745 | LR: 0.000044 | 5609s
+  Step 17000 | Loss: 3.3062 | Avg: 3.9733 | Cls: 2.256 | Box: 0.005 | IoU: 0.512 | LR: 0.000043 | 5626s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_017000
+  Step 17050 | Loss: 5.9658 | Avg: 3.9702 | Cls: 4.131 | Box: 0.006 | IoU: 0.901 | LR: 0.000042 | 5647s
+  Step 17100 | Loss: 4.3345 | Avg: 3.9715 | Cls: 3.292 | Box: 0.004 | IoU: 0.512 | LR: 0.000042 | 5667s
+  Step 17150 | Loss: 2.9605 | Avg: 3.9658 | Cls: 2.440 | Box: 0.004 | IoU: 0.249 | LR: 0.000041 | 5686s
+  Step 17200 | Loss: 6.5828 | Avg: 3.9666 | Cls: 4.793 | Box: 0.030 | IoU: 0.819 | LR: 0.000040 | 5703s
+  Step 17250 | Loss: 4.5906 | Avg: 3.9648 | Cls: 3.327 | Box: 0.003 | IoU: 0.625 | LR: 0.000039 | 5722s
+  Step 17300 | Loss: 3.7935 | Avg: 3.9657 | Cls: 2.922 | Box: 0.002 | IoU: 0.431 | LR: 0.000039 | 5740s
+  Step 17350 | Loss: 3.9207 | Avg: 3.9679 | Cls: 2.998 | Box: 0.003 | IoU: 0.453 | LR: 0.000038 | 5759s
+  Step 17400 | Loss: 4.8106 | Avg: 3.9660 | Cls: 3.378 | Box: 0.003 | IoU: 0.709 | LR: 0.000037 | 5779s
+  Step 17450 | Loss: 5.1059 | Avg: 3.9700 | Cls: 3.766 | Box: 0.004 | IoU: 0.660 | LR: 0.000036 | 5800s
+  Step 17500 | Loss: 3.9246 | Avg: 3.9731 | Cls: 2.461 | Box: 0.003 | IoU: 0.724 | LR: 0.000036 | 5816s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_017500
+  Step 17550 | Loss: 3.2295 | Avg: 3.9697 | Cls: 2.121 | Box: 0.004 | IoU: 0.546 | LR: 0.000035 | 5842s
+  Step 17600 | Loss: 5.0517 | Avg: 3.9711 | Cls: 3.194 | Box: 0.043 | IoU: 0.820 | LR: 0.000034 | 5860s
+  Step 17650 | Loss: 2.9283 | Avg: 3.9720 | Cls: 1.513 | Box: 0.016 | IoU: 0.667 | LR: 0.000033 | 5876s
+  Step 17700 | Loss: 3.9111 | Avg: 3.9755 | Cls: 2.323 | Box: 0.003 | IoU: 0.786 | LR: 0.000032 | 5892s
+  Step 17750 | Loss: 4.1124 | Avg: 3.9750 | Cls: 2.760 | Box: 0.004 | IoU: 0.667 | LR: 0.000032 | 5908s
+  Step 17800 | Loss: 4.3672 | Avg: 3.9758 | Cls: 2.462 | Box: 0.013 | IoU: 0.921 | LR: 0.000031 | 5923s
+  Step 17850 | Loss: 5.4085 | Avg: 3.9774 | Cls: 3.503 | Box: 0.014 | IoU: 0.916 | LR: 0.000030 | 5939s
+  Step 17900 | Loss: 3.6533 | Avg: 3.9756 | Cls: 2.516 | Box: 0.003 | IoU: 0.562 | LR: 0.000030 | 5953s
+  Step 17950 | Loss: 3.3265 | Avg: 3.9741 | Cls: 1.486 | Box: 0.003 | IoU: 0.913 | LR: 0.000030 | 5970s
+  Step 18000 | Loss: 2.3841 | Avg: 3.9735 | Cls: 2.146 | Box: 0.001 | IoU: 0.118 | LR: 0.000030 | 5985s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_018000
+  Step 18050 | Loss: 4.3336 | Avg: 3.9706 | Cls: 2.998 | Box: 0.004 | IoU: 0.657 | LR: 0.000030 | 6002s
+  Step 18100 | Loss: 2.6976 | Avg: 3.9721 | Cls: 2.351 | Box: 0.002 | IoU: 0.169 | LR: 0.000030 | 6017s
+  Step 18150 | Loss: 4.1210 | Avg: 3.9736 | Cls: 2.992 | Box: 0.001 | IoU: 0.562 | LR: 0.000030 | 6033s
+  Step 18200 | Loss: 2.5959 | Avg: 3.9718 | Cls: 1.506 | Box: 0.003 | IoU: 0.538 | LR: 0.000030 | 6049s
+  Step 18250 | Loss: 4.2234 | Avg: 3.9755 | Cls: 3.003 | Box: 0.004 | IoU: 0.600 | LR: 0.000030 | 6065s
+  Step 18300 | Loss: 2.7635 | Avg: 3.9755 | Cls: 1.430 | Box: 0.001 | IoU: 0.665 | LR: 0.000030 | 6081s
+  Step 18350 | Loss: 4.5077 | Avg: 3.9766 | Cls: 3.333 | Box: 0.003 | IoU: 0.579 | LR: 0.000030 | 6098s
+  Step 18400 | Loss: 3.2303 | Avg: 3.9779 | Cls: 1.450 | Box: 0.005 | IoU: 0.878 | LR: 0.000030 | 6114s
+  Step 18450 | Loss: 4.4111 | Avg: 3.9752 | Cls: 3.285 | Box: 0.003 | IoU: 0.556 | LR: 0.000030 | 6130s
+  Step 18500 | Loss: 1.8207 | Avg: 3.9761 | Cls: 1.384 | Box: 0.000 | IoU: 0.218 | LR: 0.000030 | 6145s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_018500
+  Step 18550 | Loss: 3.2332 | Avg: 3.9750 | Cls: 2.224 | Box: 0.002 | IoU: 0.499 | LR: 0.000030 | 6165s
+  Step 18600 | Loss: 4.6925 | Avg: 3.9727 | Cls: 3.252 | Box: 0.023 | IoU: 0.663 | LR: 0.000030 | 6184s
+  Step 18650 | Loss: 3.8474 | Avg: 3.9739 | Cls: 2.713 | Box: 0.005 | IoU: 0.554 | LR: 0.000030 | 6206s
+  Step 18700 | Loss: 4.0404 | Avg: 3.9734 | Cls: 2.574 | Box: 0.006 | IoU: 0.719 | LR: 0.000030 | 6223s
+  Step 18750 | Loss: 3.2654 | Avg: 3.9722 | Cls: 2.577 | Box: 0.002 | IoU: 0.340 | LR: 0.000030 | 6241s
+  Step 18800 | Loss: 3.7921 | Avg: 3.9704 | Cls: 2.903 | Box: 0.003 | IoU: 0.437 | LR: 0.000030 | 6257s
+  Step 18850 | Loss: 5.5343 | Avg: 3.9681 | Cls: 4.236 | Box: 0.008 | IoU: 0.628 | LR: 0.000030 | 6275s
+  Step 18900 | Loss: 3.6102 | Avg: 3.9687 | Cls: 2.555 | Box: 0.003 | IoU: 0.519 | LR: 0.000030 | 6292s
+  Step 18950 | Loss: 4.1964 | Avg: 3.9661 | Cls: 2.611 | Box: 0.005 | IoU: 0.780 | LR: 0.000030 | 6309s
+  Step 19000 | Loss: 4.0742 | Avg: 3.9638 | Cls: 2.267 | Box: 0.005 | IoU: 0.892 | LR: 0.000030 | 6333s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_019000
+  Step 19050 | Loss: 1.5521 | Avg: 3.9637 | Cls: 1.091 | Box: 0.001 | IoU: 0.228 | LR: 0.000030 | 6358s
+  Step 19100 | Loss: 3.5053 | Avg: 3.9632 | Cls: 2.414 | Box: 0.004 | IoU: 0.536 | LR: 0.000030 | 6379s
+  Step 19150 | Loss: 6.0461 | Avg: 3.9609 | Cls: 4.494 | Box: 0.004 | IoU: 0.766 | LR: 0.000030 | 6402s
+  Step 19200 | Loss: 3.6006 | Avg: 3.9613 | Cls: 2.249 | Box: 0.005 | IoU: 0.664 | LR: 0.000030 | 6423s
+  Step 19250 | Loss: 3.5813 | Avg: 3.9597 | Cls: 2.955 | Box: 0.003 | IoU: 0.306 | LR: 0.000030 | 6440s
+  Step 19300 | Loss: 4.7731 | Avg: 3.9613 | Cls: 3.147 | Box: 0.003 | IoU: 0.806 | LR: 0.000030 | 6457s
+  Step 19350 | Loss: 5.3470 | Avg: 3.9603 | Cls: 4.078 | Box: 0.001 | IoU: 0.631 | LR: 0.000030 | 6472s
+  Step 19400 | Loss: 3.1382 | Avg: 3.9593 | Cls: 1.961 | Box: 0.009 | IoU: 0.567 | LR: 0.000030 | 6488s
+  Step 19450 | Loss: 5.1759 | Avg: 3.9603 | Cls: 3.906 | Box: 0.003 | IoU: 0.628 | LR: 0.000030 | 6504s
+  Step 19500 | Loss: 4.9622 | Avg: 3.9586 | Cls: 3.172 | Box: 0.007 | IoU: 0.878 | LR: 0.000030 | 6520s
+  💾 Checkpoint: checkpoints/oculus_detection_v2/step_019500
+  Step 19550 | Loss: 5.3624 | Avg: 3.9577 | Cls: 3.713 | Box: 0.002 | IoU: 0.819 | LR: 0.000030 | 6540s
+  Step 19600 | Loss: 5.0495 | Avg: 3.9596 | Cls: 4.093 | Box: 0.002 | IoU: 0.473 | LR: 0.000030 | 6556s
+  Step 19650 | Loss: 3.6313 | Avg: 3.9601 | Cls: 2.870 | Box: 0.003 | IoU: 0.373 | LR: 0.000030 | 6572s
+  Step 19700 | Loss: 3.6497 | Avg: 3.9621 | Cls: 2.538 | Box: 0.002 | IoU: 0.552 | LR: 0.000030 | 6588s
+  Step 19750 | Loss: 3.8029 | Avg: 3.9627 | Cls: 2.397 | Box: 0.005 | IoU: 0.691 | LR: 0.000030 | 6606s
+  Step 19800 | Loss: 3.4012 | Avg: 3.9649 | Cls: 2.370 | Box: 0.003 | IoU: 0.508 | LR: 0.000030 | 6656s
+
+  ✓ Epoch 4 | Avg: 3.9643 | Cls: 2.718 | Box: 0.005 | GIoU: 0.000
+
+============================================================
+💾 Saving Final Model
+============================================================
+  💾 Checkpoint: checkpoints/oculus_detection_v2/final
+✅ Training complete! Model: checkpoints/oculus_detection_v2/final
+
+============================================================
+📊 RUNNING BENCHMARKS
+============================================================
+======================================================================
+🔮 OCULUS BENCHMARK EVALUATION SUITE
+======================================================================
+Model: checkpoints/oculus_detection_v2/final
+
+[Loading Model]
+  ✓ Loaded projector from checkpoints/oculus_detection_v2/final
+  ✓ Loaded trained detection heads
+[Oculus] Loading vision encoders...
+
Loading weights:   0%|          | 0/439 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/439 [00:00<00:00, 2341.88it/s, Materializing param=embeddings.cls_token]
Loading weights:   0%|          | 1/439 [00:00<00:00, 1311.13it/s, Materializing param=embeddings.cls_token]
Loading weights:   0%|          | 2/439 [00:00<00:00, 1232.35it/s, Materializing param=embeddings.mask_token]
Loading weights:   0%|          | 2/439 [00:00<00:00, 727.04it/s, Materializing param=embeddings.mask_token] 
Loading weights:   1%|          | 3/439 [00:00<00:00, 947.29it/s, Materializing param=embeddings.patch_embeddings.projection.bias]
Loading weights:   1%|          | 3/439 [00:00<00:00, 903.36it/s, Materializing param=embeddings.patch_embeddings.projection.bias]
Loading weights:   1%|          | 4/439 [00:00<00:00, 1131.53it/s, Materializing param=embeddings.patch_embeddings.projection.weight]
Loading weights:   1%|          | 4/439 [00:00<00:00, 985.97it/s, Materializing param=embeddings.patch_embeddings.projection.weight] 
Loading weights:   1%|          | 5/439 [00:00<00:00, 913.59it/s, Materializing param=embeddings.position_embeddings]               
Loading weights:   1%|          | 5/439 [00:00<00:00, 863.59it/s, Materializing param=embeddings.position_embeddings]
Loading weights:   1%|▏         | 6/439 [00:00<00:00, 1012.02it/s, Materializing param=encoder.layer.0.attention.attention.key.bias]
Loading weights:   1%|▏         | 6/439 [00:00<00:00, 987.36it/s, Materializing param=encoder.layer.0.attention.attention.key.bias] 
Loading weights:   2%|▏         | 7/439 [00:00<00:00, 955.11it/s, Materializing param=encoder.layer.0.attention.attention.key.weight]
Loading weights:   2%|▏         | 7/439 [00:00<00:00, 892.87it/s, Materializing param=encoder.layer.0.attention.attention.key.weight]
Loading weights:   2%|▏         | 8/439 [00:00<00:00, 924.77it/s, Materializing param=encoder.layer.0.attention.attention.query.bias]
Loading weights:   2%|▏         | 8/439 [00:00<00:00, 919.55it/s, Materializing param=encoder.layer.0.attention.attention.query.bias]
Loading weights:   2%|▏         | 9/439 [00:00<00:00, 931.79it/s, Materializing param=encoder.layer.0.attention.attention.query.weight]
Loading weights:   2%|▏         | 9/439 [00:00<00:00, 924.13it/s, Materializing param=encoder.layer.0.attention.attention.query.weight]
Loading weights:   2%|▏         | 10/439 [00:00<00:00, 976.96it/s, Materializing param=encoder.layer.0.attention.attention.value.bias] 
Loading weights:   2%|▏         | 10/439 [00:00<00:00, 930.43it/s, Materializing param=encoder.layer.0.attention.attention.value.bias]
Loading weights:   3%|▎         | 11/439 [00:00<00:00, 969.09it/s, Materializing param=encoder.layer.0.attention.attention.value.weight]
Loading weights:   3%|▎         | 11/439 [00:00<00:00, 964.25it/s, Materializing param=encoder.layer.0.attention.attention.value.weight]
Loading weights:   3%|▎         | 12/439 [00:00<00:00, 1032.08it/s, Materializing param=encoder.layer.0.attention.output.dense.bias]    
Loading weights:   3%|▎         | 12/439 [00:00<00:00, 1023.65it/s, Materializing param=encoder.layer.0.attention.output.dense.bias]
Loading weights:   3%|▎         | 13/439 [00:00<00:00, 1063.32it/s, Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:   3%|▎         | 13/439 [00:00<00:00, 1032.49it/s, Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:   3%|▎         | 14/439 [00:00<00:00, 991.66it/s, Materializing param=encoder.layer.0.layer_scale1.lambda1]          
Loading weights:   3%|▎         | 14/439 [00:00<00:00, 982.19it/s, Materializing param=encoder.layer.0.layer_scale1.lambda1]
Loading weights:   3%|▎         | 15/439 [00:00<00:00, 1017.66it/s, Materializing param=encoder.layer.0.layer_scale2.lambda1]
Loading weights:   3%|▎         | 15/439 [00:00<00:00, 1010.92it/s, Materializing param=encoder.layer.0.layer_scale2.lambda1]
Loading weights:   4%|▎         | 16/439 [00:00<00:00, 1069.24it/s, Materializing param=encoder.layer.0.mlp.fc1.bias]        
Loading weights:   4%|▎         | 16/439 [00:00<00:00, 1014.71it/s, Materializing param=encoder.layer.0.mlp.fc1.bias]
Loading weights:   4%|▍         | 17/439 [00:00<00:00, 1021.45it/s, Materializing param=encoder.layer.0.mlp.fc1.weight]
Loading weights:   4%|▍         | 17/439 [00:00<00:00, 1002.55it/s, Materializing param=encoder.layer.0.mlp.fc1.weight]
Loading weights:   4%|▍         | 18/439 [00:00<00:00, 1015.98it/s, Materializing param=encoder.layer.0.mlp.fc2.bias]  
Loading weights:   4%|▍         | 18/439 [00:00<00:00, 1009.20it/s, Materializing param=encoder.layer.0.mlp.fc2.bias]
Loading weights:   4%|▍         | 19/439 [00:00<00:00, 1017.58it/s, Materializing param=encoder.layer.0.mlp.fc2.weight]
Loading weights:   4%|▍         | 19/439 [00:00<00:00, 1013.94it/s, Materializing param=encoder.layer.0.mlp.fc2.weight]
Loading weights:   5%|▍         | 20/439 [00:00<00:00, 1048.44it/s, Materializing param=encoder.layer.0.norm1.bias]    
Loading weights:   5%|▍         | 20/439 [00:00<00:00, 1045.05it/s, Materializing param=encoder.layer.0.norm1.bias]
Loading weights:   5%|▍         | 21/439 [00:00<00:00, 1031.04it/s, Materializing param=encoder.layer.0.norm1.weight]
Loading weights:   5%|▍         | 21/439 [00:00<00:00, 1027.10it/s, Materializing param=encoder.layer.0.norm1.weight]
Loading weights:   5%|▌         | 22/439 [00:00<00:00, 1039.41it/s, Materializing param=encoder.layer.0.norm2.bias]  
Loading weights:   5%|▌         | 22/439 [00:00<00:00, 1027.37it/s, Materializing param=encoder.layer.0.norm2.bias]
Loading weights:   5%|▌         | 23/439 [00:00<00:00, 1053.79it/s, Materializing param=encoder.layer.0.norm2.weight]
Loading weights:   5%|▌         | 23/439 [00:00<00:00, 1049.95it/s, Materializing param=encoder.layer.0.norm2.weight]
Loading weights:   5%|▌         | 24/439 [00:00<00:00, 1081.48it/s, Materializing param=encoder.layer.1.attention.attention.key.bias]
Loading weights:   5%|▌         | 24/439 [00:00<00:00, 1060.77it/s, Materializing param=encoder.layer.1.attention.attention.key.bias]
Loading weights:   6%|▌         | 25/439 [00:00<00:00, 1097.41it/s, Materializing param=encoder.layer.1.attention.attention.key.weight]
Loading weights:   6%|▌         | 25/439 [00:00<00:00, 1062.74it/s, Materializing param=encoder.layer.1.attention.attention.key.weight]
Loading weights:   6%|▌         | 26/439 [00:00<00:00, 1073.77it/s, Materializing param=encoder.layer.1.attention.attention.query.bias]
Loading weights:   6%|▌         | 26/439 [00:00<00:00, 1057.90it/s, Materializing param=encoder.layer.1.attention.attention.query.bias]
Loading weights:   6%|▌         | 27/439 [00:00<00:00, 1072.75it/s, Materializing param=encoder.layer.1.attention.attention.query.weight]
Loading weights:   6%|▌         | 27/439 [00:00<00:00, 1069.44it/s, Materializing param=encoder.layer.1.attention.attention.query.weight]
Loading weights:   6%|▋         | 28/439 [00:00<00:00, 1101.46it/s, Materializing param=encoder.layer.1.attention.attention.value.bias]  
Loading weights:   6%|▋         | 28/439 [00:00<00:00, 1089.75it/s, Materializing param=encoder.layer.1.attention.attention.value.bias]
Loading weights:   7%|▋         | 29/439 [00:00<00:00, 1103.33it/s, Materializing param=encoder.layer.1.attention.attention.value.weight]
Loading weights:   7%|▋         | 29/439 [00:00<00:00, 1073.60it/s, Materializing param=encoder.layer.1.attention.attention.value.weight]
Loading weights:   7%|▋         | 30/439 [00:00<00:00, 1087.36it/s, Materializing param=encoder.layer.1.attention.output.dense.bias]     
Loading weights:   7%|▋         | 30/439 [00:00<00:00, 1068.39it/s, Materializing param=encoder.layer.1.attention.output.dense.bias]
Loading weights:   7%|▋         | 31/439 [00:00<00:00, 1081.28it/s, Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 31/439 [00:00<00:00, 1071.55it/s, Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 32/439 [00:00<00:00, 1098.71it/s, Materializing param=encoder.layer.1.layer_scale1.lambda1]         
Loading weights:   7%|▋         | 32/439 [00:00<00:00, 1095.56it/s, Materializing param=encoder.layer.1.layer_scale1.lambda1]
Loading weights:   8%|▊         | 33/439 [00:00<00:00, 1120.39it/s, Materializing param=encoder.layer.1.layer_scale2.lambda1]
Loading weights:   8%|▊         | 33/439 [00:00<00:00, 1115.54it/s, Materializing param=encoder.layer.1.layer_scale2.lambda1]
Loading weights:   8%|▊         | 34/439 [00:00<00:00, 1127.66it/s, Materializing param=encoder.layer.1.mlp.fc1.bias]        
Loading weights:   8%|▊         | 34/439 [00:00<00:00, 1124.72it/s, Materializing param=encoder.layer.1.mlp.fc1.bias]
Loading weights:   8%|▊         | 35/439 [00:00<00:00, 1142.64it/s, Materializing param=encoder.layer.1.mlp.fc1.weight]
Loading weights:   8%|▊         | 35/439 [00:00<00:00, 1140.59it/s, Materializing param=encoder.layer.1.mlp.fc1.weight]
Loading weights:   8%|▊         | 36/439 [00:00<00:00, 1157.74it/s, Materializing param=encoder.layer.1.mlp.fc2.bias]  
Loading weights:   8%|▊         | 36/439 [00:00<00:00, 1151.64it/s, Materializing param=encoder.layer.1.mlp.fc2.bias]
Loading weights:   8%|▊         | 37/439 [00:00<00:00, 1160.74it/s, Materializing param=encoder.layer.1.mlp.fc2.weight]
Loading weights:   8%|▊         | 37/439 [00:00<00:00, 1159.26it/s, Materializing param=encoder.layer.1.mlp.fc2.weight]
Loading weights:   9%|▊         | 38/439 [00:00<00:00, 1155.27it/s, Materializing param=encoder.layer.1.norm1.bias]    
Loading weights:   9%|▊         | 38/439 [00:00<00:00, 1153.10it/s, Materializing param=encoder.layer.1.norm1.bias]
Loading weights:   9%|▉         | 39/439 [00:00<00:00, 1140.93it/s, Materializing param=encoder.layer.1.norm1.weight]
Loading weights:   9%|▉         | 39/439 [00:00<00:00, 1108.27it/s, Materializing param=encoder.layer.1.norm1.weight]
Loading weights:   9%|▉         | 40/439 [00:00<00:00, 1107.55it/s, Materializing param=encoder.layer.1.norm2.bias]  
Loading weights:   9%|▉         | 40/439 [00:00<00:00, 1103.88it/s, Materializing param=encoder.layer.1.norm2.bias]
Loading weights:   9%|▉         | 41/439 [00:00<00:00, 1109.46it/s, Materializing param=encoder.layer.1.norm2.weight]
Loading weights:   9%|▉         | 41/439 [00:00<00:00, 1099.34it/s, Materializing param=encoder.layer.1.norm2.weight]
Loading weights:  10%|▉         | 42/439 [00:00<00:00, 1122.99it/s, Materializing param=encoder.layer.2.attention.attention.key.bias]
Loading weights:  10%|▉         | 42/439 [00:00<00:00, 1111.38it/s, Materializing param=encoder.layer.2.attention.attention.key.bias]
Loading weights:  10%|▉         | 43/439 [00:00<00:00, 1121.87it/s, Materializing param=encoder.layer.2.attention.attention.key.weight]
Loading weights:  10%|▉         | 43/439 [00:00<00:00, 1118.43it/s, Materializing param=encoder.layer.2.attention.attention.key.weight]
Loading weights:  10%|█         | 44/439 [00:00<00:00, 1140.26it/s, Materializing param=encoder.layer.2.attention.attention.query.bias]
Loading weights:  10%|█         | 44/439 [00:00<00:00, 1131.20it/s, Materializing param=encoder.layer.2.attention.attention.query.bias]
Loading weights:  10%|█         | 45/439 [00:00<00:00, 1099.60it/s, Materializing param=encoder.layer.2.attention.attention.query.weight]
Loading weights:  10%|█         | 45/439 [00:00<00:00, 1046.81it/s, Materializing param=encoder.layer.2.attention.attention.query.weight]
Loading weights:  10%|█         | 46/439 [00:00<00:00, 1062.21it/s, Materializing param=encoder.layer.2.attention.attention.value.bias]  
Loading weights:  10%|█         | 46/439 [00:00<00:00, 1050.23it/s, Materializing param=encoder.layer.2.attention.attention.value.bias]
Loading weights:  11%|█         | 47/439 [00:00<00:00, 1061.96it/s, Materializing param=encoder.layer.2.attention.attention.value.weight]
Loading weights:  11%|█         | 47/439 [00:00<00:00, 1057.75it/s, Materializing param=encoder.layer.2.attention.attention.value.weight]
Loading weights:  11%|█         | 48/439 [00:00<00:00, 1054.25it/s, Materializing param=encoder.layer.2.attention.output.dense.bias]     
Loading weights:  11%|█         | 48/439 [00:00<00:00, 1045.58it/s, Materializing param=encoder.layer.2.attention.output.dense.bias]
Loading weights:  11%|█         | 49/439 [00:00<00:00, 1049.95it/s, Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  11%|█         | 49/439 [00:00<00:00, 1048.29it/s, Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  11%|█▏        | 50/439 [00:00<00:00, 1056.82it/s, Materializing param=encoder.layer.2.layer_scale1.lambda1]         
Loading weights:  11%|█▏        | 50/439 [00:00<00:00, 1049.98it/s, Materializing param=encoder.layer.2.layer_scale1.lambda1]
Loading weights:  12%|█▏        | 51/439 [00:00<00:00, 1060.74it/s, Materializing param=encoder.layer.2.layer_scale2.lambda1]
Loading weights:  12%|█▏        | 51/439 [00:00<00:00, 1034.36it/s, Materializing param=encoder.layer.2.layer_scale2.lambda1]
Loading weights:  12%|█▏        | 52/439 [00:00<00:00, 1038.26it/s, Materializing param=encoder.layer.2.mlp.fc1.bias]        
Loading weights:  12%|█▏        | 52/439 [00:00<00:00, 1036.83it/s, Materializing param=encoder.layer.2.mlp.fc1.bias]
Loading weights:  12%|█▏        | 53/439 [00:00<00:00, 1029.51it/s, Materializing param=encoder.layer.2.mlp.fc1.weight]
Loading weights:  12%|█▏        | 53/439 [00:00<00:00, 1026.75it/s, Materializing param=encoder.layer.2.mlp.fc1.weight]
Loading weights:  12%|█▏        | 54/439 [00:00<00:00, 1041.93it/s, Materializing param=encoder.layer.2.mlp.fc2.bias]  
Loading weights:  12%|█▏        | 54/439 [00:00<00:00, 1034.47it/s, Materializing param=encoder.layer.2.mlp.fc2.bias]
Loading weights:  13%|█▎        | 55/439 [00:00<00:00, 1032.63it/s, Materializing param=encoder.layer.2.mlp.fc2.weight]
Loading weights:  13%|█▎        | 55/439 [00:00<00:00, 1028.56it/s, Materializing param=encoder.layer.2.mlp.fc2.weight]
Loading weights:  13%|█▎        | 56/439 [00:00<00:00, 1037.83it/s, Materializing param=encoder.layer.2.norm1.bias]    
Loading weights:  13%|█▎        | 56/439 [00:00<00:00, 1034.62it/s, Materializing param=encoder.layer.2.norm1.bias]
Loading weights:  13%|█▎        | 57/439 [00:00<00:00, 1026.51it/s, Materializing param=encoder.layer.2.norm1.weight]
Loading weights:  13%|█▎        | 57/439 [00:00<00:00, 1023.75it/s, Materializing param=encoder.layer.2.norm1.weight]
Loading weights:  13%|█▎        | 58/439 [00:00<00:00, 1029.65it/s, Materializing param=encoder.layer.2.norm2.bias]  
Loading weights:  13%|█▎        | 58/439 [00:00<00:00, 1026.75it/s, Materializing param=encoder.layer.2.norm2.bias]
Loading weights:  13%|█▎        | 59/439 [00:00<00:00, 1038.57it/s, Materializing param=encoder.layer.2.norm2.weight]
Loading weights:  13%|█▎        | 59/439 [00:00<00:00, 1037.64it/s, Materializing param=encoder.layer.2.norm2.weight]
Loading weights:  14%|█▎        | 60/439 [00:00<00:00, 1051.82it/s, Materializing param=encoder.layer.3.attention.attention.key.bias]
Loading weights:  14%|█▎        | 60/439 [00:00<00:00, 1050.94it/s, Materializing param=encoder.layer.3.attention.attention.key.bias]
Loading weights:  14%|█▍        | 61/439 [00:00<00:00, 1060.92it/s, Materializing param=encoder.layer.3.attention.attention.key.weight]
Loading weights:  14%|█▍        | 61/439 [00:00<00:00, 1058.30it/s, Materializing param=encoder.layer.3.attention.attention.key.weight]
Loading weights:  14%|█▍        | 62/439 [00:00<00:00, 1059.80it/s, Materializing param=encoder.layer.3.attention.attention.query.bias]
Loading weights:  14%|█▍        | 62/439 [00:00<00:00, 1057.75it/s, Materializing param=encoder.layer.3.attention.attention.query.bias]
Loading weights:  14%|█▍        | 63/439 [00:00<00:00, 1067.09it/s, Materializing param=encoder.layer.3.attention.attention.query.weight]
Loading weights:  14%|█▍        | 63/439 [00:00<00:00, 1063.08it/s, Materializing param=encoder.layer.3.attention.attention.query.weight]
Loading weights:  15%|█▍        | 64/439 [00:00<00:00, 1069.24it/s, Materializing param=encoder.layer.3.attention.attention.value.bias]  
Loading weights:  15%|█▍        | 64/439 [00:00<00:00, 1062.79it/s, Materializing param=encoder.layer.3.attention.attention.value.bias]
Loading weights:  15%|█▍        | 65/439 [00:00<00:00, 1063.38it/s, Materializing param=encoder.layer.3.attention.attention.value.weight]
Loading weights:  15%|█▍        | 65/439 [00:00<00:00, 1056.03it/s, Materializing param=encoder.layer.3.attention.attention.value.weight]
Loading weights:  15%|█▌        | 66/439 [00:00<00:00, 1060.87it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]     
Loading weights:  15%|█▌        | 66/439 [00:00<00:00, 1059.44it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]
Loading weights:  15%|█▌        | 67/439 [00:00<00:00, 1065.19it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  15%|█▌        | 67/439 [00:00<00:00, 1056.70it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  15%|█▌        | 68/439 [00:00<00:00, 1059.79it/s, Materializing param=encoder.layer.3.layer_scale1.lambda1]         
Loading weights:  15%|█▌        | 68/439 [00:00<00:00, 1058.53it/s, Materializing param=encoder.layer.3.layer_scale1.lambda1]
Loading weights:  16%|█▌        | 69/439 [00:00<00:00, 1072.76it/s, Materializing param=encoder.layer.3.layer_scale2.lambda1]
Loading weights:  16%|█▌        | 69/439 [00:00<00:00, 1071.28it/s, Materializing param=encoder.layer.3.layer_scale2.lambda1]
Loading weights:  16%|█▌        | 70/439 [00:00<00:00, 1075.95it/s, Materializing param=encoder.layer.3.mlp.fc1.bias]        
Loading weights:  16%|█▌        | 70/439 [00:00<00:00, 1075.29it/s, Materializing param=encoder.layer.3.mlp.fc1.bias]
Loading weights:  16%|█▌        | 71/439 [00:00<00:00, 1085.23it/s, Materializing param=encoder.layer.3.mlp.fc1.weight]
Loading weights:  16%|█▌        | 71/439 [00:00<00:00, 1084.07it/s, Materializing param=encoder.layer.3.mlp.fc1.weight]
Loading weights:  16%|█▋        | 72/439 [00:00<00:00, 1091.92it/s, Materializing param=encoder.layer.3.mlp.fc2.bias]  
Loading weights:  16%|█▋        | 72/439 [00:00<00:00, 1091.19it/s, Materializing param=encoder.layer.3.mlp.fc2.bias]
Loading weights:  17%|█▋        | 73/439 [00:00<00:00, 1105.26it/s, Materializing param=encoder.layer.3.mlp.fc2.weight]
Loading weights:  17%|█▋        | 73/439 [00:00<00:00, 1104.66it/s, Materializing param=encoder.layer.3.mlp.fc2.weight]
Loading weights:  17%|█▋        | 74/439 [00:00<00:00, 1118.79it/s, Materializing param=encoder.layer.3.norm1.bias]    
Loading weights:  17%|█▋        | 74/439 [00:00<00:00, 1118.18it/s, Materializing param=encoder.layer.3.norm1.bias]
Loading weights:  17%|█▋        | 75/439 [00:00<00:00, 1131.75it/s, Materializing param=encoder.layer.3.norm1.weight]
Loading weights:  17%|█▋        | 75/439 [00:00<00:00, 1131.17it/s, Materializing param=encoder.layer.3.norm1.weight]
Loading weights:  17%|█▋        | 76/439 [00:00<00:00, 1144.98it/s, Materializing param=encoder.layer.3.norm2.bias]  
Loading weights:  17%|█▋        | 76/439 [00:00<00:00, 1144.05it/s, Materializing param=encoder.layer.3.norm2.bias]
Loading weights:  18%|█▊        | 77/439 [00:00<00:00, 1157.22it/s, Materializing param=encoder.layer.3.norm2.weight]
Loading weights:  18%|█▊        | 77/439 [00:00<00:00, 1156.03it/s, Materializing param=encoder.layer.3.norm2.weight]
Loading weights:  18%|█▊        | 78/439 [00:00<00:00, 1169.54it/s, Materializing param=encoder.layer.4.attention.attention.key.bias]
Loading weights:  18%|█▊        | 78/439 [00:00<00:00, 1168.79it/s, Materializing param=encoder.layer.4.attention.attention.key.bias]
Loading weights:  18%|█▊        | 79/439 [00:00<00:00, 1182.59it/s, Materializing param=encoder.layer.4.attention.attention.key.weight]
Loading weights:  18%|█▊        | 79/439 [00:00<00:00, 1181.91it/s, Materializing param=encoder.layer.4.attention.attention.key.weight]
Loading weights:  18%|█▊        | 80/439 [00:00<00:00, 1195.75it/s, Materializing param=encoder.layer.4.attention.attention.query.bias]
Loading weights:  18%|█▊        | 80/439 [00:00<00:00, 1195.03it/s, Materializing param=encoder.layer.4.attention.attention.query.bias]
Loading weights:  18%|█▊        | 81/439 [00:00<00:00, 1208.72it/s, Materializing param=encoder.layer.4.attention.attention.query.weight]
Loading weights:  18%|█▊        | 81/439 [00:00<00:00, 1207.98it/s, Materializing param=encoder.layer.4.attention.attention.query.weight]
Loading weights:  19%|█▊        | 82/439 [00:00<00:00, 1221.70it/s, Materializing param=encoder.layer.4.attention.attention.value.bias]  
Loading weights:  19%|█▊        | 82/439 [00:00<00:00, 1221.02it/s, Materializing param=encoder.layer.4.attention.attention.value.bias]
Loading weights:  19%|█▉        | 83/439 [00:00<00:00, 1234.88it/s, Materializing param=encoder.layer.4.attention.attention.value.weight]
Loading weights:  19%|█▉        | 83/439 [00:00<00:00, 1234.24it/s, Materializing param=encoder.layer.4.attention.attention.value.weight]
Loading weights:  19%|█▉        | 84/439 [00:00<00:00, 1248.11it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]     
Loading weights:  19%|█▉        | 84/439 [00:00<00:00, 1247.42it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]
Loading weights:  19%|█▉        | 85/439 [00:00<00:00, 1261.20it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  19%|█▉        | 85/439 [00:00<00:00, 1260.62it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  20%|█▉        | 86/439 [00:00<00:00, 1274.51it/s, Materializing param=encoder.layer.4.layer_scale1.lambda1]         
Loading weights:  20%|█▉        | 86/439 [00:00<00:00, 1273.93it/s, Materializing param=encoder.layer.4.layer_scale1.lambda1]
Loading weights:  20%|█▉        | 87/439 [00:00<00:00, 1287.80it/s, Materializing param=encoder.layer.4.layer_scale2.lambda1]
Loading weights:  20%|█▉        | 87/439 [00:00<00:00, 1287.21it/s, Materializing param=encoder.layer.4.layer_scale2.lambda1]
Loading weights:  20%|██        | 88/439 [00:00<00:00, 1300.93it/s, Materializing param=encoder.layer.4.mlp.fc1.bias]        
Loading weights:  20%|██        | 88/439 [00:00<00:00, 1300.32it/s, Materializing param=encoder.layer.4.mlp.fc1.bias]
Loading weights:  20%|██        | 89/439 [00:00<00:00, 1314.12it/s, Materializing param=encoder.layer.4.mlp.fc1.weight]
Loading weights:  20%|██        | 89/439 [00:00<00:00, 1313.48it/s, Materializing param=encoder.layer.4.mlp.fc1.weight]
Loading weights:  21%|██        | 90/439 [00:00<00:00, 1327.26it/s, Materializing param=encoder.layer.4.mlp.fc2.bias]  
Loading weights:  21%|██        | 90/439 [00:00<00:00, 1326.61it/s, Materializing param=encoder.layer.4.mlp.fc2.bias]
Loading weights:  21%|██        | 91/439 [00:00<00:00, 1340.23it/s, Materializing param=encoder.layer.4.mlp.fc2.weight]
Loading weights:  21%|██        | 91/439 [00:00<00:00, 1339.54it/s, Materializing param=encoder.layer.4.mlp.fc2.weight]
Loading weights:  21%|██        | 92/439 [00:00<00:00, 1353.22it/s, Materializing param=encoder.layer.4.norm1.bias]    
Loading weights:  21%|██        | 92/439 [00:00<00:00, 1352.55it/s, Materializing param=encoder.layer.4.norm1.bias]
Loading weights:  21%|██        | 93/439 [00:00<00:00, 1366.20it/s, Materializing param=encoder.layer.4.norm1.weight]
Loading weights:  21%|██        | 93/439 [00:00<00:00, 1365.42it/s, Materializing param=encoder.layer.4.norm1.weight]
Loading weights:  21%|██▏       | 94/439 [00:00<00:00, 1379.07it/s, Materializing param=encoder.layer.4.norm2.bias]  
Loading weights:  21%|██▏       | 94/439 [00:00<00:00, 1378.38it/s, Materializing param=encoder.layer.4.norm2.bias]
Loading weights:  22%|██▏       | 95/439 [00:00<00:00, 1392.04it/s, Materializing param=encoder.layer.4.norm2.weight]
Loading weights:  22%|██▏       | 95/439 [00:00<00:00, 1391.41it/s, Materializing param=encoder.layer.4.norm2.weight]
Loading weights:  22%|██▏       | 96/439 [00:00<00:00, 1405.09it/s, Materializing param=encoder.layer.5.attention.attention.key.bias]
Loading weights:  22%|██▏       | 96/439 [00:00<00:00, 1404.39it/s, Materializing param=encoder.layer.5.attention.attention.key.bias]
Loading weights:  22%|██▏       | 97/439 [00:00<00:00, 1417.90it/s, Materializing param=encoder.layer.5.attention.attention.key.weight]
Loading weights:  22%|██▏       | 97/439 [00:00<00:00, 1417.24it/s, Materializing param=encoder.layer.5.attention.attention.key.weight]
Loading weights:  22%|██▏       | 98/439 [00:00<00:00, 1430.70it/s, Materializing param=encoder.layer.5.attention.attention.query.bias]
Loading weights:  22%|██▏       | 98/439 [00:00<00:00, 1430.01it/s, Materializing param=encoder.layer.5.attention.attention.query.bias]
Loading weights:  23%|██▎       | 99/439 [00:00<00:00, 1435.66it/s, Materializing param=encoder.layer.5.attention.attention.query.weight]
Loading weights:  23%|██▎       | 99/439 [00:00<00:00, 1434.47it/s, Materializing param=encoder.layer.5.attention.attention.query.weight]
Loading weights:  23%|██▎       | 100/439 [00:00<00:00, 1447.47it/s, Materializing param=encoder.layer.5.attention.attention.value.bias] 
Loading weights:  23%|██▎       | 100/439 [00:00<00:00, 1446.70it/s, Materializing param=encoder.layer.5.attention.attention.value.bias]
Loading weights:  23%|██▎       | 101/439 [00:00<00:00, 1459.79it/s, Materializing param=encoder.layer.5.attention.attention.value.weight]
Loading weights:  23%|██▎       | 101/439 [00:00<00:00, 1459.03it/s, Materializing param=encoder.layer.5.attention.attention.value.weight]
Loading weights:  23%|██▎       | 102/439 [00:00<00:00, 1472.25it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]     
Loading weights:  23%|██▎       | 102/439 [00:00<00:00, 1471.54it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]
Loading weights:  23%|██▎       | 103/439 [00:00<00:00, 1483.98it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  23%|██▎       | 103/439 [00:00<00:00, 1482.92it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  24%|██▎       | 104/439 [00:00<00:00, 1495.59it/s, Materializing param=encoder.layer.5.layer_scale1.lambda1]         
Loading weights:  24%|██▎       | 104/439 [00:00<00:00, 1494.79it/s, Materializing param=encoder.layer.5.layer_scale1.lambda1]
Loading weights:  24%|██▍       | 105/439 [00:00<00:00, 1507.91it/s, Materializing param=encoder.layer.5.layer_scale2.lambda1]
Loading weights:  24%|██▍       | 105/439 [00:00<00:00, 1507.17it/s, Materializing param=encoder.layer.5.layer_scale2.lambda1]
Loading weights:  24%|██▍       | 106/439 [00:00<00:00, 1520.35it/s, Materializing param=encoder.layer.5.mlp.fc1.bias]        
Loading weights:  24%|██▍       | 106/439 [00:00<00:00, 1519.59it/s, Materializing param=encoder.layer.5.mlp.fc1.bias]
Loading weights:  24%|██▍       | 107/439 [00:00<00:00, 1532.67it/s, Materializing param=encoder.layer.5.mlp.fc1.weight]
Loading weights:  24%|██▍       | 107/439 [00:00<00:00, 1531.90it/s, Materializing param=encoder.layer.5.mlp.fc1.weight]
Loading weights:  25%|██▍       | 108/439 [00:00<00:00, 1544.21it/s, Materializing param=encoder.layer.5.mlp.fc2.bias]  
Loading weights:  25%|██▍       | 108/439 [00:00<00:00, 1543.43it/s, Materializing param=encoder.layer.5.mlp.fc2.bias]
Loading weights:  25%|██▍       | 109/439 [00:00<00:00, 1556.68it/s, Materializing param=encoder.layer.5.mlp.fc2.weight]
Loading weights:  25%|██▍       | 109/439 [00:00<00:00, 1555.97it/s, Materializing param=encoder.layer.5.mlp.fc2.weight]
Loading weights:  25%|██▌       | 110/439 [00:00<00:00, 1569.07it/s, Materializing param=encoder.layer.5.norm1.bias]    
Loading weights:  25%|██▌       | 110/439 [00:00<00:00, 1568.27it/s, Materializing param=encoder.layer.5.norm1.bias]
Loading weights:  25%|██▌       | 111/439 [00:00<00:00, 1581.27it/s, Materializing param=encoder.layer.5.norm1.weight]
Loading weights:  25%|██▌       | 111/439 [00:00<00:00, 1580.43it/s, Materializing param=encoder.layer.5.norm1.weight]
Loading weights:  26%|██▌       | 112/439 [00:00<00:00, 1593.58it/s, Materializing param=encoder.layer.5.norm2.bias]  
Loading weights:  26%|██▌       | 112/439 [00:00<00:00, 1592.83it/s, Materializing param=encoder.layer.5.norm2.bias]
Loading weights:  26%|██▌       | 113/439 [00:00<00:00, 1605.96it/s, Materializing param=encoder.layer.5.norm2.weight]
Loading weights:  26%|██▌       | 113/439 [00:00<00:00, 1605.21it/s, Materializing param=encoder.layer.5.norm2.weight]
Loading weights:  26%|██▌       | 114/439 [00:00<00:00, 1618.31it/s, Materializing param=encoder.layer.6.attention.attention.key.bias]
Loading weights:  26%|██▌       | 114/439 [00:00<00:00, 1617.51it/s, Materializing param=encoder.layer.6.attention.attention.key.bias]
Loading weights:  26%|██▌       | 115/439 [00:00<00:00, 1630.46it/s, Materializing param=encoder.layer.6.attention.attention.key.weight]
Loading weights:  26%|██▌       | 115/439 [00:00<00:00, 1629.69it/s, Materializing param=encoder.layer.6.attention.attention.key.weight]
Loading weights:  26%|██▋       | 116/439 [00:00<00:00, 1642.64it/s, Materializing param=encoder.layer.6.attention.attention.query.bias]
Loading weights:  26%|██▋       | 116/439 [00:00<00:00, 1641.86it/s, Materializing param=encoder.layer.6.attention.attention.query.bias]
Loading weights:  27%|██▋       | 117/439 [00:00<00:00, 1654.81it/s, Materializing param=encoder.layer.6.attention.attention.query.weight]
Loading weights:  27%|██▋       | 117/439 [00:00<00:00, 1653.99it/s, Materializing param=encoder.layer.6.attention.attention.query.weight]
Loading weights:  27%|██▋       | 118/439 [00:00<00:00, 1666.88it/s, Materializing param=encoder.layer.6.attention.attention.value.bias]  
Loading weights:  27%|██▋       | 118/439 [00:00<00:00, 1666.08it/s, Materializing param=encoder.layer.6.attention.attention.value.bias]
Loading weights:  27%|██▋       | 119/439 [00:00<00:00, 1678.95it/s, Materializing param=encoder.layer.6.attention.attention.value.weight]
Loading weights:  27%|██▋       | 119/439 [00:00<00:00, 1678.12it/s, Materializing param=encoder.layer.6.attention.attention.value.weight]
Loading weights:  27%|██▋       | 120/439 [00:00<00:00, 1691.00it/s, Materializing param=encoder.layer.6.attention.output.dense.bias]     
Loading weights:  27%|██▋       | 120/439 [00:00<00:00, 1690.19it/s, Materializing param=encoder.layer.6.attention.output.dense.bias]
Loading weights:  28%|██▊       | 121/439 [00:00<00:00, 1703.06it/s, Materializing param=encoder.layer.6.attention.output.dense.weight]
Loading weights:  28%|██▊       | 121/439 [00:00<00:00, 1702.24it/s, Materializing param=encoder.layer.6.attention.output.dense.weight]
Loading weights:  28%|██▊       | 122/439 [00:00<00:00, 1715.05it/s, Materializing param=encoder.layer.6.layer_scale1.lambda1]         
Loading weights:  28%|██▊       | 122/439 [00:00<00:00, 1714.25it/s, Materializing param=encoder.layer.6.layer_scale1.lambda1]
Loading weights:  28%|██▊       | 123/439 [00:00<00:00, 1727.10it/s, Materializing param=encoder.layer.6.layer_scale2.lambda1]
Loading weights:  28%|██▊       | 123/439 [00:00<00:00, 1726.29it/s, Materializing param=encoder.layer.6.layer_scale2.lambda1]
Loading weights:  28%|██▊       | 124/439 [00:00<00:00, 1739.09it/s, Materializing param=encoder.layer.6.mlp.fc1.bias]        
Loading weights:  28%|██▊       | 124/439 [00:00<00:00, 1738.26it/s, Materializing param=encoder.layer.6.mlp.fc1.bias]
Loading weights:  28%|██▊       | 125/439 [00:00<00:00, 1751.02it/s, Materializing param=encoder.layer.6.mlp.fc1.weight]
Loading weights:  28%|██▊       | 125/439 [00:00<00:00, 1750.19it/s, Materializing param=encoder.layer.6.mlp.fc1.weight]
Loading weights:  29%|██▊       | 126/439 [00:00<00:00, 1762.88it/s, Materializing param=encoder.layer.6.mlp.fc2.bias]  
Loading weights:  29%|██▊       | 126/439 [00:00<00:00, 1762.04it/s, Materializing param=encoder.layer.6.mlp.fc2.bias]
Loading weights:  29%|██▉       | 127/439 [00:00<00:00, 1774.79it/s, Materializing param=encoder.layer.6.mlp.fc2.weight]
Loading weights:  29%|██▉       | 127/439 [00:00<00:00, 1773.94it/s, Materializing param=encoder.layer.6.mlp.fc2.weight]
Loading weights:  29%|██▉       | 128/439 [00:00<00:00, 1786.64it/s, Materializing param=encoder.layer.6.norm1.bias]    
Loading weights:  29%|██▉       | 128/439 [00:00<00:00, 1785.69it/s, Materializing param=encoder.layer.6.norm1.bias]
Loading weights:  29%|██▉       | 129/439 [00:00<00:00, 1798.41it/s, Materializing param=encoder.layer.6.norm1.weight]
Loading weights:  29%|██▉       | 129/439 [00:00<00:00, 1797.56it/s, Materializing param=encoder.layer.6.norm1.weight]
Loading weights:  30%|██▉       | 130/439 [00:00<00:00, 1810.28it/s, Materializing param=encoder.layer.6.norm2.bias]  
Loading weights:  30%|██▉       | 130/439 [00:00<00:00, 1809.48it/s, Materializing param=encoder.layer.6.norm2.bias]
Loading weights:  30%|██▉       | 131/439 [00:00<00:00, 1822.05it/s, Materializing param=encoder.layer.6.norm2.weight]
Loading weights:  30%|██▉       | 131/439 [00:00<00:00, 1821.20it/s, Materializing param=encoder.layer.6.norm2.weight]
Loading weights:  30%|███       | 132/439 [00:00<00:00, 1833.84it/s, Materializing param=encoder.layer.7.attention.attention.key.bias]
Loading weights:  30%|███       | 132/439 [00:00<00:00, 1832.98it/s, Materializing param=encoder.layer.7.attention.attention.key.bias]
Loading weights:  30%|███       | 133/439 [00:00<00:00, 1845.46it/s, Materializing param=encoder.layer.7.attention.attention.key.weight]
Loading weights:  30%|███       | 133/439 [00:00<00:00, 1844.56it/s, Materializing param=encoder.layer.7.attention.attention.key.weight]
Loading weights:  31%|███       | 134/439 [00:00<00:00, 1856.81it/s, Materializing param=encoder.layer.7.attention.attention.query.bias]
Loading weights:  31%|███       | 134/439 [00:00<00:00, 1855.94it/s, Materializing param=encoder.layer.7.attention.attention.query.bias]
Loading weights:  31%|███       | 135/439 [00:00<00:00, 1868.41it/s, Materializing param=encoder.layer.7.attention.attention.query.weight]
Loading weights:  31%|███       | 135/439 [00:00<00:00, 1867.56it/s, Materializing param=encoder.layer.7.attention.attention.query.weight]
Loading weights:  31%|███       | 136/439 [00:00<00:00, 1879.86it/s, Materializing param=encoder.layer.7.attention.attention.value.bias]  
Loading weights:  31%|███       | 136/439 [00:00<00:00, 1878.98it/s, Materializing param=encoder.layer.7.attention.attention.value.bias]
Loading weights:  31%|███       | 137/439 [00:00<00:00, 1891.43it/s, Materializing param=encoder.layer.7.attention.attention.value.weight]
Loading weights:  31%|███       | 137/439 [00:00<00:00, 1890.54it/s, Materializing param=encoder.layer.7.attention.attention.value.weight]
Loading weights:  31%|███▏      | 138/439 [00:00<00:00, 1902.98it/s, Materializing param=encoder.layer.7.attention.output.dense.bias]     
Loading weights:  31%|███▏      | 138/439 [00:00<00:00, 1902.03it/s, Materializing param=encoder.layer.7.attention.output.dense.bias]
Loading weights:  32%|███▏      | 139/439 [00:00<00:00, 1914.28it/s, Materializing param=encoder.layer.7.attention.output.dense.weight]
Loading weights:  32%|███▏      | 139/439 [00:00<00:00, 1913.40it/s, Materializing param=encoder.layer.7.attention.output.dense.weight]
Loading weights:  32%|███▏      | 140/439 [00:00<00:00, 1925.67it/s, Materializing param=encoder.layer.7.layer_scale1.lambda1]         
Loading weights:  32%|███▏      | 140/439 [00:00<00:00, 1924.77it/s, Materializing param=encoder.layer.7.layer_scale1.lambda1]
Loading weights:  32%|███▏      | 141/439 [00:00<00:00, 1937.19it/s, Materializing param=encoder.layer.7.layer_scale2.lambda1]
Loading weights:  32%|███▏      | 141/439 [00:00<00:00, 1936.34it/s, Materializing param=encoder.layer.7.layer_scale2.lambda1]
Loading weights:  32%|███▏      | 142/439 [00:00<00:00, 1948.60it/s, Materializing param=encoder.layer.7.mlp.fc1.bias]        
Loading weights:  32%|███▏      | 142/439 [00:00<00:00, 1947.72it/s, Materializing param=encoder.layer.7.mlp.fc1.bias]
Loading weights:  33%|███▎      | 143/439 [00:00<00:00, 1960.06it/s, Materializing param=encoder.layer.7.mlp.fc1.weight]
Loading weights:  33%|███▎      | 143/439 [00:00<00:00, 1959.17it/s, Materializing param=encoder.layer.7.mlp.fc1.weight]
Loading weights:  33%|███▎      | 144/439 [00:00<00:00, 1971.39it/s, Materializing param=encoder.layer.7.mlp.fc2.bias]  
Loading weights:  33%|███▎      | 144/439 [00:00<00:00, 1970.47it/s, Materializing param=encoder.layer.7.mlp.fc2.bias]
Loading weights:  33%|███▎      | 145/439 [00:00<00:00, 1982.80it/s, Materializing param=encoder.layer.7.mlp.fc2.weight]
Loading weights:  33%|███▎      | 145/439 [00:00<00:00, 1981.91it/s, Materializing param=encoder.layer.7.mlp.fc2.weight]
Loading weights:  33%|███▎      | 146/439 [00:00<00:00, 1994.24it/s, Materializing param=encoder.layer.7.norm1.bias]    
Loading weights:  33%|███▎      | 146/439 [00:00<00:00, 1993.34it/s, Materializing param=encoder.layer.7.norm1.bias]
Loading weights:  33%|███▎      | 147/439 [00:00<00:00, 2005.65it/s, Materializing param=encoder.layer.7.norm1.weight]
Loading weights:  33%|███▎      | 147/439 [00:00<00:00, 2004.75it/s, Materializing param=encoder.layer.7.norm1.weight]
Loading weights:  34%|███▎      | 148/439 [00:00<00:00, 2017.04it/s, Materializing param=encoder.layer.7.norm2.bias]  
Loading weights:  34%|███▎      | 148/439 [00:00<00:00, 2016.11it/s, Materializing param=encoder.layer.7.norm2.bias]
Loading weights:  34%|███▍      | 149/439 [00:00<00:00, 2028.18it/s, Materializing param=encoder.layer.7.norm2.weight]
Loading weights:  34%|███▍      | 149/439 [00:00<00:00, 2027.25it/s, Materializing param=encoder.layer.7.norm2.weight]
Loading weights:  34%|███▍      | 150/439 [00:00<00:00, 2039.52it/s, Materializing param=encoder.layer.8.attention.attention.key.bias]
Loading weights:  34%|███▍      | 150/439 [00:00<00:00, 2038.57it/s, Materializing param=encoder.layer.8.attention.attention.key.bias]
Loading weights:  34%|███▍      | 151/439 [00:00<00:00, 2050.63it/s, Materializing param=encoder.layer.8.attention.attention.key.weight]
Loading weights:  34%|███▍      | 151/439 [00:00<00:00, 2049.52it/s, Materializing param=encoder.layer.8.attention.attention.key.weight]
Loading weights:  35%|███▍      | 152/439 [00:00<00:00, 2061.61it/s, Materializing param=encoder.layer.8.attention.attention.query.bias]
Loading weights:  35%|███▍      | 152/439 [00:00<00:00, 2060.62it/s, Materializing param=encoder.layer.8.attention.attention.query.bias]
Loading weights:  35%|███▍      | 153/439 [00:00<00:00, 2072.72it/s, Materializing param=encoder.layer.8.attention.attention.query.weight]
Loading weights:  35%|███▍      | 153/439 [00:00<00:00, 2071.77it/s, Materializing param=encoder.layer.8.attention.attention.query.weight]
Loading weights:  35%|███▌      | 154/439 [00:00<00:00, 2083.87it/s, Materializing param=encoder.layer.8.attention.attention.value.bias]  
Loading weights:  35%|███▌      | 154/439 [00:00<00:00, 2082.89it/s, Materializing param=encoder.layer.8.attention.attention.value.bias]
Loading weights:  35%|███▌      | 155/439 [00:00<00:00, 2094.60it/s, Materializing param=encoder.layer.8.attention.attention.value.weight]
Loading weights:  35%|███▌      | 155/439 [00:00<00:00, 2093.64it/s, Materializing param=encoder.layer.8.attention.attention.value.weight]
Loading weights:  36%|███▌      | 156/439 [00:00<00:00, 2105.70it/s, Materializing param=encoder.layer.8.attention.output.dense.bias]     
Loading weights:  36%|███▌      | 156/439 [00:00<00:00, 2104.76it/s, Materializing param=encoder.layer.8.attention.output.dense.bias]
Loading weights:  36%|███▌      | 157/439 [00:00<00:00, 2116.79it/s, Materializing param=encoder.layer.8.attention.output.dense.weight]
Loading weights:  36%|███▌      | 157/439 [00:00<00:00, 2115.82it/s, Materializing param=encoder.layer.8.attention.output.dense.weight]
Loading weights:  36%|███▌      | 158/439 [00:00<00:00, 2127.80it/s, Materializing param=encoder.layer.8.layer_scale1.lambda1]         
Loading weights:  36%|███▌      | 158/439 [00:00<00:00, 2126.89it/s, Materializing param=encoder.layer.8.layer_scale1.lambda1]
Loading weights:  36%|███▌      | 159/439 [00:00<00:00, 2138.91it/s, Materializing param=encoder.layer.8.layer_scale2.lambda1]
Loading weights:  36%|███▌      | 159/439 [00:00<00:00, 2137.96it/s, Materializing param=encoder.layer.8.layer_scale2.lambda1]
Loading weights:  36%|███▋      | 160/439 [00:00<00:00, 2150.02it/s, Materializing param=encoder.layer.8.mlp.fc1.bias]        
Loading weights:  36%|███▋      | 160/439 [00:00<00:00, 2147.20it/s, Materializing param=encoder.layer.8.mlp.fc1.bias]
Loading weights:  37%|███▋      | 161/439 [00:00<00:00, 2156.82it/s, Materializing param=encoder.layer.8.mlp.fc1.weight]
Loading weights:  37%|███▋      | 161/439 [00:00<00:00, 2155.75it/s, Materializing param=encoder.layer.8.mlp.fc1.weight]
Loading weights:  37%|███▋      | 162/439 [00:00<00:00, 2167.55it/s, Materializing param=encoder.layer.8.mlp.fc2.bias]  
Loading weights:  37%|███▋      | 162/439 [00:00<00:00, 2166.39it/s, Materializing param=encoder.layer.8.mlp.fc2.bias]
Loading weights:  37%|███▋      | 163/439 [00:00<00:00, 2177.20it/s, Materializing param=encoder.layer.8.mlp.fc2.weight]
Loading weights:  37%|███▋      | 163/439 [00:00<00:00, 2176.04it/s, Materializing param=encoder.layer.8.mlp.fc2.weight]
Loading weights:  37%|███▋      | 164/439 [00:00<00:00, 2187.60it/s, Materializing param=encoder.layer.8.norm1.bias]    
Loading weights:  37%|███▋      | 164/439 [00:00<00:00, 2186.55it/s, Materializing param=encoder.layer.8.norm1.bias]
Loading weights:  38%|███▊      | 165/439 [00:00<00:00, 2197.51it/s, Materializing param=encoder.layer.8.norm1.weight]
Loading weights:  38%|███▊      | 165/439 [00:00<00:00, 2196.11it/s, Materializing param=encoder.layer.8.norm1.weight]
Loading weights:  38%|███▊      | 166/439 [00:00<00:00, 2207.45it/s, Materializing param=encoder.layer.8.norm2.bias]  
Loading weights:  38%|███▊      | 166/439 [00:00<00:00, 2206.39it/s, Materializing param=encoder.layer.8.norm2.bias]
Loading weights:  38%|███▊      | 167/439 [00:00<00:00, 2218.01it/s, Materializing param=encoder.layer.8.norm2.weight]
Loading weights:  38%|███▊      | 167/439 [00:00<00:00, 2217.09it/s, Materializing param=encoder.layer.8.norm2.weight]
Loading weights:  38%|███▊      | 168/439 [00:00<00:00, 2228.80it/s, Materializing param=encoder.layer.9.attention.attention.key.bias]
Loading weights:  38%|███▊      | 168/439 [00:00<00:00, 2227.80it/s, Materializing param=encoder.layer.9.attention.attention.key.bias]
Loading weights:  38%|███▊      | 169/439 [00:00<00:00, 2238.45it/s, Materializing param=encoder.layer.9.attention.attention.key.weight]
Loading weights:  38%|███▊      | 169/439 [00:00<00:00, 2237.23it/s, Materializing param=encoder.layer.9.attention.attention.key.weight]
Loading weights:  39%|███▊      | 170/439 [00:00<00:00, 2238.67it/s, Materializing param=encoder.layer.9.attention.attention.query.bias]
Loading weights:  39%|███▊      | 170/439 [00:00<00:00, 2237.64it/s, Materializing param=encoder.layer.9.attention.attention.query.bias]
Loading weights:  39%|███▉      | 171/439 [00:00<00:00, 2249.18it/s, Materializing param=encoder.layer.9.attention.attention.query.weight]
Loading weights:  39%|███▉      | 171/439 [00:00<00:00, 2248.23it/s, Materializing param=encoder.layer.9.attention.attention.query.weight]
Loading weights:  39%|███▉      | 172/439 [00:00<00:00, 2259.81it/s, Materializing param=encoder.layer.9.attention.attention.value.bias]  
Loading weights:  39%|███▉      | 172/439 [00:00<00:00, 2258.86it/s, Materializing param=encoder.layer.9.attention.attention.value.bias]
Loading weights:  39%|███▉      | 173/439 [00:00<00:00, 2270.44it/s, Materializing param=encoder.layer.9.attention.attention.value.weight]
Loading weights:  39%|███▉      | 173/439 [00:00<00:00, 2269.42it/s, Materializing param=encoder.layer.9.attention.attention.value.weight]
Loading weights:  40%|███▉      | 174/439 [00:00<00:00, 2280.99it/s, Materializing param=encoder.layer.9.attention.output.dense.bias]     
Loading weights:  40%|███▉      | 174/439 [00:00<00:00, 2280.05it/s, Materializing param=encoder.layer.9.attention.output.dense.bias]
Loading weights:  40%|███▉      | 175/439 [00:00<00:00, 2291.57it/s, Materializing param=encoder.layer.9.attention.output.dense.weight]
Loading weights:  40%|███▉      | 175/439 [00:00<00:00, 2290.55it/s, Materializing param=encoder.layer.9.attention.output.dense.weight]
Loading weights:  40%|████      | 176/439 [00:00<00:00, 2302.01it/s, Materializing param=encoder.layer.9.layer_scale1.lambda1]         
Loading weights:  40%|████      | 176/439 [00:00<00:00, 2300.99it/s, Materializing param=encoder.layer.9.layer_scale1.lambda1]
Loading weights:  40%|████      | 177/439 [00:00<00:00, 2312.43it/s, Materializing param=encoder.layer.9.layer_scale2.lambda1]
Loading weights:  40%|████      | 177/439 [00:00<00:00, 2311.40it/s, Materializing param=encoder.layer.9.layer_scale2.lambda1]
Loading weights:  41%|████      | 178/439 [00:00<00:00, 2322.95it/s, Materializing param=encoder.layer.9.mlp.fc1.bias]        
Loading weights:  41%|████      | 178/439 [00:00<00:00, 2321.98it/s, Materializing param=encoder.layer.9.mlp.fc1.bias]
Loading weights:  41%|████      | 179/439 [00:00<00:00, 2333.52it/s, Materializing param=encoder.layer.9.mlp.fc1.weight]
Loading weights:  41%|████      | 179/439 [00:00<00:00, 2332.55it/s, Materializing param=encoder.layer.9.mlp.fc1.weight]
Loading weights:  41%|████      | 180/439 [00:00<00:00, 2344.06it/s, Materializing param=encoder.layer.9.mlp.fc2.bias]  
Loading weights:  41%|████      | 180/439 [00:00<00:00, 2343.17it/s, Materializing param=encoder.layer.9.mlp.fc2.bias]
Loading weights:  41%|████      | 181/439 [00:00<00:00, 2354.75it/s, Materializing param=encoder.layer.9.mlp.fc2.weight]
Loading weights:  41%|████      | 181/439 [00:00<00:00, 2353.87it/s, Materializing param=encoder.layer.9.mlp.fc2.weight]
Loading weights:  41%|████▏     | 182/439 [00:00<00:00, 2365.45it/s, Materializing param=encoder.layer.9.norm1.bias]    
Loading weights:  41%|████▏     | 182/439 [00:00<00:00, 2364.56it/s, Materializing param=encoder.layer.9.norm1.bias]
Loading weights:  42%|████▏     | 183/439 [00:00<00:00, 2367.10it/s, Materializing param=encoder.layer.9.norm1.weight]
Loading weights:  42%|████▏     | 183/439 [00:00<00:00, 2365.75it/s, Materializing param=encoder.layer.9.norm1.weight]
Loading weights:  42%|████▏     | 184/439 [00:00<00:00, 2376.84it/s, Materializing param=encoder.layer.9.norm2.bias]  
Loading weights:  42%|████▏     | 184/439 [00:00<00:00, 2375.79it/s, Materializing param=encoder.layer.9.norm2.bias]
Loading weights:  42%|████▏     | 185/439 [00:00<00:00, 2387.01it/s, Materializing param=encoder.layer.9.norm2.weight]
Loading weights:  42%|████▏     | 185/439 [00:00<00:00, 2385.99it/s, Materializing param=encoder.layer.9.norm2.weight]
Loading weights:  42%|████▏     | 186/439 [00:00<00:00, 2397.35it/s, Materializing param=encoder.layer.10.attention.attention.key.bias]
Loading weights:  42%|████▏     | 186/439 [00:00<00:00, 2396.33it/s, Materializing param=encoder.layer.10.attention.attention.key.bias]
Loading weights:  43%|████▎     | 187/439 [00:00<00:00, 2407.50it/s, Materializing param=encoder.layer.10.attention.attention.key.weight]
Loading weights:  43%|████▎     | 187/439 [00:00<00:00, 2406.41it/s, Materializing param=encoder.layer.10.attention.attention.key.weight]
Loading weights:  43%|████▎     | 188/439 [00:00<00:00, 2417.52it/s, Materializing param=encoder.layer.10.attention.attention.query.bias]
Loading weights:  43%|████▎     | 188/439 [00:00<00:00, 2416.49it/s, Materializing param=encoder.layer.10.attention.attention.query.bias]
Loading weights:  43%|████▎     | 189/439 [00:00<00:00, 2427.16it/s, Materializing param=encoder.layer.10.attention.attention.query.weight]
Loading weights:  43%|████▎     | 189/439 [00:00<00:00, 2425.32it/s, Materializing param=encoder.layer.10.attention.attention.query.weight]
Loading weights:  43%|████▎     | 190/439 [00:00<00:00, 2433.28it/s, Materializing param=encoder.layer.10.attention.attention.value.bias]  
Loading weights:  43%|████▎     | 190/439 [00:00<00:00, 2431.76it/s, Materializing param=encoder.layer.10.attention.attention.value.bias]
Loading weights:  44%|████▎     | 191/439 [00:00<00:00, 2442.24it/s, Materializing param=encoder.layer.10.attention.attention.value.weight]
Loading weights:  44%|████▎     | 191/439 [00:00<00:00, 2441.02it/s, Materializing param=encoder.layer.10.attention.attention.value.weight]
Loading weights:  44%|████▎     | 192/439 [00:00<00:00, 2451.92it/s, Materializing param=encoder.layer.10.attention.output.dense.bias]     
Loading weights:  44%|████▎     | 192/439 [00:00<00:00, 2450.77it/s, Materializing param=encoder.layer.10.attention.output.dense.bias]
Loading weights:  44%|████▍     | 193/439 [00:00<00:00, 2461.64it/s, Materializing param=encoder.layer.10.attention.output.dense.weight]
Loading weights:  44%|████▍     | 193/439 [00:00<00:00, 2460.52it/s, Materializing param=encoder.layer.10.attention.output.dense.weight]
Loading weights:  44%|████▍     | 194/439 [00:00<00:00, 2471.46it/s, Materializing param=encoder.layer.10.layer_scale1.lambda1]         
Loading weights:  44%|████▍     | 194/439 [00:00<00:00, 2470.40it/s, Materializing param=encoder.layer.10.layer_scale1.lambda1]
Loading weights:  44%|████▍     | 195/439 [00:00<00:00, 2481.42it/s, Materializing param=encoder.layer.10.layer_scale2.lambda1]
Loading weights:  44%|████▍     | 195/439 [00:00<00:00, 2480.32it/s, Materializing param=encoder.layer.10.layer_scale2.lambda1]
Loading weights:  45%|████▍     | 196/439 [00:00<00:00, 2491.33it/s, Materializing param=encoder.layer.10.mlp.fc1.bias]        
Loading weights:  45%|████▍     | 196/439 [00:00<00:00, 2490.26it/s, Materializing param=encoder.layer.10.mlp.fc1.bias]
Loading weights:  45%|████▍     | 197/439 [00:00<00:00, 2501.31it/s, Materializing param=encoder.layer.10.mlp.fc1.weight]
Loading weights:  45%|████▍     | 197/439 [00:00<00:00, 2500.29it/s, Materializing param=encoder.layer.10.mlp.fc1.weight]
Loading weights:  45%|████▌     | 198/439 [00:00<00:00, 2511.35it/s, Materializing param=encoder.layer.10.mlp.fc2.bias]  
Loading weights:  45%|████▌     | 198/439 [00:00<00:00, 2510.33it/s, Materializing param=encoder.layer.10.mlp.fc2.bias]
Loading weights:  45%|████▌     | 199/439 [00:00<00:00, 2521.35it/s, Materializing param=encoder.layer.10.mlp.fc2.weight]
Loading weights:  45%|████▌     | 199/439 [00:00<00:00, 2520.27it/s, Materializing param=encoder.layer.10.mlp.fc2.weight]
Loading weights:  46%|████▌     | 200/439 [00:00<00:00, 2531.20it/s, Materializing param=encoder.layer.10.norm1.bias]    
Loading weights:  46%|████▌     | 200/439 [00:00<00:00, 2530.15it/s, Materializing param=encoder.layer.10.norm1.bias]
Loading weights:  46%|████▌     | 201/439 [00:00<00:00, 2541.15it/s, Materializing param=encoder.layer.10.norm1.weight]
Loading weights:  46%|████▌     | 201/439 [00:00<00:00, 2540.13it/s, Materializing param=encoder.layer.10.norm1.weight]
Loading weights:  46%|████▌     | 202/439 [00:00<00:00, 2551.22it/s, Materializing param=encoder.layer.10.norm2.bias]  
Loading weights:  46%|████▌     | 202/439 [00:00<00:00, 2550.25it/s, Materializing param=encoder.layer.10.norm2.bias]
Loading weights:  46%|████▌     | 203/439 [00:00<00:00, 2561.29it/s, Materializing param=encoder.layer.10.norm2.weight]
Loading weights:  46%|████▌     | 203/439 [00:00<00:00, 2560.29it/s, Materializing param=encoder.layer.10.norm2.weight]
Loading weights:  46%|████▋     | 204/439 [00:00<00:00, 2571.34it/s, Materializing param=encoder.layer.11.attention.attention.key.bias]
Loading weights:  46%|████▋     | 204/439 [00:00<00:00, 2570.21it/s, Materializing param=encoder.layer.11.attention.attention.key.bias]
Loading weights:  47%|████▋     | 205/439 [00:00<00:00, 2581.06it/s, Materializing param=encoder.layer.11.attention.attention.key.weight]
Loading weights:  47%|████▋     | 205/439 [00:00<00:00, 2579.92it/s, Materializing param=encoder.layer.11.attention.attention.key.weight]
Loading weights:  47%|████▋     | 206/439 [00:00<00:00, 2590.77it/s, Materializing param=encoder.layer.11.attention.attention.query.bias]
Loading weights:  47%|████▋     | 206/439 [00:00<00:00, 2589.67it/s, Materializing param=encoder.layer.11.attention.attention.query.bias]
Loading weights:  47%|████▋     | 207/439 [00:00<00:00, 2600.48it/s, Materializing param=encoder.layer.11.attention.attention.query.weight]
Loading weights:  47%|████▋     | 207/439 [00:00<00:00, 2599.36it/s, Materializing param=encoder.layer.11.attention.attention.query.weight]
Loading weights:  47%|████▋     | 208/439 [00:00<00:00, 2610.11it/s, Materializing param=encoder.layer.11.attention.attention.value.bias]  
Loading weights:  47%|████▋     | 208/439 [00:00<00:00, 2608.97it/s, Materializing param=encoder.layer.11.attention.attention.value.bias]
Loading weights:  48%|████▊     | 209/439 [00:00<00:00, 2619.77it/s, Materializing param=encoder.layer.11.attention.attention.value.weight]
Loading weights:  48%|████▊     | 209/439 [00:00<00:00, 2618.65it/s, Materializing param=encoder.layer.11.attention.attention.value.weight]
Loading weights:  48%|████▊     | 210/439 [00:00<00:00, 2629.47it/s, Materializing param=encoder.layer.11.attention.output.dense.bias]     
Loading weights:  48%|████▊     | 210/439 [00:00<00:00, 2628.39it/s, Materializing param=encoder.layer.11.attention.output.dense.bias]
Loading weights:  48%|████▊     | 211/439 [00:00<00:00, 2639.19it/s, Materializing param=encoder.layer.11.attention.output.dense.weight]
Loading weights:  48%|████▊     | 211/439 [00:00<00:00, 2638.09it/s, Materializing param=encoder.layer.11.attention.output.dense.weight]
Loading weights:  48%|████▊     | 212/439 [00:00<00:00, 2648.91it/s, Materializing param=encoder.layer.11.layer_scale1.lambda1]         
Loading weights:  48%|████▊     | 212/439 [00:00<00:00, 2647.82it/s, Materializing param=encoder.layer.11.layer_scale1.lambda1]
Loading weights:  49%|████▊     | 213/439 [00:00<00:00, 2658.65it/s, Materializing param=encoder.layer.11.layer_scale2.lambda1]
Loading weights:  49%|████▊     | 213/439 [00:00<00:00, 2657.55it/s, Materializing param=encoder.layer.11.layer_scale2.lambda1]
Loading weights:  49%|████▊     | 214/439 [00:00<00:00, 2668.40it/s, Materializing param=encoder.layer.11.mlp.fc1.bias]        
Loading weights:  49%|████▊     | 214/439 [00:00<00:00, 2667.37it/s, Materializing param=encoder.layer.11.mlp.fc1.bias]
Loading weights:  49%|████▉     | 215/439 [00:00<00:00, 2678.13it/s, Materializing param=encoder.layer.11.mlp.fc1.weight]
Loading weights:  49%|████▉     | 215/439 [00:00<00:00, 2677.06it/s, Materializing param=encoder.layer.11.mlp.fc1.weight]
Loading weights:  49%|████▉     | 216/439 [00:00<00:00, 2687.84it/s, Materializing param=encoder.layer.11.mlp.fc2.bias]  
Loading weights:  49%|████▉     | 216/439 [00:00<00:00, 2686.70it/s, Materializing param=encoder.layer.11.mlp.fc2.bias]
Loading weights:  49%|████▉     | 217/439 [00:00<00:00, 2697.40it/s, Materializing param=encoder.layer.11.mlp.fc2.weight]
Loading weights:  49%|████▉     | 217/439 [00:00<00:00, 2696.29it/s, Materializing param=encoder.layer.11.mlp.fc2.weight]
Loading weights:  50%|████▉     | 218/439 [00:00<00:00, 2707.00it/s, Materializing param=encoder.layer.11.norm1.bias]    
Loading weights:  50%|████▉     | 218/439 [00:00<00:00, 2705.93it/s, Materializing param=encoder.layer.11.norm1.bias]
Loading weights:  50%|████▉     | 219/439 [00:00<00:00, 2716.69it/s, Materializing param=encoder.layer.11.norm1.weight]
Loading weights:  50%|████▉     | 219/439 [00:00<00:00, 2715.60it/s, Materializing param=encoder.layer.11.norm1.weight]
Loading weights:  50%|█████     | 220/439 [00:00<00:00, 2726.35it/s, Materializing param=encoder.layer.11.norm2.bias]  
Loading weights:  50%|█████     | 220/439 [00:00<00:00, 2725.24it/s, Materializing param=encoder.layer.11.norm2.bias]
Loading weights:  50%|█████     | 221/439 [00:00<00:00, 2735.97it/s, Materializing param=encoder.layer.11.norm2.weight]
Loading weights:  50%|█████     | 221/439 [00:00<00:00, 2734.88it/s, Materializing param=encoder.layer.11.norm2.weight]
Loading weights:  51%|█████     | 222/439 [00:00<00:00, 2745.55it/s, Materializing param=encoder.layer.12.attention.attention.key.bias]
Loading weights:  51%|█████     | 222/439 [00:00<00:00, 2744.37it/s, Materializing param=encoder.layer.12.attention.attention.key.bias]
Loading weights:  51%|█████     | 223/439 [00:00<00:00, 2754.86it/s, Materializing param=encoder.layer.12.attention.attention.key.weight]
Loading weights:  51%|█████     | 223/439 [00:00<00:00, 2753.70it/s, Materializing param=encoder.layer.12.attention.attention.key.weight]
Loading weights:  51%|█████     | 224/439 [00:00<00:00, 2764.27it/s, Materializing param=encoder.layer.12.attention.attention.query.bias]
Loading weights:  51%|█████     | 224/439 [00:00<00:00, 2763.01it/s, Materializing param=encoder.layer.12.attention.attention.query.bias]
Loading weights:  51%|█████▏    | 225/439 [00:00<00:00, 2773.60it/s, Materializing param=encoder.layer.12.attention.attention.query.weight]
Loading weights:  51%|█████▏    | 225/439 [00:00<00:00, 2772.38it/s, Materializing param=encoder.layer.12.attention.attention.query.weight]
Loading weights:  51%|█████▏    | 226/439 [00:00<00:00, 2782.91it/s, Materializing param=encoder.layer.12.attention.attention.value.bias]  
Loading weights:  51%|█████▏    | 226/439 [00:00<00:00, 2781.78it/s, Materializing param=encoder.layer.12.attention.attention.value.bias]
Loading weights:  52%|█████▏    | 227/439 [00:00<00:00, 2792.30it/s, Materializing param=encoder.layer.12.attention.attention.value.weight]
Loading weights:  52%|█████▏    | 227/439 [00:00<00:00, 2791.14it/s, Materializing param=encoder.layer.12.attention.attention.value.weight]
Loading weights:  52%|█████▏    | 228/439 [00:00<00:00, 2801.71it/s, Materializing param=encoder.layer.12.attention.output.dense.bias]     
Loading weights:  52%|█████▏    | 228/439 [00:00<00:00, 2800.64it/s, Materializing param=encoder.layer.12.attention.output.dense.bias]
Loading weights:  52%|█████▏    | 229/439 [00:00<00:00, 2811.16it/s, Materializing param=encoder.layer.12.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 229/439 [00:00<00:00, 2809.95it/s, Materializing param=encoder.layer.12.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 230/439 [00:00<00:00, 2808.18it/s, Materializing param=encoder.layer.12.layer_scale1.lambda1]         
Loading weights:  52%|█████▏    | 230/439 [00:00<00:00, 2799.02it/s, Materializing param=encoder.layer.12.layer_scale1.lambda1]
Loading weights:  53%|█████▎    | 231/439 [00:00<00:00, 2805.14it/s, Materializing param=encoder.layer.12.layer_scale2.lambda1]
Loading weights:  53%|█████▎    | 231/439 [00:00<00:00, 2803.81it/s, Materializing param=encoder.layer.12.layer_scale2.lambda1]
Loading weights:  53%|█████▎    | 232/439 [00:00<00:00, 2813.90it/s, Materializing param=encoder.layer.12.mlp.fc1.bias]        
Loading weights:  53%|█████▎    | 232/439 [00:00<00:00, 2812.74it/s, Materializing param=encoder.layer.12.mlp.fc1.bias]
Loading weights:  53%|█████▎    | 233/439 [00:00<00:00, 2822.95it/s, Materializing param=encoder.layer.12.mlp.fc1.weight]
Loading weights:  53%|█████▎    | 233/439 [00:00<00:00, 2821.75it/s, Materializing param=encoder.layer.12.mlp.fc1.weight]
Loading weights:  53%|█████▎    | 234/439 [00:00<00:00, 2829.81it/s, Materializing param=encoder.layer.12.mlp.fc2.bias]  
Loading weights:  53%|█████▎    | 234/439 [00:00<00:00, 2827.59it/s, Materializing param=encoder.layer.12.mlp.fc2.bias]
Loading weights:  54%|█████▎    | 235/439 [00:00<00:00, 2835.77it/s, Materializing param=encoder.layer.12.mlp.fc2.weight]
Loading weights:  54%|█████▎    | 235/439 [00:00<00:00, 2834.19it/s, Materializing param=encoder.layer.12.mlp.fc2.weight]
Loading weights:  54%|█████▍    | 236/439 [00:00<00:00, 2843.86it/s, Materializing param=encoder.layer.12.norm1.bias]    
Loading weights:  54%|█████▍    | 236/439 [00:00<00:00, 2842.56it/s, Materializing param=encoder.layer.12.norm1.bias]
Loading weights:  54%|█████▍    | 237/439 [00:00<00:00, 2852.54it/s, Materializing param=encoder.layer.12.norm1.weight]
Loading weights:  54%|█████▍    | 237/439 [00:00<00:00, 2851.37it/s, Materializing param=encoder.layer.12.norm1.weight]
Loading weights:  54%|█████▍    | 238/439 [00:00<00:00, 2861.45it/s, Materializing param=encoder.layer.12.norm2.bias]  
Loading weights:  54%|█████▍    | 238/439 [00:00<00:00, 2860.17it/s, Materializing param=encoder.layer.12.norm2.bias]
Loading weights:  54%|█████▍    | 239/439 [00:00<00:00, 2870.29it/s, Materializing param=encoder.layer.12.norm2.weight]
Loading weights:  54%|█████▍    | 239/439 [00:00<00:00, 2869.09it/s, Materializing param=encoder.layer.12.norm2.weight]
Loading weights:  55%|█████▍    | 240/439 [00:00<00:00, 2879.26it/s, Materializing param=encoder.layer.13.attention.attention.key.bias]
Loading weights:  55%|█████▍    | 240/439 [00:00<00:00, 2878.01it/s, Materializing param=encoder.layer.13.attention.attention.key.bias]
Loading weights:  55%|█████▍    | 241/439 [00:00<00:00, 2888.00it/s, Materializing param=encoder.layer.13.attention.attention.key.weight]
Loading weights:  55%|█████▍    | 241/439 [00:00<00:00, 2886.82it/s, Materializing param=encoder.layer.13.attention.attention.key.weight]
Loading weights:  55%|█████▌    | 242/439 [00:00<00:00, 2896.82it/s, Materializing param=encoder.layer.13.attention.attention.query.bias]
Loading weights:  55%|█████▌    | 242/439 [00:00<00:00, 2895.60it/s, Materializing param=encoder.layer.13.attention.attention.query.bias]
Loading weights:  55%|█████▌    | 243/439 [00:00<00:00, 2905.62it/s, Materializing param=encoder.layer.13.attention.attention.query.weight]
Loading weights:  55%|█████▌    | 243/439 [00:00<00:00, 2904.41it/s, Materializing param=encoder.layer.13.attention.attention.query.weight]
Loading weights:  56%|█████▌    | 244/439 [00:00<00:00, 2914.49it/s, Materializing param=encoder.layer.13.attention.attention.value.bias]  
Loading weights:  56%|█████▌    | 244/439 [00:00<00:00, 2913.30it/s, Materializing param=encoder.layer.13.attention.attention.value.bias]
Loading weights:  56%|█████▌    | 245/439 [00:00<00:00, 2923.04it/s, Materializing param=encoder.layer.13.attention.attention.value.weight]
Loading weights:  56%|█████▌    | 245/439 [00:00<00:00, 2921.82it/s, Materializing param=encoder.layer.13.attention.attention.value.weight]
Loading weights:  56%|█████▌    | 246/439 [00:00<00:00, 2931.37it/s, Materializing param=encoder.layer.13.attention.output.dense.bias]     
Loading weights:  56%|█████▌    | 246/439 [00:00<00:00, 2930.15it/s, Materializing param=encoder.layer.13.attention.output.dense.bias]
Loading weights:  56%|█████▋    | 247/439 [00:00<00:00, 2940.21it/s, Materializing param=encoder.layer.13.attention.output.dense.weight]
Loading weights:  56%|█████▋    | 247/439 [00:00<00:00, 2939.01it/s, Materializing param=encoder.layer.13.attention.output.dense.weight]
Loading weights:  56%|█████▋    | 248/439 [00:00<00:00, 2949.01it/s, Materializing param=encoder.layer.13.layer_scale1.lambda1]         
Loading weights:  56%|█████▋    | 248/439 [00:00<00:00, 2947.82it/s, Materializing param=encoder.layer.13.layer_scale1.lambda1]
Loading weights:  57%|█████▋    | 249/439 [00:00<00:00, 2957.81it/s, Materializing param=encoder.layer.13.layer_scale2.lambda1]
Loading weights:  57%|█████▋    | 249/439 [00:00<00:00, 2956.55it/s, Materializing param=encoder.layer.13.layer_scale2.lambda1]
Loading weights:  57%|█████▋    | 250/439 [00:00<00:00, 2966.55it/s, Materializing param=encoder.layer.13.mlp.fc1.bias]        
Loading weights:  57%|█████▋    | 250/439 [00:00<00:00, 2965.32it/s, Materializing param=encoder.layer.13.mlp.fc1.bias]
Loading weights:  57%|█████▋    | 251/439 [00:00<00:00, 2975.28it/s, Materializing param=encoder.layer.13.mlp.fc1.weight]
Loading weights:  57%|█████▋    | 251/439 [00:00<00:00, 2974.08it/s, Materializing param=encoder.layer.13.mlp.fc1.weight]
Loading weights:  57%|█████▋    | 252/439 [00:00<00:00, 2984.09it/s, Materializing param=encoder.layer.13.mlp.fc2.bias]  
Loading weights:  57%|█████▋    | 252/439 [00:00<00:00, 2982.92it/s, Materializing param=encoder.layer.13.mlp.fc2.bias]
Loading weights:  58%|█████▊    | 253/439 [00:00<00:00, 2992.46it/s, Materializing param=encoder.layer.13.mlp.fc2.weight]
Loading weights:  58%|█████▊    | 253/439 [00:00<00:00, 2991.29it/s, Materializing param=encoder.layer.13.mlp.fc2.weight]
Loading weights:  58%|█████▊    | 254/439 [00:00<00:00, 3001.30it/s, Materializing param=encoder.layer.13.norm1.bias]    
Loading weights:  58%|█████▊    | 254/439 [00:00<00:00, 3000.17it/s, Materializing param=encoder.layer.13.norm1.bias]
Loading weights:  58%|█████▊    | 255/439 [00:00<00:00, 3010.13it/s, Materializing param=encoder.layer.13.norm1.weight]
Loading weights:  58%|█████▊    | 255/439 [00:00<00:00, 3008.96it/s, Materializing param=encoder.layer.13.norm1.weight]
Loading weights:  58%|█████▊    | 256/439 [00:00<00:00, 3018.98it/s, Materializing param=encoder.layer.13.norm2.bias]  
Loading weights:  58%|█████▊    | 256/439 [00:00<00:00, 3017.77it/s, Materializing param=encoder.layer.13.norm2.bias]
Loading weights:  59%|█████▊    | 257/439 [00:00<00:00, 3027.74it/s, Materializing param=encoder.layer.13.norm2.weight]
Loading weights:  59%|█████▊    | 257/439 [00:00<00:00, 3026.52it/s, Materializing param=encoder.layer.13.norm2.weight]
Loading weights:  59%|█████▉    | 258/439 [00:00<00:00, 3036.52it/s, Materializing param=encoder.layer.14.attention.attention.key.bias]
Loading weights:  59%|█████▉    | 258/439 [00:00<00:00, 3035.19it/s, Materializing param=encoder.layer.14.attention.attention.key.bias]
Loading weights:  59%|█████▉    | 259/439 [00:00<00:00, 3044.95it/s, Materializing param=encoder.layer.14.attention.attention.key.weight]
Loading weights:  59%|█████▉    | 259/439 [00:00<00:00, 3043.70it/s, Materializing param=encoder.layer.14.attention.attention.key.weight]
Loading weights:  59%|█████▉    | 260/439 [00:00<00:00, 3053.48it/s, Materializing param=encoder.layer.14.attention.attention.query.bias]
Loading weights:  59%|█████▉    | 260/439 [00:00<00:00, 3052.26it/s, Materializing param=encoder.layer.14.attention.attention.query.bias]
Loading weights:  59%|█████▉    | 261/439 [00:00<00:00, 3062.09it/s, Materializing param=encoder.layer.14.attention.attention.query.weight]
Loading weights:  59%|█████▉    | 261/439 [00:00<00:00, 3060.90it/s, Materializing param=encoder.layer.14.attention.attention.query.weight]
Loading weights:  60%|█████▉    | 262/439 [00:00<00:00, 3070.72it/s, Materializing param=encoder.layer.14.attention.attention.value.bias]  
Loading weights:  60%|█████▉    | 262/439 [00:00<00:00, 3069.50it/s, Materializing param=encoder.layer.14.attention.attention.value.bias]
Loading weights:  60%|█████▉    | 263/439 [00:00<00:00, 3079.27it/s, Materializing param=encoder.layer.14.attention.attention.value.weight]
Loading weights:  60%|█████▉    | 263/439 [00:00<00:00, 3078.04it/s, Materializing param=encoder.layer.14.attention.attention.value.weight]
Loading weights:  60%|██████    | 264/439 [00:00<00:00, 3087.83it/s, Materializing param=encoder.layer.14.attention.output.dense.bias]     
Loading weights:  60%|██████    | 264/439 [00:00<00:00, 3086.63it/s, Materializing param=encoder.layer.14.attention.output.dense.bias]
Loading weights:  60%|██████    | 265/439 [00:00<00:00, 3096.38it/s, Materializing param=encoder.layer.14.attention.output.dense.weight]
Loading weights:  60%|██████    | 265/439 [00:00<00:00, 3095.15it/s, Materializing param=encoder.layer.14.attention.output.dense.weight]
Loading weights:  61%|██████    | 266/439 [00:00<00:00, 3104.95it/s, Materializing param=encoder.layer.14.layer_scale1.lambda1]         
Loading weights:  61%|██████    | 266/439 [00:00<00:00, 3103.68it/s, Materializing param=encoder.layer.14.layer_scale1.lambda1]
Loading weights:  61%|██████    | 267/439 [00:00<00:00, 3113.46it/s, Materializing param=encoder.layer.14.layer_scale2.lambda1]
Loading weights:  61%|██████    | 267/439 [00:00<00:00, 3112.22it/s, Materializing param=encoder.layer.14.layer_scale2.lambda1]
Loading weights:  61%|██████    | 268/439 [00:00<00:00, 3122.02it/s, Materializing param=encoder.layer.14.mlp.fc1.bias]        
Loading weights:  61%|██████    | 268/439 [00:00<00:00, 3120.82it/s, Materializing param=encoder.layer.14.mlp.fc1.bias]
Loading weights:  61%|██████▏   | 269/439 [00:00<00:00, 3130.49it/s, Materializing param=encoder.layer.14.mlp.fc1.weight]
Loading weights:  61%|██████▏   | 269/439 [00:00<00:00, 3129.33it/s, Materializing param=encoder.layer.14.mlp.fc1.weight]
Loading weights:  62%|██████▏   | 270/439 [00:00<00:00, 3139.10it/s, Materializing param=encoder.layer.14.mlp.fc2.bias]  
Loading weights:  62%|██████▏   | 270/439 [00:00<00:00, 3137.87it/s, Materializing param=encoder.layer.14.mlp.fc2.bias]
Loading weights:  62%|██████▏   | 271/439 [00:00<00:00, 3147.59it/s, Materializing param=encoder.layer.14.mlp.fc2.weight]
Loading weights:  62%|██████▏   | 271/439 [00:00<00:00, 3146.38it/s, Materializing param=encoder.layer.14.mlp.fc2.weight]
Loading weights:  62%|██████▏   | 272/439 [00:00<00:00, 3156.12it/s, Materializing param=encoder.layer.14.norm1.bias]    
Loading weights:  62%|██████▏   | 272/439 [00:00<00:00, 3154.91it/s, Materializing param=encoder.layer.14.norm1.bias]
Loading weights:  62%|██████▏   | 273/439 [00:00<00:00, 3164.74it/s, Materializing param=encoder.layer.14.norm1.weight]
Loading weights:  62%|██████▏   | 273/439 [00:00<00:00, 3163.54it/s, Materializing param=encoder.layer.14.norm1.weight]
Loading weights:  62%|██████▏   | 274/439 [00:00<00:00, 3173.32it/s, Materializing param=encoder.layer.14.norm2.bias]  
Loading weights:  62%|██████▏   | 274/439 [00:00<00:00, 3171.74it/s, Materializing param=encoder.layer.14.norm2.bias]
Loading weights:  63%|██████▎   | 275/439 [00:00<00:00, 3164.74it/s, Materializing param=encoder.layer.14.norm2.weight]
Loading weights:  63%|██████▎   | 275/439 [00:00<00:00, 3160.99it/s, Materializing param=encoder.layer.14.norm2.weight]
Loading weights:  63%|██████▎   | 276/439 [00:00<00:00, 3168.88it/s, Materializing param=encoder.layer.15.attention.attention.key.bias]
Loading weights:  63%|██████▎   | 276/439 [00:00<00:00, 3167.31it/s, Materializing param=encoder.layer.15.attention.attention.key.bias]
Loading weights:  63%|██████▎   | 277/439 [00:00<00:00, 3176.43it/s, Materializing param=encoder.layer.15.attention.attention.key.weight]
Loading weights:  63%|██████▎   | 277/439 [00:00<00:00, 3175.12it/s, Materializing param=encoder.layer.15.attention.attention.key.weight]
Loading weights:  63%|██████▎   | 278/439 [00:00<00:00, 3184.46it/s, Materializing param=encoder.layer.15.attention.attention.query.bias]
Loading weights:  63%|██████▎   | 278/439 [00:00<00:00, 3183.15it/s, Materializing param=encoder.layer.15.attention.attention.query.bias]
Loading weights:  64%|██████▎   | 279/439 [00:00<00:00, 3192.44it/s, Materializing param=encoder.layer.15.attention.attention.query.weight]
Loading weights:  64%|██████▎   | 279/439 [00:00<00:00, 3191.13it/s, Materializing param=encoder.layer.15.attention.attention.query.weight]
Loading weights:  64%|██████▍   | 280/439 [00:00<00:00, 3200.52it/s, Materializing param=encoder.layer.15.attention.attention.value.bias]  
Loading weights:  64%|██████▍   | 280/439 [00:00<00:00, 3199.24it/s, Materializing param=encoder.layer.15.attention.attention.value.bias]
Loading weights:  64%|██████▍   | 281/439 [00:00<00:00, 3208.71it/s, Materializing param=encoder.layer.15.attention.attention.value.weight]
Loading weights:  64%|██████▍   | 281/439 [00:00<00:00, 3207.40it/s, Materializing param=encoder.layer.15.attention.attention.value.weight]
Loading weights:  64%|██████▍   | 282/439 [00:00<00:00, 3216.87it/s, Materializing param=encoder.layer.15.attention.output.dense.bias]     
Loading weights:  64%|██████▍   | 282/439 [00:00<00:00, 3215.59it/s, Materializing param=encoder.layer.15.attention.output.dense.bias]
Loading weights:  64%|██████▍   | 283/439 [00:00<00:00, 3225.04it/s, Materializing param=encoder.layer.15.attention.output.dense.weight]
Loading weights:  64%|██████▍   | 283/439 [00:00<00:00, 3223.83it/s, Materializing param=encoder.layer.15.attention.output.dense.weight]
Loading weights:  65%|██████▍   | 284/439 [00:00<00:00, 3233.19it/s, Materializing param=encoder.layer.15.layer_scale1.lambda1]         
Loading weights:  65%|██████▍   | 284/439 [00:00<00:00, 3231.98it/s, Materializing param=encoder.layer.15.layer_scale1.lambda1]
Loading weights:  65%|██████▍   | 285/439 [00:00<00:00, 3241.48it/s, Materializing param=encoder.layer.15.layer_scale2.lambda1]
Loading weights:  65%|██████▍   | 285/439 [00:00<00:00, 3240.30it/s, Materializing param=encoder.layer.15.layer_scale2.lambda1]
Loading weights:  65%|██████▌   | 286/439 [00:00<00:00, 3249.81it/s, Materializing param=encoder.layer.15.mlp.fc1.bias]        
Loading weights:  65%|██████▌   | 286/439 [00:00<00:00, 3248.64it/s, Materializing param=encoder.layer.15.mlp.fc1.bias]
Loading weights:  65%|██████▌   | 287/439 [00:00<00:00, 3258.07it/s, Materializing param=encoder.layer.15.mlp.fc1.weight]
Loading weights:  65%|██████▌   | 287/439 [00:00<00:00, 3256.89it/s, Materializing param=encoder.layer.15.mlp.fc1.weight]
Loading weights:  66%|██████▌   | 288/439 [00:00<00:00, 3266.34it/s, Materializing param=encoder.layer.15.mlp.fc2.bias]  
Loading weights:  66%|██████▌   | 288/439 [00:00<00:00, 3265.16it/s, Materializing param=encoder.layer.15.mlp.fc2.bias]
Loading weights:  66%|██████▌   | 289/439 [00:00<00:00, 3274.57it/s, Materializing param=encoder.layer.15.mlp.fc2.weight]
Loading weights:  66%|██████▌   | 289/439 [00:00<00:00, 3273.30it/s, Materializing param=encoder.layer.15.mlp.fc2.weight]
Loading weights:  66%|██████▌   | 290/439 [00:00<00:00, 3282.78it/s, Materializing param=encoder.layer.15.norm1.bias]    
Loading weights:  66%|██████▌   | 290/439 [00:00<00:00, 3281.55it/s, Materializing param=encoder.layer.15.norm1.bias]
Loading weights:  66%|██████▋   | 291/439 [00:00<00:00, 3291.04it/s, Materializing param=encoder.layer.15.norm1.weight]
Loading weights:  66%|██████▋   | 291/439 [00:00<00:00, 3289.85it/s, Materializing param=encoder.layer.15.norm1.weight]
Loading weights:  67%|██████▋   | 292/439 [00:00<00:00, 3299.33it/s, Materializing param=encoder.layer.15.norm2.bias]  
Loading weights:  67%|██████▋   | 292/439 [00:00<00:00, 3298.14it/s, Materializing param=encoder.layer.15.norm2.bias]
Loading weights:  67%|██████▋   | 293/439 [00:00<00:00, 3307.64it/s, Materializing param=encoder.layer.15.norm2.weight]
Loading weights:  67%|██████▋   | 293/439 [00:00<00:00, 3306.45it/s, Materializing param=encoder.layer.15.norm2.weight]
Loading weights:  67%|██████▋   | 294/439 [00:00<00:00, 3315.93it/s, Materializing param=encoder.layer.16.attention.attention.key.bias]
Loading weights:  67%|██████▋   | 294/439 [00:00<00:00, 3314.66it/s, Materializing param=encoder.layer.16.attention.attention.key.bias]
Loading weights:  67%|██████▋   | 295/439 [00:00<00:00, 3323.90it/s, Materializing param=encoder.layer.16.attention.attention.key.weight]
Loading weights:  67%|██████▋   | 295/439 [00:00<00:00, 3322.64it/s, Materializing param=encoder.layer.16.attention.attention.key.weight]
Loading weights:  67%|██████▋   | 296/439 [00:00<00:00, 3331.88it/s, Materializing param=encoder.layer.16.attention.attention.query.bias]
Loading weights:  67%|██████▋   | 296/439 [00:00<00:00, 3330.64it/s, Materializing param=encoder.layer.16.attention.attention.query.bias]
Loading weights:  68%|██████▊   | 297/439 [00:00<00:00, 3339.90it/s, Materializing param=encoder.layer.16.attention.attention.query.weight]
Loading weights:  68%|██████▊   | 297/439 [00:00<00:00, 3338.58it/s, Materializing param=encoder.layer.16.attention.attention.query.weight]
Loading weights:  68%|██████▊   | 298/439 [00:00<00:00, 3347.83it/s, Materializing param=encoder.layer.16.attention.attention.value.bias]  
Loading weights:  68%|██████▊   | 298/439 [00:00<00:00, 3346.58it/s, Materializing param=encoder.layer.16.attention.attention.value.bias]
Loading weights:  68%|██████▊   | 299/439 [00:00<00:00, 3355.78it/s, Materializing param=encoder.layer.16.attention.attention.value.weight]
Loading weights:  68%|██████▊   | 299/439 [00:00<00:00, 3354.50it/s, Materializing param=encoder.layer.16.attention.attention.value.weight]
Loading weights:  68%|██████▊   | 300/439 [00:00<00:00, 3363.72it/s, Materializing param=encoder.layer.16.attention.output.dense.bias]     
Loading weights:  68%|██████▊   | 300/439 [00:00<00:00, 3362.41it/s, Materializing param=encoder.layer.16.attention.output.dense.bias]
Loading weights:  69%|██████▊   | 301/439 [00:00<00:00, 3371.65it/s, Materializing param=encoder.layer.16.attention.output.dense.weight]
Loading weights:  69%|██████▊   | 301/439 [00:00<00:00, 3370.40it/s, Materializing param=encoder.layer.16.attention.output.dense.weight]
Loading weights:  69%|██████▉   | 302/439 [00:00<00:00, 3379.67it/s, Materializing param=encoder.layer.16.layer_scale1.lambda1]         
Loading weights:  69%|██████▉   | 302/439 [00:00<00:00, 3378.45it/s, Materializing param=encoder.layer.16.layer_scale1.lambda1]
Loading weights:  69%|██████▉   | 303/439 [00:00<00:00, 3387.79it/s, Materializing param=encoder.layer.16.layer_scale2.lambda1]
Loading weights:  69%|██████▉   | 303/439 [00:00<00:00, 3386.50it/s, Materializing param=encoder.layer.16.layer_scale2.lambda1]
Loading weights:  69%|██████▉   | 304/439 [00:00<00:00, 3395.82it/s, Materializing param=encoder.layer.16.mlp.fc1.bias]        
Loading weights:  69%|██████▉   | 304/439 [00:00<00:00, 3394.61it/s, Materializing param=encoder.layer.16.mlp.fc1.bias]
Loading weights:  69%|██████▉   | 305/439 [00:00<00:00, 3403.87it/s, Materializing param=encoder.layer.16.mlp.fc1.weight]
Loading weights:  69%|██████▉   | 305/439 [00:00<00:00, 3402.69it/s, Materializing param=encoder.layer.16.mlp.fc1.weight]
Loading weights:  70%|██████▉   | 306/439 [00:00<00:00, 3411.99it/s, Materializing param=encoder.layer.16.mlp.fc2.bias]  
Loading weights:  70%|██████▉   | 306/439 [00:00<00:00, 3410.73it/s, Materializing param=encoder.layer.16.mlp.fc2.bias]
Loading weights:  70%|██████▉   | 307/439 [00:00<00:00, 3419.93it/s, Materializing param=encoder.layer.16.mlp.fc2.weight]
Loading weights:  70%|██████▉   | 307/439 [00:00<00:00, 3418.64it/s, Materializing param=encoder.layer.16.mlp.fc2.weight]
Loading weights:  70%|███████   | 308/439 [00:00<00:00, 3427.86it/s, Materializing param=encoder.layer.16.norm1.bias]    
Loading weights:  70%|███████   | 308/439 [00:00<00:00, 3426.68it/s, Materializing param=encoder.layer.16.norm1.bias]
Loading weights:  70%|███████   | 309/439 [00:00<00:00, 3435.98it/s, Materializing param=encoder.layer.16.norm1.weight]
Loading weights:  70%|███████   | 309/439 [00:00<00:00, 3434.75it/s, Materializing param=encoder.layer.16.norm1.weight]
Loading weights:  71%|███████   | 310/439 [00:00<00:00, 3444.06it/s, Materializing param=encoder.layer.16.norm2.bias]  
Loading weights:  71%|███████   | 310/439 [00:00<00:00, 3442.84it/s, Materializing param=encoder.layer.16.norm2.bias]
Loading weights:  71%|███████   | 311/439 [00:00<00:00, 3452.11it/s, Materializing param=encoder.layer.16.norm2.weight]
Loading weights:  71%|███████   | 311/439 [00:00<00:00, 3450.92it/s, Materializing param=encoder.layer.16.norm2.weight]
Loading weights:  71%|███████   | 312/439 [00:00<00:00, 3460.25it/s, Materializing param=encoder.layer.17.attention.attention.key.bias]
Loading weights:  71%|███████   | 312/439 [00:00<00:00, 3459.02it/s, Materializing param=encoder.layer.17.attention.attention.key.bias]
Loading weights:  71%|███████▏  | 313/439 [00:00<00:00, 3468.15it/s, Materializing param=encoder.layer.17.attention.attention.key.weight]
Loading weights:  71%|███████▏  | 313/439 [00:00<00:00, 3466.96it/s, Materializing param=encoder.layer.17.attention.attention.key.weight]
Loading weights:  72%|███████▏  | 314/439 [00:00<00:00, 3476.18it/s, Materializing param=encoder.layer.17.attention.attention.query.bias]
Loading weights:  72%|███████▏  | 314/439 [00:00<00:00, 3475.03it/s, Materializing param=encoder.layer.17.attention.attention.query.bias]
Loading weights:  72%|███████▏  | 315/439 [00:00<00:00, 3484.17it/s, Materializing param=encoder.layer.17.attention.attention.query.weight]
Loading weights:  72%|███████▏  | 315/439 [00:00<00:00, 3482.78it/s, Materializing param=encoder.layer.17.attention.attention.query.weight]
Loading weights:  72%|███████▏  | 316/439 [00:00<00:00, 3491.83it/s, Materializing param=encoder.layer.17.attention.attention.value.bias]  
Loading weights:  72%|███████▏  | 316/439 [00:00<00:00, 3490.56it/s, Materializing param=encoder.layer.17.attention.attention.value.bias]
Loading weights:  72%|███████▏  | 317/439 [00:00<00:00, 3499.63it/s, Materializing param=encoder.layer.17.attention.attention.value.weight]
Loading weights:  72%|███████▏  | 317/439 [00:00<00:00, 3498.40it/s, Materializing param=encoder.layer.17.attention.attention.value.weight]
Loading weights:  72%|███████▏  | 318/439 [00:00<00:00, 3507.51it/s, Materializing param=encoder.layer.17.attention.output.dense.bias]     
Loading weights:  72%|███████▏  | 318/439 [00:00<00:00, 3506.33it/s, Materializing param=encoder.layer.17.attention.output.dense.bias]
Loading weights:  73%|███████▎  | 319/439 [00:00<00:00, 3515.35it/s, Materializing param=encoder.layer.17.attention.output.dense.weight]
Loading weights:  73%|███████▎  | 319/439 [00:00<00:00, 3514.11it/s, Materializing param=encoder.layer.17.attention.output.dense.weight]
Loading weights:  73%|███████▎  | 320/439 [00:00<00:00, 3523.14it/s, Materializing param=encoder.layer.17.layer_scale1.lambda1]         
Loading weights:  73%|███████▎  | 320/439 [00:00<00:00, 3521.87it/s, Materializing param=encoder.layer.17.layer_scale1.lambda1]
Loading weights:  73%|███████▎  | 321/439 [00:00<00:00, 3530.54it/s, Materializing param=encoder.layer.17.layer_scale2.lambda1]
Loading weights:  73%|███████▎  | 321/439 [00:00<00:00, 3529.23it/s, Materializing param=encoder.layer.17.layer_scale2.lambda1]
Loading weights:  73%|███████▎  | 322/439 [00:00<00:00, 3538.27it/s, Materializing param=encoder.layer.17.mlp.fc1.bias]        
Loading weights:  73%|███████▎  | 322/439 [00:00<00:00, 3537.03it/s, Materializing param=encoder.layer.17.mlp.fc1.bias]
Loading weights:  74%|███████▎  | 323/439 [00:00<00:00, 3541.71it/s, Materializing param=encoder.layer.17.mlp.fc1.weight]
Loading weights:  74%|███████▎  | 323/439 [00:00<00:00, 3538.33it/s, Materializing param=encoder.layer.17.mlp.fc1.weight]
Loading weights:  74%|███████▍  | 324/439 [00:00<00:00, 3545.60it/s, Materializing param=encoder.layer.17.mlp.fc2.bias]  
Loading weights:  74%|███████▍  | 324/439 [00:00<00:00, 3544.05it/s, Materializing param=encoder.layer.17.mlp.fc2.bias]
Loading weights:  74%|███████▍  | 325/439 [00:00<00:00, 3552.50it/s, Materializing param=encoder.layer.17.mlp.fc2.weight]
Loading weights:  74%|███████▍  | 325/439 [00:00<00:00, 3551.06it/s, Materializing param=encoder.layer.17.mlp.fc2.weight]
Loading weights:  74%|███████▍  | 326/439 [00:00<00:00, 3558.26it/s, Materializing param=encoder.layer.17.norm1.bias]    
Loading weights:  74%|███████▍  | 326/439 [00:00<00:00, 3556.35it/s, Materializing param=encoder.layer.17.norm1.bias]
Loading weights:  74%|███████▍  | 327/439 [00:00<00:00, 3564.69it/s, Materializing param=encoder.layer.17.norm1.weight]
Loading weights:  74%|███████▍  | 327/439 [00:00<00:00, 3563.33it/s, Materializing param=encoder.layer.17.norm1.weight]
Loading weights:  75%|███████▍  | 328/439 [00:00<00:00, 3572.06it/s, Materializing param=encoder.layer.17.norm2.bias]  
Loading weights:  75%|███████▍  | 328/439 [00:00<00:00, 3570.66it/s, Materializing param=encoder.layer.17.norm2.bias]
Loading weights:  75%|███████▍  | 329/439 [00:00<00:00, 3579.40it/s, Materializing param=encoder.layer.17.norm2.weight]
Loading weights:  75%|███████▍  | 329/439 [00:00<00:00, 3577.61it/s, Materializing param=encoder.layer.17.norm2.weight]
Loading weights:  75%|███████▌  | 330/439 [00:00<00:00, 3584.35it/s, Materializing param=encoder.layer.18.attention.attention.key.bias]
Loading weights:  75%|███████▌  | 330/439 [00:00<00:00, 3581.59it/s, Materializing param=encoder.layer.18.attention.attention.key.bias]
Loading weights:  75%|███████▌  | 331/439 [00:00<00:00, 3586.72it/s, Materializing param=encoder.layer.18.attention.attention.key.weight]
Loading weights:  75%|███████▌  | 331/439 [00:00<00:00, 3584.70it/s, Materializing param=encoder.layer.18.attention.attention.key.weight]
Loading weights:  76%|███████▌  | 332/439 [00:00<00:00, 3592.50it/s, Materializing param=encoder.layer.18.attention.attention.query.bias]
Loading weights:  76%|███████▌  | 332/439 [00:00<00:00, 3590.99it/s, Materializing param=encoder.layer.18.attention.attention.query.bias]
Loading weights:  76%|███████▌  | 333/439 [00:00<00:00, 3599.23it/s, Materializing param=encoder.layer.18.attention.attention.query.weight]
Loading weights:  76%|███████▌  | 333/439 [00:00<00:00, 3597.75it/s, Materializing param=encoder.layer.18.attention.attention.query.weight]
Loading weights:  76%|███████▌  | 334/439 [00:00<00:00, 3605.78it/s, Materializing param=encoder.layer.18.attention.attention.value.bias]  
Loading weights:  76%|███████▌  | 334/439 [00:00<00:00, 3604.20it/s, Materializing param=encoder.layer.18.attention.attention.value.bias]
Loading weights:  76%|███████▋  | 335/439 [00:00<00:00, 3612.64it/s, Materializing param=encoder.layer.18.attention.attention.value.weight]
Loading weights:  76%|███████▋  | 335/439 [00:00<00:00, 3611.24it/s, Materializing param=encoder.layer.18.attention.attention.value.weight]
Loading weights:  77%|███████▋  | 336/439 [00:00<00:00, 3619.72it/s, Materializing param=encoder.layer.18.attention.output.dense.bias]     
Loading weights:  77%|███████▋  | 336/439 [00:00<00:00, 3618.28it/s, Materializing param=encoder.layer.18.attention.output.dense.bias]
Loading weights:  77%|███████▋  | 337/439 [00:00<00:00, 3626.74it/s, Materializing param=encoder.layer.18.attention.output.dense.weight]
Loading weights:  77%|���██████▋  | 337/439 [00:00<00:00, 3625.38it/s, Materializing param=encoder.layer.18.attention.output.dense.weight]
Loading weights:  77%|███████▋  | 338/439 [00:00<00:00, 3633.91it/s, Materializing param=encoder.layer.18.layer_scale1.lambda1]         
Loading weights:  77%|███████▋  | 338/439 [00:00<00:00, 3632.54it/s, Materializing param=encoder.layer.18.layer_scale1.lambda1]
Loading weights:  77%|███████▋  | 339/439 [00:00<00:00, 3641.21it/s, Materializing param=encoder.layer.18.layer_scale2.lambda1]
Loading weights:  77%|███████▋  | 339/439 [00:00<00:00, 3639.85it/s, Materializing param=encoder.layer.18.layer_scale2.lambda1]
Loading weights:  77%|███████▋  | 340/439 [00:00<00:00, 3648.54it/s, Materializing param=encoder.layer.18.mlp.fc1.bias]        
Loading weights:  77%|███████▋  | 340/439 [00:00<00:00, 3646.47it/s, Materializing param=encoder.layer.18.mlp.fc1.bias]
Loading weights:  78%|███████▊  | 341/439 [00:00<00:00, 3653.15it/s, Materializing param=encoder.layer.18.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 341/439 [00:00<00:00, 3651.32it/s, Materializing param=encoder.layer.18.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 342/439 [00:00<00:00, 3657.95it/s, Materializing param=encoder.layer.18.mlp.fc2.bias]  
Loading weights:  78%|███████▊  | 342/439 [00:00<00:00, 3656.39it/s, Materializing param=encoder.layer.18.mlp.fc2.bias]
Loading weights:  78%|███████▊  | 343/439 [00:00<00:00, 3664.70it/s, Materializing param=encoder.layer.18.mlp.fc2.weight]
Loading weights:  78%|███████▊  | 343/439 [00:00<00:00, 3663.28it/s, Materializing param=encoder.layer.18.mlp.fc2.weight]
Loading weights:  78%|███████▊  | 344/439 [00:00<00:00, 3671.72it/s, Materializing param=encoder.layer.18.norm1.bias]    
Loading weights:  78%|███████▊  | 344/439 [00:00<00:00, 3670.36it/s, Materializing param=encoder.layer.18.norm1.bias]
Loading weights:  79%|███████▊  | 345/439 [00:00<00:00, 3678.83it/s, Materializing param=encoder.layer.18.norm1.weight]
Loading weights:  79%|███████▊  | 345/439 [00:00<00:00, 3677.49it/s, Materializing param=encoder.layer.18.norm1.weight]
Loading weights:  79%|███████▉  | 346/439 [00:00<00:00, 3686.15it/s, Materializing param=encoder.layer.18.norm2.bias]  
Loading weights:  79%|███████▉  | 346/439 [00:00<00:00, 3684.82it/s, Materializing param=encoder.layer.18.norm2.bias]
Loading weights:  79%|███████▉  | 347/439 [00:00<00:00, 3693.30it/s, Materializing param=encoder.layer.18.norm2.weight]
Loading weights:  79%|███████▉  | 347/439 [00:00<00:00, 3692.04it/s, Materializing param=encoder.layer.18.norm2.weight]
Loading weights:  79%|███████▉  | 348/439 [00:00<00:00, 3700.25it/s, Materializing param=encoder.layer.19.attention.attention.key.bias]
Loading weights:  79%|███████▉  | 348/439 [00:00<00:00, 3698.76it/s, Materializing param=encoder.layer.19.attention.attention.key.bias]
Loading weights:  79%|███████▉  | 349/439 [00:00<00:00, 3707.09it/s, Materializing param=encoder.layer.19.attention.attention.key.weight]
Loading weights:  79%|███████▉  | 349/439 [00:00<00:00, 3705.64it/s, Materializing param=encoder.layer.19.attention.attention.key.weight]
Loading weights:  80%|███████▉  | 350/439 [00:00<00:00, 3714.16it/s, Materializing param=encoder.layer.19.attention.attention.query.bias]
Loading weights:  80%|███████▉  | 350/439 [00:00<00:00, 3712.87it/s, Materializing param=encoder.layer.19.attention.attention.query.bias]
Loading weights:  80%|███████▉  | 351/439 [00:00<00:00, 3721.34it/s, Materializing param=encoder.layer.19.attention.attention.query.weight]
Loading weights:  80%|███████▉  | 351/439 [00:00<00:00, 3720.12it/s, Materializing param=encoder.layer.19.attention.attention.query.weight]
Loading weights:  80%|████████  | 352/439 [00:00<00:00, 3728.66it/s, Materializing param=encoder.layer.19.attention.attention.value.bias]  
Loading weights:  80%|████████  | 352/439 [00:00<00:00, 3727.32it/s, Materializing param=encoder.layer.19.attention.attention.value.bias]
Loading weights:  80%|████████  | 353/439 [00:00<00:00, 3735.69it/s, Materializing param=encoder.layer.19.attention.attention.value.weight]
Loading weights:  80%|████████  | 353/439 [00:00<00:00, 3734.27it/s, Materializing param=encoder.layer.19.attention.attention.value.weight]
Loading weights:  81%|████████  | 354/439 [00:00<00:00, 3742.71it/s, Materializing param=encoder.layer.19.attention.output.dense.bias]     
Loading weights:  81%|████████  | 354/439 [00:00<00:00, 3741.33it/s, Materializing param=encoder.layer.19.attention.output.dense.bias]
Loading weights:  81%|████████  | 355/439 [00:00<00:00, 3749.83it/s, Materializing param=encoder.layer.19.attention.output.dense.weight]
Loading weights:  81%|████████  | 355/439 [00:00<00:00, 3748.53it/s, Materializing param=encoder.layer.19.attention.output.dense.weight]
Loading weights:  81%|████████  | 356/439 [00:00<00:00, 3757.02it/s, Materializing param=encoder.layer.19.layer_scale1.lambda1]         
Loading weights:  81%|████████  | 356/439 [00:00<00:00, 3755.71it/s, Materializing param=encoder.layer.19.layer_scale1.lambda1]
Loading weights:  81%|████████▏ | 357/439 [00:00<00:00, 3764.23it/s, Materializing param=encoder.layer.19.layer_scale2.lambda1]
Loading weights:  81%|████████▏ | 357/439 [00:00<00:00, 3762.89it/s, Materializing param=encoder.layer.19.layer_scale2.lambda1]
Loading weights:  82%|████████▏ | 358/439 [00:00<00:00, 3771.36it/s, Materializing param=encoder.layer.19.mlp.fc1.bias]        
Loading weights:  82%|████████▏ | 358/439 [00:00<00:00, 3770.09it/s, Materializing param=encoder.layer.19.mlp.fc1.bias]
Loading weights:  82%|████████▏ | 359/439 [00:00<00:00, 3778.63it/s, Materializing param=encoder.layer.19.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 359/439 [00:00<00:00, 3777.33it/s, Materializing param=encoder.layer.19.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 360/439 [00:00<00:00, 3785.81it/s, Materializing param=encoder.layer.19.mlp.fc2.bias]  
Loading weights:  82%|████████▏ | 360/439 [00:00<00:00, 3784.58it/s, Materializing param=encoder.layer.19.mlp.fc2.bias]
Loading weights:  82%|████████▏ | 361/439 [00:00<00:00, 3793.06it/s, Materializing param=encoder.layer.19.mlp.fc2.weight]
Loading weights:  82%|████████▏ | 361/439 [00:00<00:00, 3791.75it/s, Materializing param=encoder.layer.19.mlp.fc2.weight]
Loading weights:  82%|████████▏ | 362/439 [00:00<00:00, 3800.05it/s, Materializing param=encoder.layer.19.norm1.bias]    
Loading weights:  82%|████████▏ | 362/439 [00:00<00:00, 3798.90it/s, Materializing param=encoder.layer.19.norm1.bias]
Loading weights:  83%|████████▎ | 363/439 [00:00<00:00, 3807.47it/s, Materializing param=encoder.layer.19.norm1.weight]
Loading weights:  83%|████████▎ | 363/439 [00:00<00:00, 3806.27it/s, Materializing param=encoder.layer.19.norm1.weight]
Loading weights:  83%|████████▎ | 364/439 [00:00<00:00, 3814.92it/s, Materializing param=encoder.layer.19.norm2.bias]  
Loading weights:  83%|████████▎ | 364/439 [00:00<00:00, 3813.59it/s, Materializing param=encoder.layer.19.norm2.bias]
Loading weights:  83%|████████▎ | 365/439 [00:00<00:00, 3822.24it/s, Materializing param=encoder.layer.19.norm2.weight]
Loading weights:  83%|████████▎ | 365/439 [00:00<00:00, 3821.00it/s, Materializing param=encoder.layer.19.norm2.weight]
Loading weights:  83%|████████▎ | 366/439 [00:00<00:00, 3829.66it/s, Materializing param=encoder.layer.20.attention.attention.key.bias]
Loading weights:  83%|████████▎ | 366/439 [00:00<00:00, 3828.29it/s, Materializing param=encoder.layer.20.attention.attention.key.bias]
Loading weights:  84%|████████▎ | 367/439 [00:00<00:00, 3836.43it/s, Materializing param=encoder.layer.20.attention.attention.key.weight]
Loading weights:  84%|████████▎ | 367/439 [00:00<00:00, 3835.03it/s, Materializing param=encoder.layer.20.attention.attention.key.weight]
Loading weights:  84%|████████▍ | 368/439 [00:00<00:00, 3843.46it/s, Materializing param=encoder.layer.20.attention.attention.query.bias]
Loading weights:  84%|████████▍ | 368/439 [00:00<00:00, 3842.10it/s, Materializing param=encoder.layer.20.attention.attention.query.bias]
Loading weights:  84%|████████▍ | 369/439 [00:00<00:00, 3850.45it/s, Materializing param=encoder.layer.20.attention.attention.query.weight]
Loading weights:  84%|████████▍ | 369/439 [00:00<00:00, 3849.05it/s, Materializing param=encoder.layer.20.attention.attention.query.weight]
Loading weights:  84%|████████▍ | 370/439 [00:00<00:00, 3857.43it/s, Materializing param=encoder.layer.20.attention.attention.value.bias]  
Loading weights:  84%|████████▍ | 370/439 [00:00<00:00, 3856.10it/s, Materializing param=encoder.layer.20.attention.attention.value.bias]
Loading weights:  85%|████████▍ | 371/439 [00:00<00:00, 3864.07it/s, Materializing param=encoder.layer.20.attention.attention.value.weight]
Loading weights:  85%|████████▍ | 371/439 [00:00<00:00, 3862.73it/s, Materializing param=encoder.layer.20.attention.attention.value.weight]
Loading weights:  85%|████████▍ | 372/439 [00:00<00:00, 3871.05it/s, Materializing param=encoder.layer.20.attention.output.dense.bias]     
Loading weights:  85%|████████▍ | 372/439 [00:00<00:00, 3869.68it/s, Materializing param=encoder.layer.20.attention.output.dense.bias]
Loading weights:  85%|████████▍ | 373/439 [00:00<00:00, 3877.94it/s, Materializing param=encoder.layer.20.attention.output.dense.weight]
Loading weights:  85%|████████▍ | 373/439 [00:00<00:00, 3876.54it/s, Materializing param=encoder.layer.20.attention.output.dense.weight]
Loading weights:  85%|████████▌ | 374/439 [00:00<00:00, 3884.84it/s, Materializing param=encoder.layer.20.layer_scale1.lambda1]         
Loading weights:  85%|████████▌ | 374/439 [00:00<00:00, 3883.46it/s, Materializing param=encoder.layer.20.layer_scale1.lambda1]
Loading weights:  85%|████████▌ | 375/439 [00:00<00:00, 3891.62it/s, Materializing param=encoder.layer.20.layer_scale2.lambda1]
Loading weights:  85%|████████▌ | 375/439 [00:00<00:00, 3890.25it/s, Materializing param=encoder.layer.20.layer_scale2.lambda1]
Loading weights:  86%|████████▌ | 376/439 [00:00<00:00, 3898.40it/s, Materializing param=encoder.layer.20.mlp.fc1.bias]        
Loading weights:  86%|████████▌ | 376/439 [00:00<00:00, 3897.06it/s, Materializing param=encoder.layer.20.mlp.fc1.bias]
Loading weights:  86%|████████▌ | 377/439 [00:00<00:00, 3905.32it/s, Materializing param=encoder.layer.20.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 377/439 [00:00<00:00, 3903.99it/s, Materializing param=encoder.layer.20.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 378/439 [00:00<00:00, 3912.28it/s, Materializing param=encoder.layer.20.mlp.fc2.bias]  
Loading weights:  86%|████████▌ | 378/439 [00:00<00:00, 3910.90it/s, Materializing param=encoder.layer.20.mlp.fc2.bias]
Loading weights:  86%|████████▋ | 379/439 [00:00<00:00, 3919.22it/s, Materializing param=encoder.layer.20.mlp.fc2.weight]
Loading weights:  86%|████████▋ | 379/439 [00:00<00:00, 3917.88it/s, Materializing param=encoder.layer.20.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 380/439 [00:00<00:00, 3926.07it/s, Materializing param=encoder.layer.20.norm1.bias]    
Loading weights:  87%|████████▋ | 380/439 [00:00<00:00, 3924.78it/s, Materializing param=encoder.layer.20.norm1.bias]
Loading weights:  87%|████████▋ | 381/439 [00:00<00:00, 3931.89it/s, Materializing param=encoder.layer.20.norm1.weight]
Loading weights:  87%|████████▋ | 381/439 [00:00<00:00, 3928.97it/s, Materializing param=encoder.layer.20.norm1.weight]
Loading weights:  87%|████████▋ | 382/439 [00:00<00:00, 3934.90it/s, Materializing param=encoder.layer.20.norm2.bias]  
Loading weights:  87%|████████▋ | 382/439 [00:00<00:00, 3933.36it/s, Materializing param=encoder.layer.20.norm2.bias]
Loading weights:  87%|████████▋ | 383/439 [00:00<00:00, 3941.43it/s, Materializing param=encoder.layer.20.norm2.weight]
Loading weights:  87%|████████▋ | 383/439 [00:00<00:00, 3940.13it/s, Materializing param=encoder.layer.20.norm2.weight]
Loading weights:  87%|████████▋ | 384/439 [00:00<00:00, 3948.43it/s, Materializing param=encoder.layer.21.attention.attention.key.bias]
Loading weights:  87%|████████▋ | 384/439 [00:00<00:00, 3947.10it/s, Materializing param=encoder.layer.21.attention.attention.key.bias]
Loading weights:  88%|████████▊ | 385/439 [00:00<00:00, 3955.18it/s, Materializing param=encoder.layer.21.attention.attention.key.weight]
Loading weights:  88%|████████▊ | 385/439 [00:00<00:00, 3953.31it/s, Materializing param=encoder.layer.21.attention.attention.key.weight]
Loading weights:  88%|████████▊ | 386/439 [00:00<00:00, 3961.42it/s, Materializing param=encoder.layer.21.attention.attention.query.bias]
Loading weights:  88%|████████▊ | 386/439 [00:00<00:00, 3960.04it/s, Materializing param=encoder.layer.21.attention.attention.query.bias]
Loading weights:  88%|████████▊ | 387/439 [00:00<00:00, 3968.18it/s, Materializing param=encoder.layer.21.attention.attention.query.weight]
Loading weights:  88%|████████▊ | 387/439 [00:00<00:00, 3966.75it/s, Materializing param=encoder.layer.21.attention.attention.query.weight]
Loading weights:  88%|████████▊ | 388/439 [00:00<00:00, 3974.48it/s, Materializing param=encoder.layer.21.attention.attention.value.bias]  
Loading weights:  88%|████████▊ | 388/439 [00:00<00:00, 3973.05it/s, Materializing param=encoder.layer.21.attention.attention.value.bias]
Loading weights:  89%|████████▊ | 389/439 [00:00<00:00, 3981.12it/s, Materializing param=encoder.layer.21.attention.attention.value.weight]
Loading weights:  89%|████████▊ | 389/439 [00:00<00:00, 3979.79it/s, Materializing param=encoder.layer.21.attention.attention.value.weight]
Loading weights:  89%|████████▉ | 390/439 [00:00<00:00, 3987.89it/s, Materializing param=encoder.layer.21.attention.output.dense.bias]     
Loading weights:  89%|████████▉ | 390/439 [00:00<00:00, 3986.48it/s, Materializing param=encoder.layer.21.attention.output.dense.bias]
Loading weights:  89%|████████▉ | 391/439 [00:00<00:00, 3994.54it/s, Materializing param=encoder.layer.21.attention.output.dense.weight]
Loading weights:  89%|████████▉ | 391/439 [00:00<00:00, 3993.11it/s, Materializing param=encoder.layer.21.attention.output.dense.weight]
Loading weights:  89%|████████▉ | 392/439 [00:00<00:00, 4001.39it/s, Materializing param=encoder.layer.21.layer_scale1.lambda1]         
Loading weights:  89%|████████▉ | 392/439 [00:00<00:00, 4000.16it/s, Materializing param=encoder.layer.21.layer_scale1.lambda1]
Loading weights:  90%|████████▉ | 393/439 [00:00<00:00, 4008.42it/s, Materializing param=encoder.layer.21.layer_scale2.lambda1]
Loading weights:  90%|████████▉ | 393/439 [00:00<00:00, 4007.15it/s, Materializing param=encoder.layer.21.layer_scale2.lambda1]
Loading weights:  90%|████████▉ | 394/439 [00:00<00:00, 4015.38it/s, Materializing param=encoder.layer.21.mlp.fc1.bias]        
Loading weights:  90%|████████▉ | 394/439 [00:00<00:00, 4014.15it/s, Materializing param=encoder.layer.21.mlp.fc1.bias]
Loading weights:  90%|████████▉ | 395/439 [00:00<00:00, 4022.00it/s, Materializing param=encoder.layer.21.mlp.fc1.weight]
Loading weights:  90%|████████▉ | 395/439 [00:00<00:00, 4020.73it/s, Materializing param=encoder.layer.21.mlp.fc1.weight]
Loading weights:  90%|█████████ | 396/439 [00:00<00:00, 4028.94it/s, Materializing param=encoder.layer.21.mlp.fc2.bias]  
Loading weights:  90%|█████████ | 396/439 [00:00<00:00, 4027.70it/s, Materializing param=encoder.layer.21.mlp.fc2.bias]
Loading weights:  90%|█████████ | 397/439 [00:00<00:00, 4035.79it/s, Materializing param=encoder.layer.21.mlp.fc2.weight]
Loading weights:  90%|█████████ | 397/439 [00:00<00:00, 4034.52it/s, Materializing param=encoder.layer.21.mlp.fc2.weight]
Loading weights:  91%|█████████ | 398/439 [00:00<00:00, 4039.14it/s, Materializing param=encoder.layer.21.norm1.bias]    
Loading weights:  91%|█████████ | 398/439 [00:00<00:00, 4035.70it/s, Materializing param=encoder.layer.21.norm1.bias]
Loading weights:  91%|█████████ | 399/439 [00:00<00:00, 4042.02it/s, Materializing param=encoder.layer.21.norm1.weight]
Loading weights:  91%|█████████ | 399/439 [00:00<00:00, 4040.26it/s, Materializing param=encoder.layer.21.norm1.weight]
Loading weights:  91%|█████████ | 400/439 [00:00<00:00, 4045.48it/s, Materializing param=encoder.layer.21.norm2.bias]  
Loading weights:  91%|█████████ | 400/439 [00:00<00:00, 4043.55it/s, Materializing param=encoder.layer.21.norm2.bias]
Loading weights:  91%|█████████▏| 401/439 [00:00<00:00, 4048.10it/s, Materializing param=encoder.layer.21.norm2.weight]
Loading weights:  91%|█████████▏| 401/439 [00:00<00:00, 4045.27it/s, Materializing param=encoder.layer.21.norm2.weight]
Loading weights:  92%|█████████▏| 402/439 [00:00<00:00, 4051.56it/s, Materializing param=encoder.layer.22.attention.attention.key.bias]
Loading weights:  92%|█████████▏| 402/439 [00:00<00:00, 4049.77it/s, Materializing param=encoder.layer.22.attention.attention.key.bias]
Loading weights:  92%|█████████▏| 403/439 [00:00<00:00, 4056.99it/s, Materializing param=encoder.layer.22.attention.attention.key.weight]
Loading weights:  92%|█████████▏| 403/439 [00:00<00:00, 4055.35it/s, Materializing param=encoder.layer.22.attention.attention.key.weight]
Loading weights:  92%|█████████▏| 404/439 [00:00<00:00, 4062.80it/s, Materializing param=encoder.layer.22.attention.attention.query.bias]
Loading weights:  92%|█████████▏| 404/439 [00:00<00:00, 4061.20it/s, Materializing param=encoder.layer.22.attention.attention.query.bias]
Loading weights:  92%|█████████▏| 405/439 [00:00<00:00, 4068.77it/s, Materializing param=encoder.layer.22.attention.attention.query.weight]
Loading weights:  92%|█████████▏| 405/439 [00:00<00:00, 4067.17it/s, Materializing param=encoder.layer.22.attention.attention.query.weight]
Loading weights:  92%|█████████▏| 406/439 [00:00<00:00, 4074.75it/s, Materializing param=encoder.layer.22.attention.attention.value.bias]  
Loading weights:  92%|█████████▏| 406/439 [00:00<00:00, 4073.25it/s, Materializing param=encoder.layer.22.attention.attention.value.bias]
Loading weights:  93%|█████████▎| 407/439 [00:00<00:00, 4080.90it/s, Materializing param=encoder.layer.22.attention.attention.value.weight]
Loading weights:  93%|█████████▎| 407/439 [00:00<00:00, 4079.38it/s, Materializing param=encoder.layer.22.attention.attention.value.weight]
Loading weights:  93%|█████████▎| 408/439 [00:00<00:00, 4087.08it/s, Materializing param=encoder.layer.22.attention.output.dense.bias]     
Loading weights:  93%|█████████▎| 408/439 [00:00<00:00, 4085.56it/s, Materializing param=encoder.layer.22.attention.output.dense.bias]
Loading weights:  93%|█████████▎| 409/439 [00:00<00:00, 4093.28it/s, Materializing param=encoder.layer.22.attention.output.dense.weight]
Loading weights:  93%|█████████▎| 409/439 [00:00<00:00, 4091.85it/s, Materializing param=encoder.layer.22.attention.output.dense.weight]
Loading weights:  93%|█████████▎| 410/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.attention.output.dense.weight]
Loading weights:  93%|█████████▎| 410/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.layer_scale1.lambda1]         
Loading weights:  93%|█████████▎| 410/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.layer_scale1.lambda1]
Loading weights:  94%|█████████▎| 411/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.layer_scale2.lambda1]
Loading weights:  94%|█████████▎| 411/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.layer_scale2.lambda1]
Loading weights:  94%|█████████▍| 412/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc1.bias]        
Loading weights:  94%|█████████▍| 412/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc1.bias]
Loading weights:  94%|█████████▍| 413/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 413/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 414/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc2.bias]  
Loading weights:  94%|█████████▍| 414/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc2.bias]
Loading weights:  95%|█████████▍| 415/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 415/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 416/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm1.bias]    
Loading weights:  95%|█████████▍| 416/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm1.bias]
Loading weights:  95%|█████████▍| 417/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm1.weight]
Loading weights:  95%|█████████▍| 417/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm1.weight]
Loading weights:  95%|█████████▌| 418/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm2.bias]  
Loading weights:  95%|█████████▌| 418/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm2.bias]
Loading weights:  95%|█████████▌| 419/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm2.weight]
Loading weights:  95%|█████████▌| 419/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.22.norm2.weight]
Loading weights:  96%|█████████▌| 420/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.key.bias]
Loading weights:  96%|█████████▌| 420/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.key.bias]
Loading weights:  96%|█████████▌| 421/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.key.weight]
Loading weights:  96%|█████████▌| 421/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.key.weight]
Loading weights:  96%|█████████▌| 422/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.query.bias]
Loading weights:  96%|█████████▌| 422/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.query.bias]
Loading weights:  96%|█████████▋| 423/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.query.weight]
Loading weights:  96%|█████████▋| 423/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.query.weight]
Loading weights:  97%|█████████▋| 424/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.value.bias]  
Loading weights:  97%|█████████▋| 424/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.value.bias]
Loading weights:  97%|█████████▋| 425/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.value.weight]
Loading weights:  97%|█████████▋| 425/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.attention.value.weight]
Loading weights:  97%|█████████▋| 426/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.output.dense.bias]     
Loading weights:  97%|█████████▋| 426/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.output.dense.bias]
Loading weights:  97%|█████████▋| 427/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.output.dense.weight]
Loading weights:  97%|█████████▋| 427/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.attention.output.dense.weight]
Loading weights:  97%|█████████▋| 428/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.layer_scale1.lambda1]         
Loading weights:  97%|█████████▋| 428/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.layer_scale1.lambda1]
Loading weights:  98%|█████████▊| 429/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.layer_scale2.lambda1]
Loading weights:  98%|█████████▊| 429/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.layer_scale2.lambda1]
Loading weights:  98%|█████████▊| 430/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc1.bias]        
Loading weights:  98%|█████████▊| 430/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc1.bias]
Loading weights:  98%|█████████▊| 431/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 431/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 432/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc2.bias]  
Loading weights:  98%|█████████▊| 432/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc2.bias]
Loading weights:  99%|█████████▊| 433/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc2.weight]
Loading weights:  99%|█████████▊| 433/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 434/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm1.bias]    
Loading weights:  99%|█████████▉| 434/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm1.bias]
Loading weights:  99%|█████████▉| 435/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm1.weight]
Loading weights:  99%|█████████▉| 435/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm1.weight]
Loading weights:  99%|█████████▉| 436/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm2.bias]  
Loading weights:  99%|█████████▉| 436/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm2.bias]
Loading weights: 100%|█████████▉| 437/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm2.weight]
Loading weights: 100%|█████████▉| 437/439 [00:00<00:00, 4099.76it/s, Materializing param=encoder.layer.23.norm2.weight]
Loading weights: 100%|█████████▉| 438/439 [00:00<00:00, 4099.76it/s, Materializing param=layernorm.bias]               
Loading weights: 100%|█████████▉| 438/439 [00:00<00:00, 4099.76it/s, Materializing param=layernorm.bias]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 4099.76it/s, Materializing param=layernorm.weight]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 4099.76it/s, Materializing param=layernorm.weight]
Loading weights: 100%|██████████| 439/439 [00:00<00:00, 4277.75it/s, Materializing param=layernorm.weight]
+  ✓ DINOv3: facebook/dinov2-large
+
Loading weights:   0%|          | 0/408 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/408 [00:00<00:00, 49932.19it/s, Materializing param=logit_bias]
Loading weights:   0%|          | 1/408 [00:00<00:00, 8473.34it/s, Materializing param=logit_bias] 
Loading weights:   0%|          | 2/408 [00:00<00:00, 2480.37it/s, Materializing param=logit_scale]
Loading weights:   0%|          | 2/408 [00:00<00:00, 2309.00it/s, Materializing param=logit_scale]
Loading weights:   1%|          | 3/408 [00:00<00:00, 3167.11it/s, Materializing param=text_model.embeddings.position_embedding.weight]
Loading weights:   1%|          | 3/408 [00:00<00:00, 3024.01it/s, Materializing param=text_model.embeddings.position_embedding.weight]
Loading weights:   1%|          | 4/408 [00:00<00:00, 3493.07it/s, Materializing param=text_model.embeddings.token_embedding.weight]   
Loading weights:   1%|          | 4/408 [00:00<00:00, 2313.14it/s, Materializing param=text_model.embeddings.token_embedding.weight]
Loading weights:   1%|          | 5/408 [00:00<00:00, 2039.63it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.bias]
Loading weights:   1%|          | 5/408 [00:00<00:00, 1592.73it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.bias]
Loading weights:   1%|▏         | 6/408 [00:00<00:00, 1778.25it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.weight]
Loading weights:   1%|▏         | 6/408 [00:00<00:00, 1578.88it/s, Materializing param=text_model.encoder.layers.0.layer_norm1.weight]
Loading weights:   2%|▏         | 7/408 [00:00<00:00, 1647.41it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:   2%|▏         | 7/408 [00:00<00:00, 1574.35it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.bias]
Loading weights:   2%|▏         | 8/408 [00:00<00:00, 1390.80it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.weight]
Loading weights:   2%|▏         | 8/408 [00:00<00:00, 1295.69it/s, Materializing param=text_model.encoder.layers.0.layer_norm2.weight]
Loading weights:   2%|▏         | 9/408 [00:00<00:00, 1304.47it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:   2%|▏         | 9/408 [00:00<00:00, 1270.06it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:   2%|▏         | 10/408 [00:00<00:00, 1328.15it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:   2%|▏         | 10/408 [00:00<00:00, 1270.58it/s, Materializing param=text_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:   3%|▎         | 11/408 [00:00<00:00, 1169.07it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:   3%|▎         | 11/408 [00:00<00:00, 1156.64it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:   3%|▎         | 12/408 [00:00<00:00, 1240.43it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:   3%|▎         | 12/408 [00:00<00:00, 1185.53it/s, Materializing param=text_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:   3%|▎         | 13/408 [00:00<00:00, 1085.66it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:   3%|▎         | 13/408 [00:00<00:00, 982.38it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.bias] 
Loading weights:   3%|▎         | 14/408 [00:00<00:00, 1023.89it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:   3%|▎         | 14/408 [00:00<00:00, 1017.51it/s, Materializing param=text_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:   4%|▎         | 15/408 [00:00<00:00, 1036.33it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:   4%|▎         | 15/408 [00:00<00:00, 1030.63it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:   4%|▍         | 16/408 [00:00<00:00, 1090.42it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:   4%|▍         | 16/408 [00:00<00:00, 1079.61it/s, Materializing param=text_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:   4%|▍         | 17/408 [00:00<00:00, 1038.28it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.bias]    
Loading weights:   4%|▍         | 17/408 [00:00<00:00, 1034.98it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.bias]
Loading weights:   4%|▍         | 18/408 [00:00<00:00, 1004.68it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|▍         | 18/408 [00:00<00:00, 999.99it/s, Materializing param=text_model.encoder.layers.0.self_attn.q_proj.weight] 
Loading weights:   5%|▍         | 19/408 [00:00<00:00, 1030.25it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.bias] 
Loading weights:   5%|▍         | 19/408 [00:00<00:00, 1026.40it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.bias]
Loading weights:   5%|▍         | 20/408 [00:00<00:00, 1053.29it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:   5%|▍         | 20/408 [00:00<00:00, 1036.54it/s, Materializing param=text_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 21/408 [00:00<00:00, 1052.67it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.bias]       
Loading weights:   5%|▌         | 21/408 [00:00<00:00, 1000.38it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.bias]
Loading weights:   5%|▌         | 22/408 [00:00<00:00, 1020.11it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.weight]
Loading weights:   5%|▌         | 22/408 [00:00<00:00, 971.52it/s, Materializing param=text_model.encoder.layers.1.layer_norm1.weight] 
Loading weights:   6%|▌         | 23/408 [00:00<00:00, 1006.12it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.bias] 
Loading weights:   6%|▌         | 23/408 [00:00<00:00, 979.88it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.bias] 
Loading weights:   6%|▌         | 24/408 [00:00<00:00, 993.82it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.weight]
Loading weights:   6%|▌         | 24/408 [00:00<00:00, 964.97it/s, Materializing param=text_model.encoder.layers.1.layer_norm2.weight]
Loading weights:   6%|▌         | 25/408 [00:00<00:00, 998.68it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:   6%|▌         | 25/408 [00:00<00:00, 997.00it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:   6%|▋         | 26/408 [00:00<00:00, 1019.24it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:   6%|▋         | 26/408 [00:00<00:00, 1016.45it/s, Materializing param=text_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:   7%|▋         | 27/408 [00:00<00:00, 1022.14it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:   7%|▋         | 27/408 [00:00<00:00, 1001.43it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:   7%|▋         | 28/408 [00:00<00:00, 1021.78it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:   7%|▋         | 28/408 [00:00<00:00, 1004.51it/s, Materializing param=text_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:   7%|▋         | 29/408 [00:00<00:00, 1022.78it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:   7%|▋         | 29/408 [00:00<00:00, 1018.86it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:   7%|▋         | 30/408 [00:00<00:00, 1046.24it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 30/408 [00:00<00:00, 1042.06it/s, Materializing param=text_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:   8%|▊         | 31/408 [00:00<00:00, 1060.19it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:   8%|▊         | 31/408 [00:00<00:00, 1055.92it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:   8%|▊         | 32/408 [00:00<00:00, 1033.25it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:   8%|▊         | 32/408 [00:00<00:00, 1018.58it/s, Materializing param=text_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:   8%|▊         | 33/408 [00:00<00:00, 1026.82it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.bias]    
Loading weights:   8%|▊         | 33/408 [00:00<00:00, 1024.20it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.bias]
Loading weights:   8%|▊         | 34/408 [00:00<00:00, 1028.61it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/408 [00:00<00:00, 1014.34it/s, Materializing param=text_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:   9%|▊         | 35/408 [00:00<00:00, 1031.98it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.bias]  
Loading weights:   9%|▊         | 35/408 [00:00<00:00, 1030.77it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.bias]
Loading weights:   9%|▉         | 36/408 [00:00<00:00, 1021.48it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:   9%|▉         | 36/408 [00:00<00:00, 1018.27it/s, Materializing param=text_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:   9%|▉         | 37/408 [00:00<00:00, 1043.92it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.bias]       
Loading weights:   9%|▉         | 37/408 [00:00<00:00, 1039.23it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.bias]
Loading weights:   9%|▉         | 38/408 [00:00<00:00, 1037.37it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.weight]
Loading weights:   9%|▉         | 38/408 [00:00<00:00, 1025.69it/s, Materializing param=text_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  10%|▉         | 39/408 [00:00<00:00, 1047.68it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  10%|▉         | 39/408 [00:00<00:00, 1035.33it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  10%|▉         | 40/408 [00:00<00:00, 1052.10it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  10%|▉         | 40/408 [00:00<00:00, 1050.64it/s, Materializing param=text_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  10%|█         | 41/408 [00:00<00:00, 1072.42it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  10%|█         | 41/408 [00:00<00:00, 1038.76it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  10%|█         | 42/408 [00:00<00:00, 1045.89it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  10%|█         | 42/408 [00:00<00:00, 1033.53it/s, Materializing param=text_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  11%|█         | 43/408 [00:00<00:00, 1054.84it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  11%|█         | 43/408 [00:00<00:00, 1051.96it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  11%|█         | 44/408 [00:00<00:00, 1050.01it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  11%|█         | 44/408 [00:00<00:00, 1023.32it/s, Materializing param=text_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  11%|█         | 45/408 [00:00<00:00, 1040.43it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  11%|█         | 45/408 [00:00<00:00, 1032.63it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  11%|█▏        | 46/408 [00:00<00:00, 1024.70it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  11%|█▏        | 46/408 [00:00<00:00, 1023.29it/s, Materializing param=text_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 47/408 [00:00<00:00, 1006.57it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  12%|█▏        | 47/408 [00:00<00:00, 962.09it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.bias] 
Loading weights:  12%|█▏        | 48/408 [00:00<00:00, 974.90it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  12%|█▏        | 48/408 [00:00<00:00, 972.72it/s, Materializing param=text_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  12%|█▏        | 49/408 [00:00<00:00, 963.50it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.bias]    
Loading weights:  12%|█▏        | 49/408 [00:00<00:00, 961.14it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.bias]
Loading weights:  12%|█▏        | 50/408 [00:00<00:00, 973.04it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  12%|█▏        | 50/408 [00:00<00:00, 960.68it/s, Materializing param=text_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  12%|█▎        | 51/408 [00:00<00:00, 969.10it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.bias]  
Loading weights:  12%|█▎        | 51/408 [00:00<00:00, 962.19it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.bias]
Loading weights:  13%|█▎        | 52/408 [00:00<00:00, 968.46it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 52/408 [00:00<00:00, 960.97it/s, Materializing param=text_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 53/408 [00:00<00:00, 976.96it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.bias]       
Loading weights:  13%|█▎        | 53/408 [00:00<00:00, 975.97it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  13%|█▎        | 54/408 [00:00<00:00, 967.84it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  13%|█▎        | 54/408 [00:00<00:00, 962.09it/s, Materializing param=text_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  13%|█▎        | 55/408 [00:00<00:00, 960.88it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  13%|█▎        | 55/408 [00:00<00:00, 953.35it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  14%|█▎        | 56/408 [00:00<00:00, 960.81it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  14%|█▎        | 56/408 [00:00<00:00, 960.10it/s, Materializing param=text_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  14%|█▍        | 57/408 [00:00<00:00, 948.64it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  14%|█▍        | 57/408 [00:00<00:00, 942.52it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  14%|█▍        | 58/408 [00:00<00:00, 956.90it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  14%|█▍        | 58/408 [00:00<00:00, 947.32it/s, Materializing param=text_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  14%|█▍        | 59/408 [00:00<00:00, 946.28it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  14%|█▍        | 59/408 [00:00<00:00, 938.07it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  15%|█▍        | 60/408 [00:00<00:00, 941.79it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  15%|█▍        | 60/408 [00:00<00:00, 941.19it/s, Materializing param=text_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  15%|█▍        | 61/408 [00:00<00:00, 945.12it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  15%|█▍        | 61/408 [00:00<00:00, 935.87it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  15%|█▌        | 62/408 [00:00<00:00, 936.24it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  15%|█▌        | 62/408 [00:00<00:00, 935.32it/s, Materializing param=text_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  15%|█▌        | 63/408 [00:00<00:00, 943.98it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  15%|█▌        | 63/408 [00:00<00:00, 940.16it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  16%|█▌        | 64/408 [00:00<00:00, 953.70it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  16%|█▌        | 64/408 [00:00<00:00, 952.80it/s, Materializing param=text_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  16%|█▌        | 65/408 [00:00<00:00, 964.85it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.bias]    
Loading weights:  16%|█▌        | 65/408 [00:00<00:00, 964.10it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.bias]
Loading weights:  16%|█▌        | 66/408 [00:00<00:00, 977.92it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  16%|█▌        | 66/408 [00:00<00:00, 977.37it/s, Materializing param=text_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  16%|█▋        | 67/408 [00:00<00:00, 991.31it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.bias]  
Loading weights:  16%|█▋        | 67/408 [00:00<00:00, 990.81it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.bias]
Loading weights:  17%|█▋        | 68/408 [00:00<00:00, 1004.75it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  17%|█▋        | 68/408 [00:00<00:00, 1004.27it/s, Materializing param=text_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  17%|█▋        | 69/408 [00:00<00:00, 1018.21it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.bias]       
Loading weights:  17%|█▋        | 69/408 [00:00<00:00, 1017.73it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  17%|█▋        | 70/408 [00:00<00:00, 1031.65it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  17%|█▋        | 70/408 [00:00<00:00, 1030.64it/s, Materializing param=text_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  17%|█▋        | 71/408 [00:00<00:00, 1043.73it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  17%|█▋        | 71/408 [00:00<00:00, 1043.04it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  18%|█▊        | 72/408 [00:00<00:00, 1056.68it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  18%|█▊        | 72/408 [00:00<00:00, 1056.10it/s, Materializing param=text_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  18%|█▊        | 73/408 [00:00<00:00, 1069.84it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  18%|█▊        | 73/408 [00:00<00:00, 1069.28it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  18%|█▊        | 74/408 [00:00<00:00, 1082.98it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  18%|█▊        | 74/408 [00:00<00:00, 1082.40it/s, Materializing param=text_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  18%|█▊        | 75/408 [00:00<00:00, 1096.12it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  18%|█▊        | 75/408 [00:00<00:00, 1095.56it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  19%|█▊        | 76/408 [00:00<00:00, 1109.28it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  19%|█▊        | 76/408 [00:00<00:00, 1108.72it/s, Materializing param=text_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  19%|█▉        | 77/408 [00:00<00:00, 1122.36it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  19%|█▉        | 77/408 [00:00<00:00, 1121.78it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  19%|█▉        | 78/408 [00:00<00:00, 1135.42it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 78/408 [00:00<00:00, 1134.85it/s, Materializing param=text_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 79/408 [00:00<00:00, 1148.48it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  19%|█▉        | 79/408 [00:00<00:00, 1147.93it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  20%|█▉        | 80/408 [00:00<00:00, 1161.51it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  20%|█▉        | 80/408 [00:00<00:00, 1160.95it/s, Materializing param=text_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  20%|█▉        | 81/408 [00:00<00:00, 1174.52it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.bias]    
Loading weights:  20%|█▉        | 81/408 [00:00<00:00, 1173.94it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.bias]
Loading weights:  20%|██        | 82/408 [00:00<00:00, 1187.46it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  20%|██        | 82/408 [00:00<00:00, 1186.87it/s, Materializing param=text_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  20%|██        | 83/408 [00:00<00:00, 1200.43it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.bias]  
Loading weights:  20%|██        | 83/408 [00:00<00:00, 1199.82it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.bias]
Loading weights:  21%|██        | 84/408 [00:00<00:00, 1213.35it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  21%|██        | 84/408 [00:00<00:00, 1212.75it/s, Materializing param=text_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  21%|██        | 85/408 [00:00<00:00, 1226.23it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.bias]       
Loading weights:  21%|██        | 85/408 [00:00<00:00, 1225.62it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  21%|██        | 86/408 [00:00<00:00, 1239.12it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  21%|██        | 86/408 [00:00<00:00, 1238.49it/s, Materializing param=text_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  21%|██▏       | 87/408 [00:00<00:00, 1251.94it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  21%|██▏       | 87/408 [00:00<00:00, 1251.33it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  22%|██▏       | 88/408 [00:00<00:00, 1264.73it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  22%|██▏       | 88/408 [00:00<00:00, 1264.07it/s, Materializing param=text_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  22%|██▏       | 89/408 [00:00<00:00, 1277.54it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  22%|██▏       | 89/408 [00:00<00:00, 1276.94it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  22%|██▏       | 90/408 [00:00<00:00, 1290.30it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  22%|██▏       | 90/408 [00:00<00:00, 1289.69it/s, Materializing param=text_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  22%|██▏       | 91/408 [00:00<00:00, 1303.07it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  22%|██▏       | 91/408 [00:00<00:00, 1302.43it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  23%|██▎       | 92/408 [00:00<00:00, 1315.79it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  23%|██▎       | 92/408 [00:00<00:00, 1315.16it/s, Materializing param=text_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  23%|██▎       | 93/408 [00:00<00:00, 1328.47it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  23%|██▎       | 93/408 [00:00<00:00, 1327.81it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  23%|██▎       | 94/408 [00:00<00:00, 1341.09it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  23%|██▎       | 94/408 [00:00<00:00, 1340.44it/s, Materializing param=text_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  23%|██▎       | 95/408 [00:00<00:00, 1353.72it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  23%|██▎       | 95/408 [00:00<00:00, 1353.10it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  24%|██▎       | 96/408 [00:00<00:00, 1366.31it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  24%|██▎       | 96/408 [00:00<00:00, 1365.65it/s, Materializing param=text_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  24%|██▍       | 97/408 [00:00<00:00, 1378.84it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.bias]    
Loading weights:  24%|██▍       | 97/408 [00:00<00:00, 1378.17it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.bias]
Loading weights:  24%|██▍       | 98/408 [00:00<00:00, 1391.33it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  24%|██▍       | 98/408 [00:00<00:00, 1390.66it/s, Materializing param=text_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  24%|██▍       | 99/408 [00:00<00:00, 1403.81it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.bias]  
Loading weights:  24%|██▍       | 99/408 [00:00<00:00, 1403.16it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.bias]
Loading weights:  25%|██▍       | 100/408 [00:00<00:00, 1416.29it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  25%|██▍       | 100/408 [00:00<00:00, 1415.58it/s, Materializing param=text_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  25%|██▍       | 101/408 [00:00<00:00, 1428.71it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.bias]       
Loading weights:  25%|██▍       | 101/408 [00:00<00:00, 1428.02it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  25%|██▌       | 102/408 [00:00<00:00, 1441.10it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  25%|██▌       | 102/408 [00:00<00:00, 1440.43it/s, Materializing param=text_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  25%|██▌       | 103/408 [00:00<00:00, 1453.42it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  25%|██▌       | 103/408 [00:00<00:00, 1452.73it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  25%|██▌       | 104/408 [00:00<00:00, 1465.80it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  25%|██▌       | 104/408 [00:00<00:00, 1465.10it/s, Materializing param=text_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  26%|██▌       | 105/408 [00:00<00:00, 1478.14it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  26%|██▌       | 105/408 [00:00<00:00, 1477.43it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  26%|██▌       | 106/408 [00:00<00:00, 1490.41it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  26%|██▌       | 106/408 [00:00<00:00, 1489.72it/s, Materializing param=text_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  26%|██▌       | 107/408 [00:00<00:00, 1502.66it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  26%|██▌       | 107/408 [00:00<00:00, 1501.98it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  26%|██▋       | 108/408 [00:00<00:00, 1514.98it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  26%|██▋       | 108/408 [00:00<00:00, 1514.34it/s, Materializing param=text_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  27%|██▋       | 109/408 [00:00<00:00, 1527.38it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  27%|██▋       | 109/408 [00:00<00:00, 1526.73it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  27%|██▋       | 110/408 [00:00<00:00, 1539.67it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 110/408 [00:00<00:00, 1538.99it/s, Materializing param=text_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 111/408 [00:00<00:00, 1551.94it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  27%|██▋       | 111/408 [00:00<00:00, 1551.29it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  27%|██▋       | 112/408 [00:00<00:00, 1564.13it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  27%|██▋       | 112/408 [00:00<00:00, 1563.41it/s, Materializing param=text_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  28%|██▊       | 113/408 [00:00<00:00, 1576.21it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.bias]    
Loading weights:  28%|██▊       | 113/408 [00:00<00:00, 1575.50it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.bias]
Loading weights:  28%|██▊       | 114/408 [00:00<00:00, 1588.33it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 114/408 [00:00<00:00, 1587.60it/s, Materializing param=text_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 115/408 [00:00<00:00, 1600.40it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.bias]  
Loading weights:  28%|██▊       | 115/408 [00:00<00:00, 1599.68it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.bias]
Loading weights:  28%|██▊       | 116/408 [00:00<00:00, 1612.45it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 116/408 [00:00<00:00, 1611.69it/s, Materializing param=text_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  29%|██▊       | 117/408 [00:00<00:00, 1624.48it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.bias]       
Loading weights:  29%|██▊       | 117/408 [00:00<00:00, 1623.76it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  29%|██▉       | 118/408 [00:00<00:00, 1636.48it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  29%|██▉       | 118/408 [00:00<00:00, 1635.73it/s, Materializing param=text_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  29%|██▉       | 119/408 [00:00<00:00, 1648.40it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  29%|██▉       | 119/408 [00:00<00:00, 1647.69it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  29%|██▉       | 120/408 [00:00<00:00, 1660.44it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  29%|██▉       | 120/408 [00:00<00:00, 1659.75it/s, Materializing param=text_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  30%|██▉       | 121/408 [00:00<00:00, 1672.49it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  30%|██▉       | 121/408 [00:00<00:00, 1671.52it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  30%|██▉       | 122/408 [00:00<00:00, 1684.17it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  30%|██▉       | 122/408 [00:00<00:00, 1683.43it/s, Materializing param=text_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  30%|███       | 123/408 [00:00<00:00, 1696.03it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  30%|███       | 123/408 [00:00<00:00, 1695.26it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  30%|███       | 124/408 [00:00<00:00, 1707.82it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  30%|███       | 124/408 [00:00<00:00, 1707.04it/s, Materializing param=text_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  31%|███       | 125/408 [00:00<00:00, 1719.58it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  31%|███       | 125/408 [00:00<00:00, 1718.78it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  31%|███       | 126/408 [00:00<00:00, 1731.24it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  31%|███       | 126/408 [00:00<00:00, 1730.41it/s, Materializing param=text_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  31%|███       | 127/408 [00:00<00:00, 1742.92it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  31%|███       | 127/408 [00:00<00:00, 1742.11it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  31%|███▏      | 128/408 [00:00<00:00, 1754.53it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  31%|███▏      | 128/408 [00:00<00:00, 1753.76it/s, Materializing param=text_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  32%|███▏      | 129/408 [00:00<00:00, 1766.20it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.bias]    
Loading weights:  32%|███▏      | 129/408 [00:00<00:00, 1765.35it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.bias]
Loading weights:  32%|███▏      | 130/408 [00:00<00:00, 1777.77it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 130/408 [00:00<00:00, 1776.99it/s, Materializing param=text_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 131/408 [00:00<00:00, 1789.37it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.bias]  
Loading weights:  32%|███▏      | 131/408 [00:00<00:00, 1788.59it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.bias]
Loading weights:  32%|███▏      | 132/408 [00:00<00:00, 1800.94it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  32%|███▏      | 132/408 [00:00<00:00, 1800.07it/s, Materializing param=text_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 133/408 [00:00<00:00, 1812.45it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.bias]       
Loading weights:  33%|███▎      | 133/408 [00:00<00:00, 1811.64it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  33%|███▎      | 134/408 [00:00<00:00, 1823.94it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  33%|███▎      | 134/408 [00:00<00:00, 1823.10it/s, Materializing param=text_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  33%|███▎      | 135/408 [00:00<00:00, 1835.38it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  33%|███▎      | 135/408 [00:00<00:00, 1834.51it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  33%|███▎      | 136/408 [00:00<00:00, 1846.79it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  33%|███▎      | 136/408 [00:00<00:00, 1845.97it/s, Materializing param=text_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  34%|███▎      | 137/408 [00:00<00:00, 1858.22it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  34%|███▎      | 137/408 [00:00<00:00, 1857.37it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  34%|███▍      | 138/408 [00:00<00:00, 1869.56it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  34%|███▍      | 138/408 [00:00<00:00, 1868.70it/s, Materializing param=text_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  34%|███▍      | 139/408 [00:00<00:00, 1880.91it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  34%|███▍      | 139/408 [00:00<00:00, 1880.08it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  34%|███▍      | 140/408 [00:00<00:00, 1892.30it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  34%|███▍      | 140/408 [00:00<00:00, 1891.46it/s, Materializing param=text_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  35%|███▍      | 141/408 [00:00<00:00, 1903.65it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  35%|███▍      | 141/408 [00:00<00:00, 1902.78it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  35%|███▍      | 142/408 [00:00<00:00, 1914.85it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 142/408 [00:00<00:00, 1913.92it/s, Materializing param=text_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  35%|███▌      | 143/408 [00:00<00:00, 1926.05it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  35%|███▌      | 143/408 [00:00<00:00, 1925.17it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  35%|███▌      | 144/408 [00:00<00:00, 1937.25it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  35%|███▌      | 144/408 [00:00<00:00, 1936.32it/s, Materializing param=text_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  36%|███▌      | 145/408 [00:00<00:00, 1948.40it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.bias]    
Loading weights:  36%|███▌      | 145/408 [00:00<00:00, 1947.53it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.bias]
Loading weights:  36%|███▌      | 146/408 [00:00<00:00, 1959.57it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  36%|███▌      | 146/408 [00:00<00:00, 1958.70it/s, Materializing param=text_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  36%|███▌      | 147/408 [00:00<00:00, 1970.74it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.bias]  
Loading weights:  36%|███▌      | 147/408 [00:00<00:00, 1969.63it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.bias]
Loading weights:  36%|███▋      | 148/408 [00:00<00:00, 1981.65it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  36%|███▋      | 148/408 [00:00<00:00, 1980.75it/s, Materializing param=text_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  37%|███▋      | 149/408 [00:00<00:00, 1992.70it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.bias]       
Loading weights:  37%|███▋      | 149/408 [00:00<00:00, 1991.79it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  37%|███▋      | 150/408 [00:00<00:00, 2003.76it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  37%|███▋      | 150/408 [00:00<00:00, 2002.83it/s, Materializing param=text_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  37%|███▋      | 151/408 [00:00<00:00, 2014.81it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  37%|███▋      | 151/408 [00:00<00:00, 2013.92it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  37%|███▋      | 152/408 [00:00<00:00, 2025.83it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  37%|███▋      | 152/408 [00:00<00:00, 2024.91it/s, Materializing param=text_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  38%|███▊      | 153/408 [00:00<00:00, 2036.85it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  38%|███▊      | 153/408 [00:00<00:00, 2035.95it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  38%|███▊      | 154/408 [00:00<00:00, 2047.81it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  38%|███▊      | 154/408 [00:00<00:00, 2046.92it/s, Materializing param=text_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  38%|███▊      | 155/408 [00:00<00:00, 2058.79it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  38%|███▊      | 155/408 [00:00<00:00, 2057.88it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  38%|███▊      | 156/408 [00:00<00:00, 2069.73it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  38%|███▊      | 156/408 [00:00<00:00, 2068.83it/s, Materializing param=text_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  38%|███▊      | 157/408 [00:00<00:00, 2080.65it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  38%|███▊      | 157/408 [00:00<00:00, 2079.71it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  39%|███▊      | 158/408 [00:00<00:00, 2091.49it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  39%|███▊      | 158/408 [00:00<00:00, 2090.55it/s, Materializing param=text_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 159/408 [00:00<00:00, 2102.34it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  39%|███▉      | 159/408 [00:00<00:00, 2101.36it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  39%|███▉      | 160/408 [00:00<00:00, 2113.10it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  39%|███▉      | 160/408 [00:00<00:00, 2112.15it/s, Materializing param=text_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  39%|███▉      | 161/408 [00:00<00:00, 2123.87it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.bias]    
Loading weights:  39%|███▉      | 161/408 [00:00<00:00, 2122.94it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.bias]
Loading weights:  40%|███▉      | 162/408 [00:00<00:00, 2122.74it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 162/408 [00:00<00:00, 2120.60it/s, Materializing param=text_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 163/408 [00:00<00:00, 2131.22it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.bias]  
Loading weights:  40%|███▉      | 163/408 [00:00<00:00, 2130.02it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.bias]
Loading weights:  40%|███���      | 164/408 [00:00<00:00, 2140.99it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  40%|████      | 164/408 [00:00<00:00, 2139.92it/s, Materializing param=text_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  40%|████      | 165/408 [00:00<00:00, 2151.23it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.bias]      
Loading weights:  40%|████      | 165/408 [00:00<00:00, 2150.20it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  41%|████      | 166/408 [00:00<00:00, 2161.56it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  41%|████      | 166/408 [00:00<00:00, 2160.56it/s, Materializing param=text_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  41%|████      | 167/408 [00:00<00:00, 2171.99it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  41%|████      | 167/408 [00:00<00:00, 2170.96it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  41%|████      | 168/408 [00:00<00:00, 2182.07it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  41%|████      | 168/408 [00:00<00:00, 2180.31it/s, Materializing param=text_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  41%|████▏     | 169/408 [00:00<00:00, 2191.44it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  41%|████▏     | 169/408 [00:00<00:00, 2190.42it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  42%|████▏     | 170/408 [00:00<00:00, 2201.81it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  42%|████▏     | 170/408 [00:00<00:00, 2200.87it/s, Materializing param=text_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  42%|████▏     | 171/408 [00:00<00:00, 2212.36it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  42%|████▏     | 171/408 [00:00<00:00, 2211.41it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  42%|████▏     | 172/408 [00:00<00:00, 2222.96it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  42%|████▏     | 172/408 [00:00<00:00, 2222.07it/s, Materializing param=text_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  42%|████▏     | 173/408 [00:00<00:00, 2233.11it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  42%|████▏     | 173/408 [00:00<00:00, 2221.42it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  43%|████▎     | 174/408 [00:00<00:00, 2231.76it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  43%|████▎     | 174/408 [00:00<00:00, 2230.62it/s, Materializing param=text_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  43%|████▎     | 175/408 [00:00<00:00, 2241.55it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  43%|████▎     | 175/408 [00:00<00:00, 2240.54it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  43%|████▎     | 176/408 [00:00<00:00, 2251.56it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  43%|████▎     | 176/408 [00:00<00:00, 2250.49it/s, Materializing param=text_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  43%|████▎     | 177/408 [00:00<00:00, 2261.60it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.bias]    
Loading weights:  43%|████▎     | 177/408 [00:00<00:00, 2260.62it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.bias]
Loading weights:  44%|████▎     | 178/408 [00:00<00:00, 2271.68it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  44%|████▎     | 178/408 [00:00<00:00, 2270.02it/s, Materializing param=text_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  44%|████▍     | 179/408 [00:00<00:00, 2280.83it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.bias]  
Loading weights:  44%|████▍     | 179/408 [00:00<00:00, 2279.79it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.bias]
Loading weights:  44%|████▍     | 180/408 [00:00<00:00, 2290.92it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  44%|████▍     | 180/408 [00:00<00:00, 2289.95it/s, Materializing param=text_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  44%|████▍     | 181/408 [00:00<00:00, 2301.13it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.bias]       
Loading weights:  44%|████▍     | 181/408 [00:00<00:00, 2300.10it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  45%|████▍     | 182/408 [00:00<00:00, 2311.22it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  45%|████▍     | 182/408 [00:00<00:00, 2310.25it/s, Materializing param=text_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  45%|████▍     | 183/408 [00:00<00:00, 2321.44it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  45%|████▍     | 183/408 [00:00<00:00, 2320.50it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  45%|████▌     | 184/408 [00:00<00:00, 2331.68it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  45%|████▌     | 184/408 [00:00<00:00, 2330.73it/s, Materializing param=text_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  45%|████▌     | 185/408 [00:00<00:00, 2341.89it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  45%|████▌     | 185/408 [00:00<00:00, 2340.88it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  46%|████▌     | 186/408 [00:00<00:00, 2351.99it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  46%|████▌     | 186/408 [00:00<00:00, 2350.95it/s, Materializing param=text_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  46%|████▌     | 187/408 [00:00<00:00, 2362.03it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  46%|████▌     | 187/408 [00:00<00:00, 2361.07it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  46%|████▌     | 188/408 [00:00<00:00, 2372.14it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  46%|████▌     | 188/408 [00:00<00:00, 2371.19it/s, Materializing param=text_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  46%|████▋     | 189/408 [00:00<00:00, 2382.23it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  46%|████▋     | 189/408 [00:00<00:00, 2381.22it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  47%|████▋     | 190/408 [00:00<00:00, 2392.28it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 190/408 [00:00<00:00, 2391.26it/s, Materializing param=text_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 191/408 [00:00<00:00, 2402.21it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  47%|████▋     | 191/408 [00:00<00:00, 2401.21it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  47%|████▋     | 192/408 [00:00<00:00, 2412.24it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  47%|████▋     | 192/408 [00:00<00:00, 2411.18it/s, Materializing param=text_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  47%|████▋     | 193/408 [00:00<00:00, 2422.15it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.bias]    
Loading weights:  47%|████▋     | 193/408 [00:00<00:00, 2421.15it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.bias]
Loading weights:  48%|████▊     | 194/408 [00:00<00:00, 2432.14it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 194/408 [00:00<00:00, 2431.13it/s, Materializing param=text_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 195/408 [00:00<00:00, 2442.13it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.bias]  
Loading weights:  48%|████▊     | 195/408 [00:00<00:00, 2441.13it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.bias]
Loading weights:  48%|████▊     | 196/408 [00:00<00:00, 2452.11it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 196/408 [00:00<00:00, 2451.10it/s, Materializing param=text_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 197/408 [00:00<00:00, 2462.00it/s, Materializing param=text_model.final_layer_norm.bias]                    
Loading weights:  48%|████▊     | 197/408 [00:00<00:00, 2461.01it/s, Materializing param=text_model.final_layer_norm.bias]
Loading weights:  49%|████▊     | 198/408 [00:00<00:00, 2472.08it/s, Materializing param=text_model.final_layer_norm.weight]
Loading weights:  49%|████▊     | 198/408 [00:00<00:00, 2471.13it/s, Materializing param=text_model.final_layer_norm.weight]
Loading weights:  49%|████▉     | 199/408 [00:00<00:00, 2482.12it/s, Materializing param=text_model.head.bias]              
Loading weights:  49%|████▉     | 199/408 [00:00<00:00, 2481.14it/s, Materializing param=text_model.head.bias]
Loading weights:  49%|████▉     | 200/408 [00:00<00:00, 2492.17it/s, Materializing param=text_model.head.weight]
Loading weights:  49%|████▉     | 200/408 [00:00<00:00, 2491.24it/s, Materializing param=text_model.head.weight]
Loading weights:  49%|████▉     | 201/408 [00:00<00:00, 2502.29it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  49%|████▉     | 201/408 [00:00<00:00, 2501.27it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  50%|████▉     | 202/408 [00:00<00:00, 2511.99it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  50%|████▉     | 202/408 [00:00<00:00, 2510.93it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  50%|████▉     | 203/408 [00:00<00:00, 2521.82it/s, Materializing param=vision_model.embeddings.position_embedding.weight]
Loading weights:  50%|████▉     | 203/408 [00:00<00:00, 2520.82it/s, Materializing param=vision_model.embeddings.position_embedding.weight]
Loading weights:  50%|█████     | 204/408 [00:00<00:00, 2531.64it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]   
Loading weights:  50%|█████     | 204/408 [00:00<00:00, 2530.60it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  50%|█████     | 205/408 [00:00<00:00, 2541.33it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  50%|█████     | 205/408 [00:00<00:00, 2540.33it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  50%|█████     | 206/408 [00:00<00:00, 2551.14it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:  50%|█████     | 206/408 [00:00<00:00, 2550.14it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]
Loading weights:  51%|█████     | 207/408 [00:00<00:00, 2560.87it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  51%|█████     | 207/408 [00:00<00:00, 2559.82it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  51%|█████     | 208/408 [00:00<00:00, 2570.62it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:  51%|█████     | 208/408 [00:00<00:00, 2569.61it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:  51%|█████     | 209/408 [00:00<00:00, 2580.34it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  51%|█████     | 209/408 [00:00<00:00, 2579.28it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  51%|█████▏    | 210/408 [00:00<00:00, 2590.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:  51%|█████▏    | 210/408 [00:00<00:00, 2588.91it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:  52%|█████▏    | 211/408 [00:00<00:00, 2599.57it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  52%|█████▏    | 211/408 [00:00<00:00, 2598.55it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  52%|█████▏    | 212/408 [00:00<00:00, 2609.22it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:  52%|█████▏    | 212/408 [00:00<00:00, 2608.17it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.bias]
Loading weights:  52%|█████▏    | 213/408 [00:00<00:00, 2618.83it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 213/408 [00:00<00:00, 2617.76it/s, Materializing param=vision_model.encoder.layers.0.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 214/408 [00:00<00:00, 2628.41it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:  52%|█████▏    | 214/408 [00:00<00:00, 2627.31it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.bias]
Loading weights:  53%|█████▎    | 215/408 [00:00<00:00, 2637.87it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 215/408 [00:00<00:00, 2636.83it/s, Materializing param=vision_model.encoder.layers.0.self_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 216/408 [00:00<00:00, 2647.45it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.bias]    
Loading weights:  53%|█████▎    | 216/408 [00:00<00:00, 2646.31it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.bias]
Loading weights:  53%|█████▎    | 217/408 [00:00<00:00, 2656.93it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 217/408 [00:00<00:00, 2655.89it/s, Materializing param=vision_model.encoder.layers.0.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 218/408 [00:00<00:00, 2666.50it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.bias]  
Loading weights:  53%|█████▎    | 218/408 [00:00<00:00, 2665.42it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.bias]
Loading weights:  54%|█████▎    | 219/408 [00:00<00:00, 2675.98it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:  54%|█████▎    | 219/408 [00:00<00:00, 2674.93it/s, Materializing param=vision_model.encoder.layers.0.self_attn.v_proj.weight]
Loading weights:  54%|█████▍    | 220/408 [00:00<00:00, 2685.44it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]       
Loading weights:  54%|█████▍    | 220/408 [00:00<00:00, 2684.07it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]
Loading weights:  54%|█████▍    | 221/408 [00:00<00:00, 2694.53it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  54%|█████▍    | 221/408 [00:00<00:00, 2693.41it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  54%|█████▍    | 222/408 [00:00<00:00, 2703.85it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]  
Loading weights:  54%|█████▍    | 222/408 [00:00<00:00, 2702.73it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]
Loading weights:  55%|█████▍    | 223/408 [00:00<00:00, 2713.26it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  55%|█████▍    | 223/408 [00:00<00:00, 2712.19it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  55%|█████▍    | 224/408 [00:00<00:00, 2722.77it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:  55%|█████▍    | 224/408 [00:00<00:00, 2721.78it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:  55%|█████▌    | 225/408 [00:00<00:00, 2732.30it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  55%|█████▌    | 225/408 [00:00<00:00, 2731.20it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  55%|█████▌    | 226/408 [00:00<00:00, 2741.65it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:  55%|█████▌    | 226/408 [00:00<00:00, 2740.55it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:  56%|█████▌    | 227/408 [00:00<00:00, 2751.01it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  56%|█████▌    | 227/408 [00:00<00:00, 2749.91it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  56%|█████▌    | 228/408 [00:00<00:00, 2760.32it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:  56%|█████▌    | 228/408 [00:00<00:00, 2759.21it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.bias]
Loading weights:  56%|█████▌    | 229/408 [00:00<00:00, 2769.60it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 229/408 [00:00<00:00, 2768.43it/s, Materializing param=vision_model.encoder.layers.1.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 230/408 [00:00<00:00, 2778.81it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:  56%|█████▋    | 230/408 [00:00<00:00, 2777.70it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.bias]
Loading weights:  57%|█████▋    | 231/408 [00:00<00:00, 2788.06it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:  57%|█████▋    | 231/408 [00:00<00:00, 2786.99it/s, Materializing param=vision_model.encoder.layers.1.self_attn.out_proj.weight]
Loading weights:  57%|█████▋    | 232/408 [00:00<00:00, 2797.26it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.bias]    
Loading weights:  57%|█████▋    | 232/408 [00:00<00:00, 2796.19it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.bias]
Loading weights:  57%|█████▋    | 233/408 [00:00<00:00, 2806.51it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 233/408 [00:00<00:00, 2805.43it/s, Materializing param=vision_model.encoder.layers.1.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 234/408 [00:00<00:00, 2815.74it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.bias]  
Loading weights:  57%|█████▋    | 234/408 [00:00<00:00, 2814.56it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.bias]
Loading weights:  58%|█████▊    | 235/408 [00:00<00:00, 2824.92it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 235/408 [00:00<00:00, 2823.80it/s, Materializing param=vision_model.encoder.layers.1.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 236/408 [00:00<00:00, 2834.08it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]       
Loading weights:  58%|█████▊    | 236/408 [00:00<00:00, 2832.98it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]
Loading weights:  58%|█████▊    | 237/408 [00:00<00:00, 2843.29it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  58%|█████▊    | 237/408 [00:00<00:00, 2842.09it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  58%|█████▊    | 238/408 [00:00<00:00, 2852.38it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  58%|█████▊    | 238/408 [00:00<00:00, 2851.17it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  59%|█████▊    | 239/408 [00:00<00:00, 2861.37it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  59%|█████▊    | 239/408 [00:00<00:00, 2860.25it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  59%|█████▉    | 240/408 [00:00<00:00, 2870.46it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  59%|█████▉    | 240/408 [00:00<00:00, 2869.30it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  59%|█████▉    | 241/408 [00:00<00:00, 2879.46it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  59%|█████▉    | 241/408 [00:00<00:00, 2878.32it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  59%|█████▉    | 242/408 [00:00<00:00, 2888.44it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  59%|█████▉    | 242/408 [00:00<00:00, 2887.30it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  60%|█████▉    | 243/408 [00:00<00:00, 2897.43it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  60%|█████▉    | 243/408 [00:00<00:00, 2896.26it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  60%|█████▉    | 244/408 [00:00<00:00, 2906.28it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  60%|█████▉    | 244/408 [00:00<00:00, 2905.10it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.bias]
Loading weights:  60%|██████    | 245/408 [00:00<00:00, 2915.21it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  60%|██████    | 245/408 [00:00<00:00, 2914.02it/s, Materializing param=vision_model.encoder.layers.2.self_attn.k_proj.weight]
Loading weights:  60%|██████    | 246/408 [00:00<00:00, 2924.11it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  60%|██████    | 246/408 [00:00<00:00, 2922.97it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.bias]
Loading weights:  61%|██████    | 247/408 [00:00<00:00, 2933.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  61%|██████    | 247/408 [00:00<00:00, 2931.78it/s, Materializing param=vision_model.encoder.layers.2.self_attn.out_proj.weight]
Loading weights:  61%|██████    | 248/408 [00:00<00:00, 2941.80it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.bias]    
Loading weights:  61%|██████    | 248/408 [00:00<00:00, 2940.61it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.bias]
Loading weights:  61%|██████    | 249/408 [00:00<00:00, 2950.51it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  61%|██████    | 249/408 [00:00<00:00, 2949.32it/s, Materializing param=vision_model.encoder.layers.2.self_attn.q_proj.weight]
Loading weights:  61%|██████▏   | 250/408 [00:00<00:00, 2959.17it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.bias]  
Loading weights:  61%|██████▏   | 250/408 [00:00<00:00, 2957.94it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.bias]
Loading weights:  62%|██████▏   | 251/408 [00:00<00:00, 2967.87it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 251/408 [00:00<00:00, 2966.72it/s, Materializing param=vision_model.encoder.layers.2.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 252/408 [00:00<00:00, 2976.74it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]       
Loading weights:  62%|██████▏   | 252/408 [00:00<00:00, 2975.51it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  62%|██████▏   | 253/408 [00:00<00:00, 2985.49it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  62%|██████▏   | 253/408 [00:00<00:00, 2984.29it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  62%|██████▏   | 254/408 [00:00<00:00, 2994.28it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  62%|██████▏   | 254/408 [00:00<00:00, 2993.15it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  62%|██████▎   | 255/408 [00:00<00:00, 3003.06it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  62%|██████▎   | 255/408 [00:00<00:00, 3001.94it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  63%|██████▎   | 256/408 [00:00<00:00, 3011.86it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  63%|██████▎   | 256/408 [00:00<00:00, 3010.58it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  63%|██████▎   | 257/408 [00:00<00:00, 3020.49it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  63%|██████▎   | 257/408 [00:00<00:00, 3019.30it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  63%|██████▎   | 258/408 [00:00<00:00, 3029.09it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  63%|██████▎   | 258/408 [00:00<00:00, 3027.88it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  63%|██████▎   | 259/408 [00:00<00:00, 3037.80it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  63%|██████▎   | 259/408 [00:00<00:00, 3036.59it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  64%|██████▎   | 260/408 [00:00<00:00, 3046.49it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  64%|██████▎   | 260/408 [00:00<00:00, 3045.24it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.bias]
Loading weights:  64%|██████▍   | 261/408 [00:00<00:00, 3055.02it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 261/408 [00:00<00:00, 3053.80it/s, Materializing param=vision_model.encoder.layers.3.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 262/408 [00:00<00:00, 3063.29it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  64%|██████▍   | 262/408 [00:00<00:00, 3044.63it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.bias]
Loading weights:  64%|██████▍   | 263/408 [00:00<00:00, 3052.71it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  64%|██████▍   | 263/408 [00:00<00:00, 3051.50it/s, Materializing param=vision_model.encoder.layers.3.self_attn.out_proj.weight]
Loading weights:  65%|██████▍   | 264/408 [00:00<00:00, 3061.22it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.bias]    
Loading weights:  65%|██████▍   | 264/408 [00:00<00:00, 3060.09it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.bias]
Loading weights:  65%|██████▍   | 265/408 [00:00<00:00, 3069.86it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 265/408 [00:00<00:00, 3068.69it/s, Materializing param=vision_model.encoder.layers.3.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 266/408 [00:00<00:00, 3078.52it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.bias]  
Loading weights:  65%|██████▌   | 266/408 [00:00<00:00, 3077.38it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.bias]
Loading weights:  65%|██████▌   | 267/408 [00:00<00:00, 3087.12it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 267/408 [00:00<00:00, 3085.98it/s, Materializing param=vision_model.encoder.layers.3.self_attn.v_proj.weight]
Loading weights:  66%|██████▌   | 268/408 [00:00<00:00, 3095.65it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]       
Loading weights:  66%|██████▌   | 268/408 [00:00<00:00, 3094.51it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  66%|██████▌   | 269/408 [00:00<00:00, 3104.22it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  66%|██████▌   | 269/408 [00:00<00:00, 3103.08it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  66%|██████▌   | 270/408 [00:00<00:00, 3112.85it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  66%|██████▌   | 270/408 [00:00<00:00, 3111.60it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  66%|██████▋   | 271/408 [00:00<00:00, 3121.24it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  66%|██████▋   | 271/408 [00:00<00:00, 3120.07it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  67%|██████▋   | 272/408 [00:00<00:00, 3129.78it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  67%|██████▋   | 272/408 [00:00<00:00, 3128.62it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  67%|██████▋   | 273/408 [00:00<00:00, 3138.24it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  67%|██████▋   | 273/408 [00:00<00:00, 3137.09it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  67%|██████▋   | 274/408 [00:00<00:00, 3146.78it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  67%|██████▋   | 274/408 [00:00<00:00, 3145.66it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  67%|██████▋   | 275/408 [00:00<00:00, 3155.36it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  67%|██████▋   | 275/408 [00:00<00:00, 3154.21it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  68%|██████▊   | 276/408 [00:00<00:00, 3163.89it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  68%|██████▊   | 276/408 [00:00<00:00, 3162.77it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.bias]
Loading weights:  68%|██████▊   | 277/408 [00:00<00:00, 3172.49it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  68%|██████▊   | 277/408 [00:00<00:00, 3171.36it/s, Materializing param=vision_model.encoder.layers.4.self_attn.k_proj.weight]
Loading weights:  68%|██████▊   | 278/408 [00:00<00:00, 3180.95it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  68%|██████▊   | 278/408 [00:00<00:00, 3179.79it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.bias]
Loading weights:  68%|██████▊   | 279/408 [00:00<00:00, 3189.37it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  68%|██████▊   | 279/408 [00:00<00:00, 3188.17it/s, Materializing param=vision_model.encoder.layers.4.self_attn.out_proj.weight]
Loading weights:  69%|██████▊   | 280/408 [00:00<00:00, 3197.84it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.bias]    
Loading weights:  69%|██████▊   | 280/408 [00:00<00:00, 3196.70it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.bias]
Loading weights:  69%|██████▉   | 281/408 [00:00<00:00, 3206.22it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  69%|██████▉   | 281/408 [00:00<00:00, 3204.97it/s, Materializing param=vision_model.encoder.layers.4.self_attn.q_proj.weight]
Loading weights:  69%|██████▉   | 282/408 [00:00<00:00, 3214.47it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.bias]  
Loading weights:  69%|██████▉   | 282/408 [00:00<00:00, 3213.27it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.bias]
Loading weights:  69%|██████▉   | 283/408 [00:00<00:00, 3222.79it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  69%|██████▉   | 283/408 [00:00<00:00, 3221.57it/s, Materializing param=vision_model.encoder.layers.4.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 284/408 [00:00<00:00, 3231.04it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]       
Loading weights:  70%|██████▉   | 284/408 [00:00<00:00, 3229.80it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  70%|██████▉   | 285/408 [00:00<00:00, 3239.26it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  70%|██████▉   | 285/408 [00:00<00:00, 3238.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  70%|███████   | 286/408 [00:00<00:00, 3247.48it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  70%|███████   | 286/408 [00:00<00:00, 3246.27it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  70%|███████   | 287/408 [00:00<00:00, 3255.77it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  70%|███████   | 287/408 [00:00<00:00, 3254.55it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  71%|███████   | 288/408 [00:00<00:00, 3263.82it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  71%|███████   | 288/408 [00:00<00:00, 3262.60it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  71%|███████   | 289/408 [00:00<00:00, 3272.03it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  71%|███████   | 289/408 [00:00<00:00, 3270.81it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  71%|███████   | 290/408 [00:00<00:00, 3280.20it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  71%|███████   | 290/408 [00:00<00:00, 3279.02it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  71%|███████▏  | 291/408 [00:00<00:00, 3288.39it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  71%|███████▏  | 291/408 [00:00<00:00, 3287.20it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  72%|███████▏  | 292/408 [00:00<00:00, 3296.60it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  72%|███████▏  | 292/408 [00:00<00:00, 3295.37it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.bias]
Loading weights:  72%|███████▏  | 293/408 [00:00<00:00, 3304.53it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 293/408 [00:00<00:00, 3303.15it/s, Materializing param=vision_model.encoder.layers.5.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 294/408 [00:00<00:00, 3312.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  72%|███████▏  | 294/408 [00:00<00:00, 3311.32it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.bias]
Loading weights:  72%|███████▏  | 295/408 [00:00<00:00, 3320.64it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  72%|███████▏  | 295/408 [00:00<00:00, 3319.37it/s, Materializing param=vision_model.encoder.layers.5.self_attn.out_proj.weight]
Loading weights:  73%|███████▎  | 296/408 [00:00<00:00, 3328.68it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.bias]    
Loading weights:  73%|███████▎  | 296/408 [00:00<00:00, 3327.40it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.bias]
Loading weights:  73%|███████▎  | 297/408 [00:00<00:00, 3336.73it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  73%|███████▎  | 297/408 [00:00<00:00, 3335.50it/s, Materializing param=vision_model.encoder.layers.5.self_attn.q_proj.weight]
Loading weights:  73%|███████▎  | 298/408 [00:00<00:00, 3344.78it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.bias]  
Loading weights:  73%|███████▎  | 298/408 [00:00<00:00, 3343.42it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.bias]
Loading weights:  73%|█████��█▎  | 299/408 [00:00<00:00, 3352.68it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  73%|███████▎  | 299/408 [00:00<00:00, 3351.45it/s, Materializing param=vision_model.encoder.layers.5.self_attn.v_proj.weight]
Loading weights:  74%|███████▎  | 300/408 [00:00<00:00, 3360.67it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]       
Loading weights:  74%|███████▎  | 300/408 [00:00<00:00, 3359.34it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  74%|███████▍  | 301/408 [00:00<00:00, 3368.47it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  74%|███████▍  | 301/408 [00:00<00:00, 3367.18it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  74%|███████▍  | 302/408 [00:00<00:00, 3376.29it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  74%|███████▍  | 302/408 [00:00<00:00, 3375.09it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  74%|███████▍  | 303/408 [00:00<00:00, 3384.15it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  74%|███████▍  | 303/408 [00:00<00:00, 3382.90it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  75%|███████▍  | 304/408 [00:00<00:00, 3392.12it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  75%|███████▍  | 304/408 [00:00<00:00, 3390.65it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  75%|███████▍  | 305/408 [00:00<00:00, 3399.50it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  75%|███████▍  | 305/408 [00:00<00:00, 3398.17it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  75%|███████▌  | 306/408 [00:00<00:00, 3407.18it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  75%|███████▌  | 306/408 [00:00<00:00, 3405.86it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  75%|███████▌  | 307/408 [00:00<00:00, 3414.98it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  75%|███████▌  | 307/408 [00:00<00:00, 3413.65it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  75%|███████▌  | 308/408 [00:00<00:00, 3422.67it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  75%|███████▌  | 308/408 [00:00<00:00, 3421.37it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.bias]
Loading weights:  76%|███████▌  | 309/408 [00:00<00:00, 3430.35it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 309/408 [00:00<00:00, 3429.06it/s, Materializing param=vision_model.encoder.layers.6.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 310/408 [00:00<00:00, 3438.17it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  76%|███████▌  | 310/408 [00:00<00:00, 3436.92it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.bias]
Loading weights:  76%|███████▌  | 311/408 [00:00<00:00, 3445.94it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  76%|███████▌  | 311/408 [00:00<00:00, 3444.65it/s, Materializing param=vision_model.encoder.layers.6.self_attn.out_proj.weight]
Loading weights:  76%|███████▋  | 312/408 [00:00<00:00, 3453.69it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.bias]    
Loading weights:  76%|███████▋  | 312/408 [00:00<00:00, 3452.38it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.bias]
Loading weights:  77%|███████▋  | 313/408 [00:00<00:00, 3461.47it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 313/408 [00:00<00:00, 3460.20it/s, Materializing param=vision_model.encoder.layers.6.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 314/408 [00:00<00:00, 3469.26it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.bias]  
Loading weights:  77%|███████▋  | 314/408 [00:00<00:00, 3468.07it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.bias]
Loading weights:  77%|███████▋  | 315/408 [00:00<00:00, 3477.20it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 315/408 [00:00<00:00, 3475.93it/s, Materializing param=vision_model.encoder.layers.6.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 316/408 [00:00<00:00, 3485.04it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]       
Loading weights:  77%|███████▋  | 316/408 [00:00<00:00, 3483.77it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  78%|███████▊  | 317/408 [00:00<00:00, 3492.76it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  78%|███████▊  | 317/408 [00:00<00:00, 3491.48it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  78%|███████▊  | 318/408 [00:00<00:00, 3500.54it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  78%|███████▊  | 318/408 [00:00<00:00, 3499.31it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  78%|███████▊  | 319/408 [00:00<00:00, 3508.34it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  78%|███████▊  | 319/408 [00:00<00:00, 3507.11it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  78%|███████▊  | 320/408 [00:00<00:00, 3516.13it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  78%|███████▊  | 320/408 [00:00<00:00, 3514.78it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  79%|███████▊  | 321/408 [00:00<00:00, 3523.67it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  79%|███████▊  | 321/408 [00:00<00:00, 3522.27it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  79%|███████▉  | 322/408 [00:00<00:00, 3531.23it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  79%|███████▉  | 322/408 [00:00<00:00, 3530.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  79%|███████▉  | 323/408 [00:00<00:00, 3539.06it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  79%|███████▉  | 323/408 [00:00<00:00, 3537.81it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  79%|███████▉  | 324/408 [00:00<00:00, 3546.91it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  79%|███████▉  | 324/408 [00:00<00:00, 3545.70it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.bias]
Loading weights:  80%|███████▉  | 325/408 [00:00<00:00, 3554.70it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  80%|███████▉  | 325/408 [00:00<00:00, 3553.46it/s, Materializing param=vision_model.encoder.layers.7.self_attn.k_proj.weight]
Loading weights:  80%|███████▉  | 326/408 [00:00<00:00, 3562.40it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  80%|███████▉  | 326/408 [00:00<00:00, 3561.16it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.bias]
Loading weights:  80%|████████  | 327/408 [00:00<00:00, 3569.98it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  80%|████████  | 327/408 [00:00<00:00, 3568.69it/s, Materializing param=vision_model.encoder.layers.7.self_attn.out_proj.weight]
Loading weights:  80%|████████  | 328/408 [00:00<00:00, 3577.58it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.bias]    
Loading weights:  80%|████████  | 328/408 [00:00<00:00, 3576.29it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.bias]
Loading weights:  81%|████████  | 329/408 [00:00<00:00, 3585.12it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  81%|████████  | 329/408 [00:00<00:00, 3583.87it/s, Materializing param=vision_model.encoder.layers.7.self_attn.q_proj.weight]
Loading weights:  81%|████████  | 330/408 [00:00<00:00, 3592.73it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.bias]  
Loading weights:  81%|████████  | 330/408 [00:00<00:00, 3591.40it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.bias]
Loading weights:  81%|████████  | 331/408 [00:00<00:00, 3600.20it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  81%|████████  | 331/408 [00:00<00:00, 3598.92it/s, Materializing param=vision_model.encoder.layers.7.self_attn.v_proj.weight]
Loading weights:  81%|████████▏ | 332/408 [00:00<00:00, 3607.75it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]       
Loading weights:  81%|████████▏ | 332/408 [00:00<00:00, 3606.49it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  82%|████████▏ | 333/408 [00:00<00:00, 3615.40it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  82%|████████▏ | 333/408 [00:00<00:00, 3614.10it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  82%|████████▏ | 334/408 [00:00<00:00, 3622.98it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  82%|████████▏ | 334/408 [00:00<00:00, 3621.76it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  82%|████████▏ | 335/408 [00:00<00:00, 3630.68it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  82%|████████▏ | 335/408 [00:00<00:00, 3629.42it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  82%|████████▏ | 336/408 [00:00<00:00, 3638.21it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  82%|████████▏ | 336/408 [00:00<00:00, 3636.91it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  83%|████████▎ | 337/408 [00:00<00:00, 3645.64it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 337/408 [00:00<00:00, 3644.30it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 338/408 [00:00<00:00, 3653.10it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  83%|████████▎ | 338/408 [00:00<00:00, 3651.80it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  83%|████████▎ | 339/408 [00:00<00:00, 3660.54it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 339/408 [00:00<00:00, 3659.28it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 340/408 [00:00<00:00, 3667.98it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  83%|████████▎ | 340/408 [00:00<00:00, 3666.55it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.bias]
Loading weights:  84%|████████▎ | 341/408 [00:00<00:00, 3674.80it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  84%|████████▎ | 341/408 [00:00<00:00, 3673.41it/s, Materializing param=vision_model.encoder.layers.8.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 342/408 [00:00<00:00, 3682.16it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  84%|████████▍ | 342/408 [00:00<00:00, 3680.81it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.bias]
Loading weights:  84%|████████▍ | 343/408 [00:00<00:00, 3689.43it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  84%|████████▍ | 343/408 [00:00<00:00, 3688.09it/s, Materializing param=vision_model.encoder.layers.8.self_attn.out_proj.weight]
Loading weights:  84%|████████▍ | 344/408 [00:00<00:00, 3696.74it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.bias]    
Loading weights:  84%|████████▍ | 344/408 [00:00<00:00, 3695.38it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.bias]
Loading weights:  85%|████████▍ | 345/408 [00:00<00:00, 3704.10it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 345/408 [00:00<00:00, 3702.70it/s, Materializing param=vision_model.encoder.layers.8.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 346/408 [00:00<00:00, 3711.24it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.bias]  
Loading weights:  85%|████████▍ | 346/408 [00:00<00:00, 3709.93it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.bias]
Loading weights:  85%|████████▌ | 347/408 [00:00<00:00, 3718.58it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 347/408 [00:00<00:00, 3717.06it/s, Materializing param=vision_model.encoder.layers.8.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 348/408 [00:00<00:00, 3725.66it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]       
Loading weights:  85%|████████▌ | 348/408 [00:00<00:00, 3724.23it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  86%|████████▌ | 349/408 [00:00<00:00, 3732.85it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  86%|████████▌ | 349/408 [00:00<00:00, 3731.58it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  86%|████████▌ | 350/408 [00:00<00:00, 3740.27it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  86%|████████▌ | 350/408 [00:00<00:00, 3738.91it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  86%|████████▌ | 351/408 [00:00<00:00, 3747.48it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  86%|████████▌ | 351/408 [00:00<00:00, 3746.15it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  86%|████████▋ | 352/408 [00:00<00:00, 3754.77it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  86%|████████▋ | 352/408 [00:00<00:00, 3753.38it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  87%|████████▋ | 353/408 [00:00<00:00, 3761.92it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 353/408 [00:00<00:00, 3760.68it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 354/408 [00:00<00:00, 3769.27it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  87%|████████▋ | 354/408 [00:00<00:00, 3767.99it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  87%|████████▋ | 355/408 [00:00<00:00, 3776.58it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 355/408 [00:00<00:00, 3775.30it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 356/408 [00:00<00:00, 3783.89it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  87%|████████▋ | 356/408 [00:00<00:00, 3782.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.bias]
Loading weights:  88%|████████▊ | 357/408 [00:00<00:00, 3791.05it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  88%|████████▊ | 357/408 [00:00<00:00, 3789.64it/s, Materializing param=vision_model.encoder.layers.9.self_attn.k_proj.weight]
Loading weights:  88%|████████▊ | 358/408 [00:00<00:00, 3798.11it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  88%|████████▊ | 358/408 [00:00<00:00, 3796.79it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.bias]
Loading weights:  88%|████████▊ | 359/408 [00:00<00:00, 3805.29it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  88%|████████▊ | 359/408 [00:00<00:00, 3803.96it/s, Materializing param=vision_model.encoder.layers.9.self_attn.out_proj.weight]
Loading weights:  88%|████████▊ | 360/408 [00:00<00:00, 3812.50it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.bias]    
Loading weights:  88%|████████▊ | 360/408 [00:00<00:00, 3811.13it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.bias]
Loading weights:  88%|████████▊ | 361/408 [00:00<00:00, 3819.57it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  88%|████████▊ | 361/408 [00:00<00:00, 3818.23it/s, Materializing param=vision_model.encoder.layers.9.self_attn.q_proj.weight]
Loading weights:  89%|████████▊ | 362/408 [00:00<00:00, 3826.71it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.bias]  
Loading weights:  89%|████████▊ | 362/408 [00:00<00:00, 3825.38it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.bias]
Loading weights:  89%|████████▉ | 363/408 [00:00<00:00, 3833.88it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 363/408 [00:00<00:00, 3832.46it/s, Materializing param=vision_model.encoder.layers.9.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 364/408 [00:00<00:00, 3840.87it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]      
Loading weights:  89%|████████▉ | 364/408 [00:00<00:00, 3839.53it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  89%|████████▉ | 365/408 [00:00<00:00, 3847.93it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  89%|████████▉ | 365/408 [00:00<00:00, 3846.58it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  90%|████████▉ | 366/408 [00:00<00:00, 3855.10it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  90%|████████▉ | 366/408 [00:00<00:00, 3853.75it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  90%|████████▉ | 367/408 [00:00<00:00, 3862.22it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  90%|████████▉ | 367/408 [00:00<00:00, 3860.87it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  90%|█████████ | 368/408 [00:00<00:00, 3869.36it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  90%|█████████ | 368/408 [00:00<00:00, 3867.98it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  90%|█████████ | 369/408 [00:00<00:00, 3876.28it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  90%|█████████ | 369/408 [00:00<00:00, 3874.94it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  91%|█████████ | 370/408 [00:00<00:00, 3883.45it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  91%|█████████ | 370/408 [00:00<00:00, 3879.09it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  91%|█████████ | 371/408 [00:00<00:00, 3882.17it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  91%|█████████ | 371/408 [00:00<00:00, 3880.06it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  91%|█████████ | 372/408 [00:00<00:00, 3887.55it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  91%|█████████ | 372/408 [00:00<00:00, 3885.92it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.bias]
Loading weights:  91%|█████████▏| 373/408 [00:00<00:00, 3893.69it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 373/408 [00:00<00:00, 3892.14it/s, Materializing param=vision_model.encoder.layers.10.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 374/408 [00:00<00:00, 3900.01it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  92%|█████████▏| 374/408 [00:00<00:00, 3898.50it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.bias]
Loading weights:  92%|█████████▏| 375/408 [00:00<00:00, 3906.52it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  92%|█████████▏| 375/408 [00:00<00:00, 3905.06it/s, Materializing param=vision_model.encoder.layers.10.self_attn.out_proj.weight]
Loading weights:  92%|█████████▏| 376/408 [00:00<00:00, 3913.16it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.bias]    
Loading weights:  92%|█████████▏| 376/408 [00:00<00:00, 3911.68it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.bias]
Loading weights:  92%|█████████▏| 377/408 [00:00<00:00, 3919.69it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 377/408 [00:00<00:00, 3918.22it/s, Materializing param=vision_model.encoder.layers.10.self_attn.q_proj.weight]
Loading weights:  93%|█████████▎| 378/408 [00:00<00:00, 3926.28it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.bias]  
Loading weights:  93%|█████████▎| 378/408 [00:00<00:00, 3924.82it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.bias]
Loading weights:  93%|█████████▎| 379/408 [00:00<00:00, 3933.04it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 379/408 [00:00<00:00, 3931.61it/s, Materializing param=vision_model.encoder.layers.10.self_attn.v_proj.weight]
Loading weights:  93%|███��█████▎| 380/408 [00:00<00:00, 3939.73it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]       
Loading weights:  93%|█████████▎| 380/408 [00:00<00:00, 3938.34it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  93%|█████████▎| 381/408 [00:00<00:00, 3941.72it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  93%|█████████▎| 381/408 [00:00<00:00, 3938.75it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  94%|█████████▎| 382/408 [00:00<00:00, 3945.78it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  94%|█████████▎| 382/408 [00:00<00:00, 3943.99it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  94%|█████████▍| 383/408 [00:00<00:00, 3951.58it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  94%|█████████▍| 383/408 [00:00<00:00, 3949.99it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  94%|█████████▍| 384/408 [00:00<00:00, 3957.89it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  94%|█████████▍| 384/408 [00:00<00:00, 3956.43it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  94%|█████████▍| 385/408 [00:00<00:00, 3963.35it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 385/408 [00:00<00:00, 3961.03it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  95%|█████████▍| 386/408 [00:00<00:00, 3968.66it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  95%|█████████▍| 386/408 [00:00<00:00, 3967.23it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  95%|█████████▍| 387/408 [00:00<00:00, 3975.33it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 387/408 [00:00<00:00, 3973.99it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  95%|█████████▌| 388/408 [00:00<00:00, 3982.04it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  95%|█████████▌| 388/408 [00:00<00:00, 3980.67it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.bias]
Loading weights:  95%|█████████▌| 389/408 [00:00<00:00, 3988.68it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  95%|█████████▌| 389/408 [00:00<00:00, 3987.28it/s, Materializing param=vision_model.encoder.layers.11.self_attn.k_proj.weight]
Loading weights:  96%|█████████▌| 390/408 [00:00<00:00, 3995.32it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  96%|█████████▌| 390/408 [00:00<00:00, 3993.88it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.bias]
Loading weights:  96%|█████████▌| 391/408 [00:00<00:00, 4001.95it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  96%|█████████▌| 391/408 [00:00<00:00, 4000.53it/s, Materializing param=vision_model.encoder.layers.11.self_attn.out_proj.weight]
Loading weights:  96%|█████████▌| 392/408 [00:00<00:00, 4008.54it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.bias]    
Loading weights:  96%|█████████▌| 392/408 [00:00<00:00, 4007.15it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.bias]
Loading weights:  96%|█████████▋| 393/408 [00:00<00:00, 4015.19it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  96%|█████████▋| 393/408 [00:00<00:00, 4013.84it/s, Materializing param=vision_model.encoder.layers.11.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 394/408 [00:00<00:00, 4021.88it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.bias]  
Loading weights:  97%|█████████▋| 394/408 [00:00<00:00, 4020.52it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.bias]
Loading weights:  97%|█████████▋| 395/408 [00:00<00:00, 4028.55it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 395/408 [00:00<00:00, 4027.11it/s, Materializing param=vision_model.encoder.layers.11.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 396/408 [00:00<00:00, 4035.12it/s, Materializing param=vision_model.head.attention.in_proj_bias]              
Loading weights:  97%|█████████▋| 396/408 [00:00<00:00, 4033.77it/s, Materializing param=vision_model.head.attention.in_proj_bias]
Loading weights:  97%|█████████▋| 397/408 [00:00<00:00, 4041.08it/s, Materializing param=vision_model.head.attention.in_proj_weight]
Loading weights:  97%|█████████▋| 397/408 [00:00<00:00, 4039.68it/s, Materializing param=vision_model.head.attention.in_proj_weight]
Loading weights:  98%|█████████▊| 398/408 [00:00<00:00, 4032.61it/s, Materializing param=vision_model.head.attention.out_proj.bias] 
Loading weights:  98%|█████████▊| 398/408 [00:00<00:00, 4030.62it/s, Materializing param=vision_model.head.attention.out_proj.bias]
Loading weights:  98%|█████████▊| 399/408 [00:00<00:00, 4038.08it/s, Materializing param=vision_model.head.attention.out_proj.weight]
Loading weights:  98%|█████████▊| 399/408 [00:00<00:00, 4036.61it/s, Materializing param=vision_model.head.attention.out_proj.weight]
Loading weights:  98%|█████████▊| 400/408 [00:00<00:00, 4044.45it/s, Materializing param=vision_model.head.layernorm.bias]           
Loading weights:  98%|█████████▊| 400/408 [00:00<00:00, 4043.05it/s, Materializing param=vision_model.head.layernorm.bias]
Loading weights:  98%|█████████▊| 401/408 [00:00<00:00, 4051.10it/s, Materializing param=vision_model.head.layernorm.weight]
Loading weights:  98%|█████████▊| 401/408 [00:00<00:00, 4049.73it/s, Materializing param=vision_model.head.layernorm.weight]
Loading weights:  99%|█████████▊| 402/408 [00:00<00:00, 4057.72it/s, Materializing param=vision_model.head.mlp.fc1.bias]    
Loading weights:  99%|█████████▊| 402/408 [00:00<00:00, 4056.39it/s, Materializing param=vision_model.head.mlp.fc1.bias]
Loading weights:  99%|█████████▉| 403/408 [00:00<00:00, 4064.41it/s, Materializing param=vision_model.head.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 403/408 [00:00<00:00, 4063.11it/s, Materializing param=vision_model.head.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 404/408 [00:00<00:00, 4071.13it/s, Materializing param=vision_model.head.mlp.fc2.bias]  
Loading weights:  99%|█████████▉| 404/408 [00:00<00:00, 4069.82it/s, Materializing param=vision_model.head.mlp.fc2.bias]
Loading weights:  99%|█████████▉| 405/408 [00:00<00:00, 4077.85it/s, Materializing param=vision_model.head.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 405/408 [00:00<00:00, 4076.54it/s, Materializing param=vision_model.head.mlp.fc2.weight]
Loading weights: 100%|█████████▉| 406/408 [00:00<00:00, 4084.58it/s, Materializing param=vision_model.head.probe]         
Loading weights: 100%|█████████▉| 406/408 [00:00<00:00, 4083.22it/s, Materializing param=vision_model.head.probe]
Loading weights: 100%|█████████▉| 407/408 [00:00<00:00, 4091.31it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|█████████▉| 407/408 [00:00<00:00, 4090.03it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 4098.10it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 4096.75it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 408/408 [00:00<00:00, 4093.95it/s, Materializing param=vision_model.post_layernorm.weight]
+  ✓ SigLIP: google/siglip-base-patch16-224
+[Oculus] Loading language model...
+
Loading weights:   0%|          | 0/472 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/472 [00:00<00:00, 41527.76it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.bias]
Loading weights:   0%|          | 1/472 [00:00<00:00, 5817.34it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.bias] 
Loading weights:   0%|          | 2/472 [00:00<00:00, 3119.60it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.weight]
Loading weights:   0%|          | 2/472 [00:00<00:00, 1583.35it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.weight]
Loading weights:   1%|          | 3/472 [00:00<00:00, 1691.93it/s, Materializing param=text_decoder.bert.embeddings.position_embeddings.weight]
Loading weights:   1%|          | 3/472 [00:00<00:00, 1103.38it/s, Materializing param=text_decoder.bert.embeddings.position_embeddings.weight]
Loading weights:   1%|          | 4/472 [00:00<00:00, 1238.81it/s, Materializing param=text_decoder.bert.embeddings.word_embeddings.weight]    
Loading weights:   1%|          | 4/472 [00:00<00:00, 1211.70it/s, Materializing param=text_decoder.bert.embeddings.word_embeddings.weight]
Loading weights:   1%|          | 5/472 [00:00<00:00, 1308.92it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   1%|          | 5/472 [00:00<00:00, 1283.37it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   1%|▏         | 6/472 [00:00<00:00, 1487.69it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   1%|▏         | 6/472 [00:00<00:00, 1369.27it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   1%|▏         | 7/472 [00:00<00:00, 1350.08it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.bias]      
Loading weights:   1%|▏         | 7/472 [00:00<00:00, 1238.72it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.bias]
Loading weights:   2%|▏         | 8/472 [00:00<00:00, 1257.05it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.weight]
Loading weights:   2%|▏         | 8/472 [00:00<00:00, 1176.81it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.weight]
Loading weights:   2%|▏         | 9/472 [00:00<00:00, 1243.13it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.bias]      
Loading weights:   2%|▏         | 9/472 [00:00<00:00, 1214.25it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.bias]
Loading weights:   2%|▏         | 10/472 [00:00<00:00, 1268.73it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.weight]
Loading weights:   2%|▏         | 10/472 [00:00<00:00, 1209.47it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.weight]
Loading weights:   2%|▏         | 11/472 [00:00<00:00, 1288.03it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.bias]
Loading weights:   2%|▏         | 11/472 [00:00<00:00, 1223.32it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.bias]
Loading weights:   3%|▎         | 12/472 [00:00<00:00, 1309.19it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.weight]
Loading weights:   3%|▎         | 12/472 [00:00<00:00, 1287.68it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.weight]
Loading weights:   3%|▎         | 13/472 [00:00<00:00, 1375.08it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.bias]  
Loading weights:   3%|▎         | 13/472 [00:00<00:00, 1286.47it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.bias]
Loading weights:   3%|▎         | 14/472 [00:00<00:00, 1297.86it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.weight]
Loading weights:   3%|▎         | 14/472 [00:00<00:00, 1278.75it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.weight]
Loading weights:   3%|▎         | 15/472 [00:00<00:00, 1313.26it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:   3%|▎         | 15/472 [00:00<00:00, 1245.32it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:   3%|▎         | 16/472 [00:00<00:00, 1240.30it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:   3%|▎         | 16/472 [00:00<00:00, 1225.20it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:   4%|▎         | 17/472 [00:00<00:00, 1260.84it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.bias]      
Loading weights:   4%|▎         | 17/472 [00:00<00:00, 1252.96it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.bias]
Loading weights:   4%|▍         | 18/472 [00:00<00:00, 1259.89it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:   4%|▍         | 18/472 [00:00<00:00, 1249.32it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:   4%|▍         | 19/472 [00:00<00:00, 1282.99it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.bias]      
Loading weights:   4%|▍         | 19/472 [00:00<00:00, 1276.54it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.bias]
Loading weights:   4%|▍         | 20/472 [00:00<00:00, 1305.92it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.weight]
Loading weights:   4%|▍         | 20/472 [00:00<00:00, 1249.44it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.weight]
Loading weights:   4%|▍         | 21/472 [00:00<00:00, 1262.10it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.bias]
Loading weights:   4%|▍         | 21/472 [00:00<00:00, 1223.63it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.bias]
Loading weights:   5%|▍         | 22/472 [00:00<00:00, 1273.16it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.weight]
Loading weights:   5%|▍         | 22/472 [00:00<00:00, 1267.58it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.weight]
Loading weights:   5%|▍         | 23/472 [00:00<00:00, 1257.65it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.bias]  
Loading weights:   5%|▍         | 23/472 [00:00<00:00, 1252.50it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.bias]
Loading weights:   5%|▌         | 24/472 [00:00<00:00, 1253.98it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.weight]
Loading weights:   5%|▌         | 24/472 [00:00<00:00, 1234.81it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.weight]
Loading weights:   5%|▌         | 25/472 [00:00<00:00, 1249.26it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.bias]         
Loading weights:   5%|▌         | 25/472 [00:00<00:00, 1212.11it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.bias]
Loading weights:   6%|▌         | 26/472 [00:00<00:00, 1232.87it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.weight]
Loading weights:   6%|▌         | 26/472 [00:00<00:00, 1212.24it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.weight]
Loading weights:   6%|▌         | 27/472 [00:00<00:00, 1207.19it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.bias]    
Loading weights:   6%|▌         | 27/472 [00:00<00:00, 1204.49it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.bias]
Loading weights:   6%|▌         | 28/472 [00:00<00:00, 1212.23it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.weight]
Loading weights:   6%|▌         | 28/472 [00:00<00:00, 1193.97it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.weight]
Loading weights:   6%|▌         | 29/472 [00:00<00:00, 1215.62it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.bias]      
Loading weights:   6%|▌         | 29/472 [00:00<00:00, 1209.95it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.bias]
Loading weights:   6%|▋         | 30/472 [00:00<00:00, 1232.99it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.weight]
Loading weights:   6%|▋         | 30/472 [00:00<00:00, 1207.33it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.weight]
Loading weights:   7%|▋         | 31/472 [00:00<00:00, 1239.25it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:   7%|▋         | 31/472 [00:00<00:00, 1224.87it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:   7%|▋         | 32/472 [00:00<00:00, 1241.99it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:   7%|▋         | 32/472 [00:00<00:00, 1197.77it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:   7%|▋         | 33/472 [00:00<00:00, 1230.15it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.bias]      
Loading weights:   7%|▋         | 33/472 [00:00<00:00, 1225.22it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.bias]
Loading weights:   7%|▋         | 34/472 [00:00<00:00, 1243.19it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 34/472 [00:00<00:00, 1238.89it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.weight]
Loading weights:   7%|▋         | 35/472 [00:00<00:00, 1229.32it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.bias]      
Loading weights:   7%|▋         | 35/472 [00:00<00:00, 1212.50it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.bias]
Loading weights:   8%|▊         | 36/472 [00:00<00:00, 1214.90it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.weight]
Loading weights:   8%|▊         | 36/472 [00:00<00:00, 1198.68it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.weight]
Loading weights:   8%|▊         | 37/472 [00:00<00:00, 1225.70it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.bias]
Loading weights:   8%|▊         | 37/472 [00:00<00:00, 1221.68it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.bias]
Loading weights:   8%|▊         | 38/472 [00:00<00:00, 1208.72it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.weight]
Loading weights:   8%|▊         | 38/472 [00:00<00:00, 1204.13it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.weight]
Loading weights:   8%|▊         | 39/472 [00:00<00:00, 1231.45it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.bias]  
Loading weights:   8%|▊         | 39/472 [00:00<00:00, 1218.26it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.bias]
Loading weights:   8%|▊         | 40/472 [00:00<00:00, 1245.49it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.weight]
Loading weights:   8%|▊         | 40/472 [00:00<00:00, 1232.40it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.weight]
Loading weights:   9%|▊         | 41/472 [00:00<00:00, 1259.45it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:   9%|▊         | 41/472 [00:00<00:00, 1257.71it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:   9%|▉         | 42/472 [00:00<00:00, 1266.43it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:   9%|▉         | 42/472 [00:00<00:00, 1254.63it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:   9%|▉         | 43/472 [00:00<00:00, 1262.77it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.bias]      
Loading weights:   9%|▉         | 43/472 [00:00<00:00, 1257.79it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.bias]
Loading weights:   9%|▉         | 44/472 [00:00<00:00, 1258.76it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:   9%|▉         | 44/472 [00:00<00:00, 1256.89it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:  10%|▉         | 45/472 [00:00<00:00, 1282.45it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.bias]      
Loading weights:  10%|▉         | 45/472 [00:00<00:00, 1280.88it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.bias]
Loading weights:  10%|▉         | 46/472 [00:00<00:00, 1271.60it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.weight]
Loading weights:  10%|▉         | 46/472 [00:00<00:00, 1268.15it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.weight]
Loading weights:  10%|▉         | 47/472 [00:00<00:00, 1255.20it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.bias]
Loading weights:  10%|▉         | 47/472 [00:00<00:00, 1252.26it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.bias]
Loading weights:  10%|█         | 48/472 [00:00<00:00, 1261.53it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.weight]
Loading weights:  10%|█         | 48/472 [00:00<00:00, 1259.38it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.weight]
Loading weights:  10%|█         | 49/472 [00:00<00:00, 1277.96it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.bias]  
Loading weights:  10%|█         | 49/472 [00:00<00:00, 1253.26it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.bias]
Loading weights:  11%|█         | 50/472 [00:00<00:00, 1251.19it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.weight]
Loading weights:  11%|█         | 50/472 [00:00<00:00, 1238.30it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.weight]
Loading weights:  11%|█         | 51/472 [00:00<00:00, 1245.42it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.bias]         
Loading weights:  11%|█         | 51/472 [00:00<00:00, 1243.93it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.bias]
Loading weights:  11%|█         | 52/472 [00:00<00:00, 1264.49it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.weight]
Loading weights:  11%|█         | 52/472 [00:00<00:00, 1262.96it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.weight]
Loading weights:  11%|█         | 53/472 [00:00<00:00, 1273.64it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.bias]    
Loading weights:  11%|█         | 53/472 [00:00<00:00, 1249.94it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.bias]
Loading weights:  11%|█▏        | 54/472 [00:00<00:00, 1250.99it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.weight]
Loading weights:  11%|█▏        | 54/472 [00:00<00:00, 1249.65it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.weight]
Loading weights:  12%|█▏        | 55/472 [00:00<00:00, 1250.06it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.bias]      
Loading weights:  12%|█▏        | 55/472 [00:00<00:00, 1231.86it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.bias]
Loading weights:  12%|█▏        | 56/472 [00:00<00:00, 1244.30it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.weight]
Loading weights:  12%|█▏        | 56/472 [00:00<00:00, 1233.72it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.weight]
Loading weights:  12%|█▏        | 57/472 [00:00<00:00, 1239.77it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  12%|█▏        | 57/472 [00:00<00:00, 1237.46it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  12%|█▏        | 58/472 [00:00<00:00, 1219.61it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  12%|█▏        | 58/472 [00:00<00:00, 1217.03it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  12%|█▎        | 59/472 [00:00<00:00, 1234.95it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.bias]      
Loading weights:  12%|█▎        | 59/472 [00:00<00:00, 1221.03it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.bias]
Loading weights:  13%|█▎        | 60/472 [00:00<00:00, 1205.66it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.weight]
Loading weights:  13%|█▎        | 60/472 [00:00<00:00, 1197.48it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.weight]
Loading weights:  13%|█▎        | 61/472 [00:00<00:00, 1207.01it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.bias]      
Loading weights:  13%|█▎        | 61/472 [00:00<00:00, 1205.73it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.bias]
Loading weights:  13%|█▎        | 62/472 [00:00<00:00, 1217.38it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.weight]
Loading weights:  13%|█▎        | 62/472 [00:00<00:00, 1208.13it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.weight]
Loading weights:  13%|█▎        | 63/472 [00:00<00:00, 1217.51it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.bias]
Loading weights:  13%|█▎        | 63/472 [00:00<00:00, 1215.47it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.bias]
Loading weights:  14%|█▎        | 64/472 [00:00<00:00, 1231.79it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.weight]
Loading weights:  14%|█▎        | 64/472 [00:00<00:00, 1230.25it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.weight]
Loading weights:  14%|█▍        | 65/472 [00:00<00:00, 1225.97it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.bias]  
Loading weights:  14%|█▍        | 65/472 [00:00<00:00, 1217.48it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.bias]
Loading weights:  14%|█▍        | 66/472 [00:00<00:00, 1209.30it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.weight]
Loading weights:  14%|█▍        | 66/472 [00:00<00:00, 1207.93it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.weight]
Loading weights:  14%|█▍        | 67/472 [00:00<00:00, 1212.60it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:  14%|█▍        | 67/472 [00:00<00:00, 1204.84it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:  14%|█▍        | 68/472 [00:00<00:00, 1198.60it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:  14%|█▍        | 68/472 [00:00<00:00, 1188.35it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:  15%|█▍        | 69/472 [00:00<00:00, 1185.77it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.bias]      
Loading weights:  15%|█▍        | 69/472 [00:00<00:00, 1174.05it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.bias]
Loading weights:  15%|█▍        | 70/472 [00:00<00:00, 1181.61it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:  15%|█��        | 70/472 [00:00<00:00, 1169.73it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:  15%|█▌        | 71/472 [00:00<00:00, 1181.41it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.bias]      
Loading weights:  15%|█▌        | 71/472 [00:00<00:00, 1180.36it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.bias]
Loading weights:  15%|█▌        | 72/472 [00:00<00:00, 1194.16it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.weight]
Loading weights:  15%|█▌        | 72/472 [00:00<00:00, 1188.81it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.weight]
Loading weights:  15%|█▌        | 73/472 [00:00<00:00, 1198.18it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.bias]
Loading weights:  15%|█▌        | 73/472 [00:00<00:00, 1197.39it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.bias]
Loading weights:  16%|█▌        | 74/472 [00:00<00:00, 1197.48it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.weight]
Loading weights:  16%|█▌        | 74/472 [00:00<00:00, 1196.70it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.weight]
Loading weights:  16%|█▌        | 75/472 [00:00<00:00, 1211.20it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.bias]  
Loading weights:  16%|█▌        | 75/472 [00:00<00:00, 1210.42it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.bias]
Loading weights:  16%|█▌        | 76/472 [00:00<00:00, 1217.37it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  16%|█▌        | 76/472 [00:00<00:00, 1211.23it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  16%|█▋        | 77/472 [00:00<00:00, 1220.94it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.bias]         
Loading weights:  16%|█▋        | 77/472 [00:00<00:00, 1220.00it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.bias]
Loading weights:  17%|█▋        | 78/472 [00:00<00:00, 1226.74it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.weight]
Loading weights:  17%|█▋        | 78/472 [00:00<00:00, 1224.64it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.weight]
Loading weights:  17%|█▋        | 79/472 [00:00<00:00, 1237.11it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.bias]    
Loading weights:  17%|█▋        | 79/472 [00:00<00:00, 1236.33it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.bias]
Loading weights:  17%|█▋        | 80/472 [00:00<00:00, 1244.09it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 80/472 [00:00<00:00, 1231.83it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 81/472 [00:00<00:00, 1240.03it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.bias]      
Loading weights:  17%|█▋        | 81/472 [00:00<00:00, 1239.22it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.bias]
Loading weights:  17%|█▋        | 82/472 [00:00<00:00, 1251.87it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.weight]
Loading weights:  17%|█▋        | 82/472 [00:00<00:00, 1247.23it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.weight]
Loading weights:  18%|█▊        | 83/472 [00:00<00:00, 1250.17it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  18%|█▊        | 83/472 [00:00<00:00, 1248.53it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  18%|█▊        | 84/472 [00:00<00:00, 1256.85it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  18%|█▊        | 84/472 [00:00<00:00, 1249.18it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  18%|█▊        | 85/472 [00:00<00:00, 1262.49it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.bias]      
Loading weights:  18%|█▊        | 85/472 [00:00<00:00, 1260.33it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.bias]
Loading weights:  18%|█▊        | 86/472 [00:00<00:00, 1266.25it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.weight]
Loading weights:  18%|█▊        | 86/472 [00:00<00:00, 1265.39it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.weight]
Loading weights:  18%|█▊        | 87/472 [00:00<00:00, 1272.45it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.bias]      
Loading weights:  18%|█▊        | 87/472 [00:00<00:00, 1271.09it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.bias]
Loading weights:  19%|█▊        | 88/472 [00:00<00:00, 1273.68it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.weight]
Loading weights:  19%|█▊        | 88/472 [00:00<00:00, 1272.12it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.weight]
Loading weights:  19%|█▉        | 89/472 [00:00<00:00, 1275.47it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.bias]
Loading weights:  19%|█▉        | 89/472 [00:00<00:00, 1258.20it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.bias]
Loading weights:  19%|█▉        | 90/472 [00:00<00:00, 1268.02it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.weight]
Loading weights:  19%|█▉        | 90/472 [00:00<00:00, 1265.43it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.weight]
Loading weights:  19%|█▉        | 91/472 [00:00<00:00, 1256.49it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.bias]  
Loading weights:  19%|█▉        | 91/472 [00:00<00:00, 1255.59it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.bias]
Loading weights:  19%|█▉        | 92/472 [00:00<00:00, 1268.16it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.weight]
Loading weights:  19%|█▉        | 92/472 [00:00<00:00, 1267.51it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.weight]
Loading weights:  20%|█▉        | 93/472 [00:00<00:00, 1280.22it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  20%|█▉        | 93/472 [00:00<00:00, 1279.57it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  20%|█▉        | 94/472 [00:00<00:00, 1292.22it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  20%|█▉        | 94/472 [00:00<00:00, 1291.60it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  20%|██        | 95/472 [00:00<00:00, 1304.28it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.bias]      
Loading weights:  20%|██        | 95/472 [00:00<00:00, 1303.64it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.bias]
Loading weights:  20%|██        | 96/472 [00:00<00:00, 1316.33it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  20%|██        | 96/472 [00:00<00:00, 1315.70it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  21%|██        | 97/472 [00:00<00:00, 1328.35it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.bias]      
Loading weights:  21%|██        | 97/472 [00:00<00:00, 1327.71it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.bias]
Loading weights:  21%|██        | 98/472 [00:00<00:00, 1340.37it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  21%|██        | 98/472 [00:00<00:00, 1339.77it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  21%|██        | 99/472 [00:00<00:00, 1352.42it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  21%|██        | 99/472 [00:00<00:00, 1351.79it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  21%|██        | 100/472 [00:00<00:00, 1364.42it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  21%|██        | 100/472 [00:00<00:00, 1363.79it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  21%|██▏       | 101/472 [00:00<00:00, 1376.37it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.bias]  
Loading weights:  21%|██▏       | 101/472 [00:00<00:00, 1375.76it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.bias]
Loading weights:  22%|██▏       | 102/472 [00:00<00:00, 1388.32it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  22%|██▏       | 102/472 [00:00<00:00, 1387.64it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  22%|██▏       | 103/472 [00:00<00:00, 1400.18it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.bias]         
Loading weights:  22%|██▏       | 103/472 [00:00<00:00, 1399.55it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.bias]
Loading weights:  22%|██▏       | 104/472 [00:00<00:00, 1412.12it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.weight]
Loading weights:  22%|██▏       | 104/472 [00:00<00:00, 1411.47it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.weight]
Loading weights:  22%|██▏       | 105/472 [00:00<00:00, 1423.94it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.bias]    
Loading weights:  22%|██▏       | 105/472 [00:00<00:00, 1423.09it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.bias]
Loading weights:  22%|██▏       | 106/472 [00:00<00:00, 1434.82it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  22%|██▏       | 106/472 [00:00<00:00, 1434.00it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  23%|██▎       | 107/472 [00:00<00:00, 1446.22it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.bias]      
Loading weights:  23%|██▎       | 107/472 [00:00<00:00, 1445.52it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.bias]
Loading weights:  23%|██▎       | 108/472 [00:00<00:00, 1457.84it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.weight]
Loading weights:  23%|██▎       | 108/472 [00:00<00:00, 1457.11it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.weight]
Loading weights:  23%|██▎       | 109/472 [00:00<00:00, 1469.46it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  23%|██▎       | 109/472 [00:00<00:00, 1468.76it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  23%|██▎       | 110/472 [00:00<00:00, 1481.04it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  23%|██▎       | 110/472 [00:00<00:00, 1480.35it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  24%|██▎       | 111/472 [00:00<00:00, 1484.63it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.bias]      
Loading weights:  24%|██▎       | 111/472 [00:00<00:00, 1483.78it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.bias]
Loading weights:  24%|██▎       | 112/472 [00:00<00:00, 1495.77it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.weight]
Loading weights:  24%|██▎       | 112/472 [00:00<00:00, 1495.05it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.weight]
Loading weights:  24%|██▍       | 113/472 [00:00<00:00, 1507.13it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.bias]      
Loading weights:  24%|██▍       | 113/472 [00:00<00:00, 1506.45it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.bias]
Loading weights:  24%|██▍       | 114/472 [00:00<00:00, 1518.57it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.weight]
Loading weights:  24%|██▍       | 114/472 [00:00<00:00, 1517.49it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.weight]
Loading weights:  24%|██▍       | 115/472 [00:00<00:00, 1528.93it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.bias]
Loading weights:  24%|██▍       | 115/472 [00:00<00:00, 1528.12it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.bias]
Loading weights:  25%|██▍       | 116/472 [00:00<00:00, 1540.03it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.weight]
Loading weights:  25%|██▍       | 116/472 [00:00<00:00, 1539.28it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.weight]
Loading weights:  25%|██▍       | 117/472 [00:00<00:00, 1551.27it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.bias]  
Loading weights:  25%|██▍       | 117/472 [00:00<00:00, 1550.51it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.bias]
Loading weights:  25%|██▌       | 118/472 [00:00<00:00, 1562.52it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.weight]
Loading weights:  25%|██▌       | 118/472 [00:00<00:00, 1561.75it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.weight]
Loading weights:  25%|██▌       | 119/472 [00:00<00:00, 1573.76it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  25%|██▌       | 119/472 [00:00<00:00, 1572.99it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  25%|██▌       | 120/472 [00:00<00:00, 1584.66it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  25%|██▌       | 120/472 [00:00<00:00, 1583.90it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  26%|██▌       | 121/472 [00:00<00:00, 1595.82it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.bias]      
Loading weights:  26%|██▌       | 121/472 [00:00<00:00, 1595.07it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.bias]
Loading weights:  26%|██▌       | 122/472 [00:00<00:00, 1606.97it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  26%|██▌       | 122/472 [00:00<00:00, 1606.23it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  26%|██▌       | 123/472 [00:00<00:00, 1618.14it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.bias]      
Loading weights:  26%|██▌       | 123/472 [00:00<00:00, 1617.40it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.bias]
Loading weights:  26%|██▋       | 124/472 [00:00<00:00, 1629.33it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  26%|██▋       | 124/472 [00:00<00:00, 1628.58it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  26%|██▋       | 125/472 [00:00<00:00, 1640.46it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  26%|██▋       | 125/472 [00:00<00:00, 1639.71it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  27%|██▋       | 126/472 [00:00<00:00, 1651.59it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  27%|██▋       | 126/472 [00:00<00:00, 1650.83it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  27%|██▋       | 127/472 [00:00<00:00, 1662.70it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.bias]  
Loading weights:  27%|██▋       | 127/472 [00:00<00:00, 1661.93it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.bias]
Loading weights:  27%|██▋       | 128/472 [00:00<00:00, 1673.79it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  27%|██▋       | 128/472 [00:00<00:00, 1673.03it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  27%|██▋       | 129/472 [00:00<00:00, 1684.87it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.bias]         
Loading weights:  27%|██▋       | 129/472 [00:00<00:00, 1684.09it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.bias]
Loading weights:  28%|██▊       | 130/472 [00:00<00:00, 1695.86it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.weight]
Loading weights:  28%|██▊       | 130/472 [00:00<00:00, 1695.12it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.weight]
Loading weights:  28%|██▊       | 131/472 [00:00<00:00, 1706.91it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.bias]    
Loading weights:  28%|██▊       | 131/472 [00:00<00:00, 1706.15it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.bias]
Loading weights:  28%|██▊       | 132/472 [00:00<00:00, 1717.87it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  28%|██▊       | 132/472 [00:00<00:00, 1717.12it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  28%|██▊       | 133/472 [00:00<00:00, 1728.89it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.bias]      
Loading weights:  28%|██▊       | 133/472 [00:00<00:00, 1728.13it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.bias]
Loading weights:  28%|██▊       | 134/472 [00:00<00:00, 1739.92it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.weight]
Loading weights:  28%|██▊       | 134/472 [00:00<00:00, 1739.13it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.weight]
Loading weights:  29%|██▊       | 135/472 [00:00<00:00, 1750.93it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  29%|██▊       | 135/472 [00:00<00:00, 1750.15it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  29%|██▉       | 136/472 [00:00<00:00, 1761.72it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  29%|██▉       | 136/472 [00:00<00:00, 1760.93it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  29%|██▉       | 137/472 [00:00<00:00, 1772.59it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.bias]      
Loading weights:  29%|██▉       | 137/472 [00:00<00:00, 1771.81it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.bias]
Loading weights:  29%|██▉       | 138/472 [00:00<00:00, 1783.43it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.weight]
Loading weights:  29%|██▉       | 138/472 [00:00<00:00, 1782.60it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.weight]
Loading weights:  29%|██▉       | 139/472 [00:00<00:00, 1794.22it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.bias]      
Loading weights:  29%|██▉       | 139/472 [00:00<00:00, 1793.41it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.bias]
Loading weights:  30%|██▉       | 140/472 [00:00<00:00, 1804.96it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.weight]
Loading weights:  30%|██▉       | 140/472 [00:00<00:00, 1804.15it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.weight]
Loading weights:  30%|██▉       | 141/472 [00:00<00:00, 1815.68it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.bias]
Loading weights:  30%|██▉       | 141/472 [00:00<00:00, 1814.86it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.bias]
Loading weights:  30%|███       | 142/472 [00:00<00:00, 1826.41it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.weight]
Loading weights:  30%|███       | 142/472 [00:00<00:00, 1825.62it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.weight]
Loading weights:  30%|███       | 143/472 [00:00<00:00, 1837.15it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.bias]  
Loading weights:  30%|███       | 143/472 [00:00<00:00, 1836.32it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.bias]
Loading weights:  31%|███       | 144/472 [00:00<00:00, 1847.86it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.weight]
Loading weights:  31%|███       | 144/472 [00:00<00:00, 1847.06it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.weight]
Loading weights:  31%|███       | 145/472 [00:00<00:00, 1858.57it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  31%|███       | 145/472 [00:00<00:00, 1857.76it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  31%|███       | 146/472 [00:00<00:00, 1869.16it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  31%|███       | 146/472 [00:00<00:00, 1868.32it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  31%|███       | 147/472 [00:00<00:00, 1879.75it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.bias]      
Loading weights:  31%|███       | 147/472 [00:00<00:00, 1878.93it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.bias]
Loading weights:  31%|███▏      | 148/472 [00:00<00:00, 1890.36it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  31%|███▏      | 148/472 [00:00<00:00, 1889.54it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  32%|███▏      | 149/472 [00:00<00:00, 1900.99it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.bias]      
Loading weights:  32%|███▏      | 149/472 [00:00<00:00, 1900.17it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.bias]
Loading weights:  32%|███▏      | 150/472 [00:00<00:00, 1911.53it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  32%|███▏      | 150/472 [00:00<00:00, 1910.71it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  32%|███▏      | 151/472 [00:00<00:00, 1922.10it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  32%|███▏      | 151/472 [00:00<00:00, 1921.27it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  32%|███▏      | 152/472 [00:00<00:00, 1932.64it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  32%|███▏      | 152/472 [00:00<00:00, 1931.78it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  32%|███▏      | 153/472 [00:00<00:00, 1943.15it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.bias]  
Loading weights:  32%|███▏      | 153/472 [00:00<00:00, 1942.29it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.bias]
Loading weights:  33%|███▎      | 154/472 [00:00<00:00, 1953.65it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  33%|███▎      | 154/472 [00:00<00:00, 1952.78it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  33%|███▎      | 155/472 [00:00<00:00, 1964.03it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.bias]         
Loading weights:  33%|███▎      | 155/472 [00:00<00:00, 1963.19it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.bias]
Loading weights:  33%|███▎      | 156/472 [00:00<00:00, 1974.51it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.weight]
Loading weights:  33%|███▎      | 156/472 [00:00<00:00, 1973.66it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.weight]
Loading weights:  33%|███▎      | 157/472 [00:00<00:00, 1984.95it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.bias]    
Loading weights:  33%|███▎      | 157/472 [00:00<00:00, 1984.15it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.bias]
Loading weights:  33%|███▎      | 158/472 [00:00<00:00, 1995.51it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  33%|███▎      | 158/472 [00:00<00:00, 1994.70it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  34%|███▎      | 159/472 [00:00<00:00, 2006.04it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.bias]      
Loading weights:  34%|███▎      | 159/472 [00:00<00:00, 2005.22it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.bias]
Loading weights:  34%|███▍      | 160/472 [00:00<00:00, 2016.50it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.weight]
Loading weights:  34%|███▍      | 160/472 [00:00<00:00, 2015.72it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.weight]
Loading weights:  34%|███▍      | 161/472 [00:00<00:00, 2027.04it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  34%|███▍      | 161/472 [00:00<00:00, 2026.25it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  34%|███▍      | 162/472 [00:00<00:00, 2037.50it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  34%|███▍      | 162/472 [00:00<00:00, 2036.69it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  35%|███▍      | 163/472 [00:00<00:00, 2047.94it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.bias]      
Loading weights:  35%|███▍      | 163/472 [00:00<00:00, 2047.12it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.bias]
Loading weights:  35%|███▍      | 164/472 [00:00<00:00, 2058.33it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.weight]
Loading weights:  35%|███▍      | 164/472 [00:00<00:00, 2057.51it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.weight]
Loading weights:  35%|███▍      | 165/472 [00:00<00:00, 2068.70it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.bias]      
Loading weights:  35%|███▍      | 165/472 [00:00<00:00, 2067.90it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.bias]
Loading weights:  35%|███▌      | 166/472 [00:00<00:00, 2079.08it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.weight]
Loading weights:  35%|███▌      | 166/472 [00:00<00:00, 2078.22it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.weight]
Loading weights:  35%|███▌      | 167/472 [00:00<00:00, 2089.33it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.bias]
Loading weights:  35%|███▌      | 167/472 [00:00<00:00, 2088.46it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.bias]
Loading weights:  36%|███▌      | 168/472 [00:00<00:00, 2099.58it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.weight]
Loading weights:  36%|███▌      | 168/472 [00:00<00:00, 2098.76it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.weight]
Loading weights:  36%|███▌      | 169/472 [00:00<00:00, 2109.86it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.bias]  
Loading weights:  36%|███▌      | 169/472 [00:00<00:00, 2109.02it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.bias]
Loading weights:  36%|███▌      | 170/472 [00:00<00:00, 2120.09it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.weight]
Loading weights:  36%|███▌      | 170/472 [00:00<00:00, 2119.25it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.weight]
Loading weights:  36%|███▌      | 171/472 [00:00<00:00, 2130.37it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  36%|███▌      | 171/472 [00:00<00:00, 2129.54it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  36%|███▋      | 172/472 [00:00<00:00, 2140.53it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  36%|███▋      | 172/472 [00:00<00:00, 2139.60it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  37%|███▋      | 173/472 [00:00<00:00, 2150.17it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.bias]      
Loading weights:  37%|███▋      | 173/472 [00:00<00:00, 2149.28it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.bias]
Loading weights:  37%|███▋      | 174/472 [00:00<00:00, 2160.25it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  37%|███▋      | 174/472 [00:00<00:00, 2159.40it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  37%|███▋      | 175/472 [00:00<00:00, 2170.32it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.bias]      
Loading weights:  37%|███▋      | 175/472 [00:00<00:00, 2169.44it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.bias]
Loading weights:  37%|███▋      | 176/472 [00:00<00:00, 2180.35it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  37%|███▋      | 176/472 [00:00<00:00, 2179.48it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  38%|███▊      | 177/472 [00:00<00:00, 2190.43it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  38%|███▊      | 177/472 [00:00<00:00, 2189.51it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  38%|███▊      | 178/472 [00:00<00:00, 2200.30it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  38%|███▊      | 178/472 [00:00<00:00, 2199.35it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  38%|███▊      | 179/472 [00:00<00:00, 2210.15it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.bias]  
Loading weights:  38%|███▊      | 179/472 [00:00<00:00, 2209.19it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.bias]
Loading weights:  38%|███▊      | 180/472 [00:00<00:00, 2219.97it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  38%|███▊      | 180/472 [00:00<00:00, 2219.04it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  38%|███▊      | 181/472 [00:00<00:00, 2229.83it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.bias]         
Loading weights:  38%|███▊      | 181/472 [00:00<00:00, 2228.90it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.bias]
Loading weights:  39%|███▊      | 182/472 [00:00<00:00, 2239.70it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.weight]
Loading weights:  39%|███▊      | 182/472 [00:00<00:00, 2238.73it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.weight]
Loading weights:  39%|███▉      | 183/472 [00:00<00:00, 2249.54it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.bias]    
Loading weights:  39%|███▉      | 183/472 [00:00<00:00, 2248.57it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.bias]
Loading weights:  39%|███▉      | 184/472 [00:00<00:00, 2259.27it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  39%|███▉      | 184/472 [00:00<00:00, 2258.33it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  39%|███▉      | 185/472 [00:00<00:00, 2268.77it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.bias]      
Loading weights:  39%|███▉      | 185/472 [00:00<00:00, 2267.80it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.bias]
Loading weights:  39%|███▉      | 186/472 [00:00<00:00, 2278.52it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.weight]
Loading weights:  39%|███▉      | 186/472 [00:00<00:00, 2277.54it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.weight]
Loading weights:  40%|███▉      | 187/472 [00:00<00:00, 2288.27it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  40%|███▉      | 187/472 [00:00<00:00, 2287.29it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  40%|███▉      | 188/472 [00:00<00:00, 2297.84it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  40%|███▉      | 188/472 [00:00<00:00, 2296.86it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  40%|████      | 189/472 [00:00<00:00, 2307.47it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.bias]      
Loading weights:  40%|████      | 189/472 [00:00<00:00, 2306.48it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.bias]
Loading weights:  40%|████      | 190/472 [00:00<00:00, 2317.10it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.weight]
Loading weights:  40%|████      | 190/472 [00:00<00:00, 2316.11it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.weight]
Loading weights:  40%|████      | 191/472 [00:00<00:00, 2326.74it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.bias]      
Loading weights:  40%|████      | 191/472 [00:00<00:00, 2325.78it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.bias]
Loading weights:  41%|████      | 192/472 [00:00<00:00, 2336.39it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.weight]
Loading weights:  41%|████      | 192/472 [00:00<00:00, 2335.43it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.weight]
Loading weights:  41%|████      | 193/472 [00:00<00:00, 2345.98it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.bias]
Loading weights:  41%|████      | 193/472 [00:00<00:00, 2344.99it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.bias]
Loading weights:  41%|████      | 194/472 [00:00<00:00, 2355.57it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.weight]
Loading weights:  41%|████      | 194/472 [00:00<00:00, 2354.56it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.weight]
Loading weights:  41%|████▏     | 195/472 [00:00<00:00, 2365.15it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.bias]  
Loading weights:  41%|████▏     | 195/472 [00:00<00:00, 2364.18it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.bias]
Loading weights:  42%|████▏     | 196/472 [00:00<00:00, 2374.69it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.weight]
Loading weights:  42%|████▏     | 196/472 [00:00<00:00, 2373.69it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.weight]
Loading weights:  42%|████▏     | 197/472 [00:00<00:00, 2384.14it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  42%|████▏     | 197/472 [00:00<00:00, 2383.11it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  42%|████▏     | 198/472 [00:00<00:00, 2393.56it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  42%|████▏     | 198/472 [00:00<00:00, 2392.51it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  42%|████▏     | 199/472 [00:00<00:00, 2402.95it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.bias]      
Loading weights:  42%|████▏     | 199/472 [00:00<00:00, 2401.93it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.bias]
Loading weights:  42%|████▏     | 200/472 [00:00<00:00, 2412.37it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  42%|████▏     | 200/472 [00:00<00:00, 2411.36it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  43%|████▎     | 201/472 [00:00<00:00, 2421.77it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.bias]      
Loading weights:  43%|████▎     | 201/472 [00:00<00:00, 2420.75it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.bias]
Loading weights:  43%|████▎     | 202/472 [00:00<00:00, 2431.12it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  43%|████▎     | 202/472 [00:00<00:00, 2430.11it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  43%|████▎     | 203/472 [00:00<00:00, 2440.49it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  43%|████▎     | 203/472 [00:00<00:00, 2439.49it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  43%|████▎     | 204/472 [00:00<00:00, 2449.80it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  43%|████▎     | 204/472 [00:00<00:00, 2448.80it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  43%|████▎     | 205/472 [00:00<00:00, 2459.19it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.bias]  
Loading weights:  43%|████▎     | 205/472 [00:00<00:00, 2458.18it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.bias]
Loading weights:  44%|████▎     | 206/472 [00:00<00:00, 2468.48it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  44%|████▎     | 206/472 [00:00<00:00, 2467.48it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  44%|████▍     | 207/472 [00:00<00:00, 2477.79it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.bias]         
Loading weights:  44%|████▍     | 207/472 [00:00<00:00, 2476.79it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.bias]
Loading weights:  44%|████▍     | 208/472 [00:00<00:00, 2487.17it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.weight]
Loading weights:  44%|████▍     | 208/472 [00:00<00:00, 2486.17it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.weight]
Loading weights:  44%|████▍     | 209/472 [00:00<00:00, 2496.50it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.bias]    
Loading weights:  44%|████▍     | 209/472 [00:00<00:00, 2495.52it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.bias]
Loading weights:  44%|████▍     | 210/472 [00:00<00:00, 2505.79it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  44%|████▍     | 210/472 [00:00<00:00, 2504.80it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  45%|████▍     | 211/472 [00:00<00:00, 2515.01it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.bias]      
Loading weights:  45%|████▍     | 211/472 [00:00<00:00, 2514.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.bias]
Loading weights:  45%|████▍     | 212/472 [00:00<00:00, 2524.29it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.weight]
Loading weights:  45%|████▍     | 212/472 [00:00<00:00, 2523.26it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.weight]
Loading weights:  45%|████▌     | 213/472 [00:00<00:00, 2533.55it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  45%|████▌     | 213/472 [00:00<00:00, 2532.49it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  45%|████▌     | 214/472 [00:00<00:00, 2542.69it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  45%|████▌     | 214/472 [00:00<00:00, 2541.62it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  46%|████▌     | 215/472 [00:00<00:00, 2551.78it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.bias]      
Loading weights:  46%|████▌     | 215/472 [00:00<00:00, 2550.75it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.bias]
Loading weights:  46%|████▌     | 216/472 [00:00<00:00, 2560.91it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.weight]
Loading weights:  46%|████▌     | 216/472 [00:00<00:00, 2559.87it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.weight]
Loading weights:  46%|████▌     | 217/472 [00:00<00:00, 2570.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.bias]      
Loading weights:  46%|████▌     | 217/472 [00:00<00:00, 2568.99it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.bias]
Loading weights:  46%|████▌     | 218/472 [00:00<00:00, 2579.18it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.weight]
Loading weights:  46%|████▌     | 218/472 [00:00<00:00, 2578.14it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.weight]
Loading weights:  46%|████▋     | 219/472 [00:00<00:00, 2588.28it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.bias]
Loading weights:  46%|████▋     | 219/472 [00:00<00:00, 2587.24it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.bias]
Loading weights:  47%|████▋     | 220/472 [00:00<00:00, 2597.31it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.weight]
Loading weights:  47%|████▋     | 220/472 [00:00<00:00, 2594.76it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.weight]
Loading weights:  47%|████▋     | 221/472 [00:00<00:00, 2602.48it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.bias]  
Loading weights:  47%|████▋     | 221/472 [00:00<00:00, 2601.29it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.bias]
Loading weights:  47%|████▋     | 222/472 [00:00<00:00, 2611.06it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.weight]
Loading weights:  47%|████▋     | 222/472 [00:00<00:00, 2609.95it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.weight]
Loading weights:  47%|████▋     | 223/472 [00:00<00:00, 2619.83it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  47%|████▋     | 223/472 [00:00<00:00, 2618.72it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  47%|████▋     | 224/472 [00:00<00:00, 2628.46it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  47%|████▋     | 224/472 [00:00<00:00, 2627.41it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  48%|████▊     | 225/472 [00:00<00:00, 2637.25it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.bias]      
Loading weights:  48%|████▊     | 225/472 [00:00<00:00, 2634.87it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.bias]
Loading weights:  48%|████▊     | 226/472 [00:00<00:00, 2643.12it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  48%|████▊     | 226/472 [00:00<00:00, 2641.70it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  48%|████▊     | 227/472 [00:00<00:00, 2650.63it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.bias]      
Loading weights:  48%|████▊     | 227/472 [00:00<00:00, 2649.30it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.bias]
Loading weights:  48%|████▊     | 228/472 [00:00<00:00, 2658.39it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  48%|████▊     | 228/472 [00:00<00:00, 2656.29it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  49%|████▊     | 229/472 [00:00<00:00, 2665.39it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  49%|████▊     | 229/472 [00:00<00:00, 2664.18it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  49%|████▊     | 230/472 [00:00<00:00, 2673.39it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  49%|████▊     | 230/472 [00:00<00:00, 2672.24it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  49%|████▉     | 231/472 [00:00<00:00, 2681.90it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.bias]  
Loading weights:  49%|████▉     | 231/472 [00:00<00:00, 2680.81it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.bias]
Loading weights:  49%|████▉     | 232/472 [00:00<00:00, 2690.60it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  49%|████▉     | 232/472 [00:00<00:00, 2689.58it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  49%|████▉     | 233/472 [00:00<00:00, 2699.38it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.bias]         
Loading weights:  49%|████▉     | 233/472 [00:00<00:00, 2698.35it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.bias]
Loading weights:  50%|████▉     | 234/472 [00:00<00:00, 2708.18it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.weight]
Loading weights:  50%|████▉     | 234/472 [00:00<00:00, 2707.14it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.weight]
Loading weights:  50%|████▉     | 235/472 [00:00<00:00, 2716.98it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.bias]    
Loading weights:  50%|████▉     | 235/472 [00:00<00:00, 2715.95it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.bias]
Loading weights:  50%|█████     | 236/472 [00:00<00:00, 2725.77it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  50%|█████     | 236/472 [00:00<00:00, 2724.70it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  50%|█████     | 237/472 [00:00<00:00, 2734.51it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.bias]      
Loading weights:  50%|█████     | 237/472 [00:00<00:00, 2733.50it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.bias]
Loading weights:  50%|█████     | 238/472 [00:00<00:00, 2743.33it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.weight]
Loading weights:  50%|█████     | 238/472 [00:00<00:00, 2742.28it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.weight]
Loading weights:  51%|█████     | 239/472 [00:00<00:00, 2752.09it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  51%|█████     | 239/472 [00:00<00:00, 2751.02it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  51%|█████     | 240/472 [00:00<00:00, 2760.68it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  51%|█████     | 240/472 [00:00<00:00, 2759.57it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  51%|█████     | 241/472 [00:00<00:00, 2769.28it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.bias]      
Loading weights:  51%|█████     | 241/472 [00:00<00:00, 2768.24it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.bias]
Loading weights:  51%|█████▏    | 242/472 [00:00<00:00, 2777.94it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.weight]
Loading weights:  51%|█████▏    | 242/472 [00:00<00:00, 2776.92it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.weight]
Loading weights:  51%|█████▏    | 243/472 [00:00<00:00, 2786.60it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.bias]      
Loading weights:  51%|█████▏    | 243/472 [00:00<00:00, 2785.54it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.bias]
Loading weights:  52%|█████▏    | 244/472 [00:00<00:00, 2795.09it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.weight]
Loading weights:  52%|█████▏    | 244/472 [00:00<00:00, 2794.10it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.weight]
Loading weights:  52%|█████▏    | 245/472 [00:00<00:00, 2803.82it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.bias]
Loading weights:  52%|█████▏    | 245/472 [00:00<00:00, 2802.62it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.bias]
Loading weights:  52%|█████▏    | 246/472 [00:00<00:00, 2812.29it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.weight]
Loading weights:  52%|█████▏    | 246/472 [00:00<00:00, 2809.95it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.weight]
Loading weights:  52%|█████▏    | 247/472 [00:00<00:00, 2818.35it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.bias]  
Loading weights:  52%|█████▏    | 247/472 [00:00<00:00, 2817.10it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.bias]
Loading weights:  53%|█████▎    | 248/472 [00:00<00:00, 2826.40it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.weight]
Loading weights:  53%|█████▎    | 248/472 [00:00<00:00, 2825.24it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.weight]
Loading weights:  53%|█████▎    | 249/472 [00:00<00:00, 2834.57it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  53%|█████▎    | 249/472 [00:00<00:00, 2833.34it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  53%|█████▎    | 250/472 [00:00<00:00, 2842.52it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  53%|█████▎    | 250/472 [00:00<00:00, 2841.35it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  53%|█████▎    | 251/472 [00:00<00:00, 2850.71it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.bias]      
Loading weights:  53%|█████▎    | 251/472 [00:00<00:00, 2849.58it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.bias]
Loading weights:  53%|█████▎    | 252/472 [00:00<00:00, 2858.96it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  53%|█████▎    | 252/472 [00:00<00:00, 2857.76it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  54%|█████▎    | 253/472 [00:00<00:00, 2867.14it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.bias]      
Loading weights:  54%|█████▎    | 253/472 [00:00<00:00, 2866.01it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.bias]
Loading weights:  54%|█████▍    | 254/472 [00:00<00:00, 2875.29it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  54%|█████▍    | 254/472 [00:00<00:00, 2874.11it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  54%|█████▍    | 255/472 [00:00<00:00, 2883.50it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  54%|█████▍    | 255/472 [00:00<00:00, 2882.33it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  54%|█████▍    | 256/472 [00:00<00:00, 2891.77it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  54%|█████▍    | 256/472 [00:00<00:00, 2890.60it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  54%|█████▍    | 257/472 [00:00<00:00, 2899.99it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.bias]  
Loading weights:  54%|█████▍    | 257/472 [00:00<00:00, 2898.78it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.bias]
Loading weights:  55%|█████▍    | 258/472 [00:00<00:00, 2908.05it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  55%|█████▍    | 258/472 [00:00<00:00, 2906.91it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  55%|█████▍    | 259/472 [00:00<00:00, 2916.24it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.bias]         
Loading weights:  55%|█████▍    | 259/472 [00:00<00:00, 2915.06it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.bias]
Loading weights:  55%|█████▌    | 260/472 [00:00<00:00, 2924.37it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.weight]
Loading weights:  55%|█████▌    | 260/472 [00:00<00:00, 2923.22it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.weight]
Loading weights:  55%|█████▌    | 261/472 [00:00<00:00, 2932.71it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.bias]    
Loading weights:  55%|█████▌    | 261/472 [00:00<00:00, 2931.60it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.bias]
Loading weights:  56%|█████▌    | 262/472 [00:00<00:00, 2940.88it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  56%|█████▌    | 262/472 [00:00<00:00, 2939.76it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  56%|█████▌    | 263/472 [00:00<00:00, 2949.08it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.bias]      
Loading weights:  56%|█████▌    | 263/472 [00:00<00:00, 2947.97it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.bias]
Loading weights:  56%|█████▌    | 264/472 [00:00<00:00, 2957.36it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.weight]
Loading weights:  56%|█████▌    | 264/472 [00:00<00:00, 2956.12it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.weight]
Loading weights:  56%|█████▌    | 265/472 [00:00<00:00, 2965.56it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  56%|█████▌    | 265/472 [00:00<00:00, 2964.38it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  56%|█████▋    | 266/472 [00:00<00:00, 2973.63it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  56%|█████▋    | 266/472 [00:00<00:00, 2972.46it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  57%|█████▋    | 267/472 [00:00<00:00, 2981.74it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.bias]      
Loading weights:  57%|█████▋    | 267/472 [00:00<00:00, 2980.51it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.bias]
Loading weights:  57%|█████▋    | 268/472 [00:00<00:00, 2989.76it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.weight]
Loading weights:  57%|█████▋    | 268/472 [00:00<00:00, 2988.57it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.weight]
Loading weights:  57%|█████▋    | 269/472 [00:00<00:00, 2997.85it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.bias]      
Loading weights:  57%|█████▋    | 269/472 [00:00<00:00, 2996.68it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.bias]
Loading weights:  57%|█████▋    | 270/472 [00:00<00:00, 3005.98it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.weight]
Loading weights:  57%|█████▋    | 270/472 [00:00<00:00, 3004.84it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.weight]
Loading weights:  57%|█████▋    | 271/472 [00:00<00:00, 3014.02it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.bias]
Loading weights:  57%|█████▋    | 271/472 [00:00<00:00, 3012.88it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.bias]
Loading weights:  58%|█████▊    | 272/472 [00:00<00:00, 3022.12it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.weight]
Loading weights:  58%|█████▊    | 272/472 [00:00<00:00, 3020.88it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.weight]
Loading weights:  58%|█████▊    | 273/472 [00:00<00:00, 3030.10it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.bias]  
Loading weights:  58%|█████▊    | 273/472 [00:00<00:00, 3028.83it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.bias]
Loading weights:  58%|█████▊    | 274/472 [00:00<00:00, 3037.99it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.weight]
Loading weights:  58%|█████▊    | 274/472 [00:00<00:00, 3036.72it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.weight]
Loading weights:  58%|█████▊    | 275/472 [00:00<00:00, 3045.84it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  58%|█████▊    | 275/472 [00:00<00:00, 3044.63it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  58%|█████▊    | 276/472 [00:00<00:00, 3053.70it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  58%|█████▊    | 276/472 [00:00<00:00, 3052.49it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  59%|█████▊    | 277/472 [00:00<00:00, 3061.55it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.bias]      
Loading weights:  59%|█████▊    | 277/472 [00:00<00:00, 3060.40it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.bias]
Loading weights:  59%|█████▉    | 278/472 [00:00<00:00, 3069.52it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  59%|█████▉    | 278/472 [00:00<00:00, 3068.29it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  59%|█████▉    | 279/472 [00:00<00:00, 3077.36it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.bias]      
Loading weights:  59%|█████▉    | 279/472 [00:00<00:00, 3076.22it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.bias]
Loading weights:  59%|█████▉    | 280/472 [00:00<00:00, 3085.29it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  59%|█████▉    | 280/472 [00:00<00:00, 3084.14it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  60%|█████▉    | 281/472 [00:00<00:00, 3093.24it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  60%|█████▉    | 281/472 [00:00<00:00, 3092.10it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  60%|█████▉    | 282/472 [00:00<00:00, 3101.18it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  60%|█████▉    | 282/472 [00:00<00:00, 3099.95it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  60%|█████▉    | 283/472 [00:00<00:00, 3109.06it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.bias]  
Loading weights:  60%|█████▉    | 283/472 [00:00<00:00, 3107.84it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.bias]
Loading weights:  60%|██████    | 284/472 [00:00<00:00, 3116.83it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  60%|██████    | 284/472 [00:00<00:00, 3115.54it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  60%|██████    | 285/472 [00:00<00:00, 3124.51it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.bias]         
Loading weights:  60%|██████    | 285/472 [00:00<00:00, 3123.31it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.bias]
Loading weights:  61%|██████    | 286/472 [00:00<00:00, 3132.36it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.weight]
Loading weights:  61%|██████    | 286/472 [00:00<00:00, 3131.20it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.weight]
Loading weights:  61%|██████    | 287/472 [00:00<00:00, 3140.21it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.bias]    
Loading weights:  61%|██████    | 287/472 [00:00<00:00, 3139.01it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.bias]
Loading weights:  61%|██████    | 288/472 [00:00<00:00, 3148.09it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  61%|██████    | 288/472 [00:00<00:00, 3146.96it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  61%|██████    | 289/472 [00:00<00:00, 3155.95it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.bias]      
Loading weights:  61%|██████    | 289/472 [00:00<00:00, 3154.82it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.bias]
Loading weights:  61%|██████▏   | 290/472 [00:00<00:00, 3163.76it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.weight]
Loading weights:  61%|██████▏   | 290/472 [00:00<00:00, 3162.58it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.weight]
Loading weights:  62%|██████▏   | 291/472 [00:00<00:00, 3171.66it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  62%|██████▏   | 291/472 [00:00<00:00, 3170.45it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  62%|██████▏   | 292/472 [00:00<00:00, 3179.30it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  62%|██████▏   | 292/472 [00:00<00:00, 3178.09it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  62%|██████▏   | 293/472 [00:00<00:00, 3186.96it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.bias]      
Loading weights:  62%|██████▏   | 293/472 [00:00<00:00, 3185.68it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.bias]
Loading weights:  62%|██████▏   | 294/472 [00:00<00:00, 3194.54it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.weight]
Loading weights:  62%|██████▏   | 294/472 [00:00<00:00, 3193.33it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.weight]
Loading weights:  62%|██████▎   | 295/472 [00:00<00:00, 3202.20it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.bias]      
Loading weights:  62%|██████▎   | 295/472 [00:00<00:00, 3200.92it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.bias]
Loading weights:  63%|██████▎   | 296/472 [00:00<00:00, 3209.75it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.weight]
Loading weights:  63%|██████▎   | 296/472 [00:00<00:00, 3208.57it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.weight]
Loading weights:  63%|██████▎   | 297/472 [00:00<00:00, 3217.42it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.bias]
Loading weights:  63%|██████▎   | 297/472 [00:00<00:00, 3216.16it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.bias]
Loading weights:  63%|██████▎   | 298/472 [00:00<00:00, 3225.03it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.weight]
Loading weights:  63%|██████▎   | 298/472 [00:00<00:00, 3223.82it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.weight]
Loading weights:  63%|██████▎   | 299/472 [00:00<00:00, 3232.71it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.bias]  
Loading weights:  63%|██████▎   | 299/472 [00:00<00:00, 3231.45it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.bias]
Loading weights:  64%|██████▎   | 300/472 [00:00<00:00, 3240.33it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.weight]
Loading weights:  64%|██████▎   | 300/472 [00:00<00:00, 3239.19it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.weight]
Loading weights:  64%|██████▍   | 301/472 [00:00<00:00, 3247.94it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  64%|██████▍   | 301/472 [00:00<00:00, 3246.64it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  64%|██████▍   | 302/472 [00:00<00:00, 3255.40it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  64%|██████▍   | 302/472 [00:00<00:00, 3254.21it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  64%|██████▍   | 303/472 [00:00<00:00, 3262.97it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.bias]      
Loading weights:  64%|██████▍   | 303/472 [00:00<00:00, 3261.75it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.bias]
Loading weights:  64%|██████▍   | 304/472 [00:00<00:00, 3270.54it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  64%|██████▍   | 304/472 [00:00<00:00, 3269.27it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  65%|██████▍   | 305/472 [00:00<00:00, 3278.06it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.bias]      
Loading weights:  65%|██████▍   | 305/472 [00:00<00:00, 3276.78it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.bias]
Loading weights:  65%|██████▍   | 306/472 [00:00<00:00, 3285.45it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  65%|██████▍   | 306/472 [00:00<00:00, 3284.25it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  65%|██████▌   | 307/472 [00:00<00:00, 3292.93it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  65%|██████▌   | 307/472 [00:00<00:00, 3291.69it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  65%|██████▌   | 308/472 [00:00<00:00, 3300.44it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  65%|██████▌   | 308/472 [00:00<00:00, 3299.16it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  65%|██████▌   | 309/472 [00:00<00:00, 3307.92it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.bias]  
Loading weights:  65%|██████▌   | 309/472 [00:00<00:00, 3306.65it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.bias]
Loading weights:  66%|██████▌   | 310/472 [00:00<00:00, 3315.29it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  66%|██████▌   | 310/472 [00:00<00:00, 3314.05it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  66%|██████▌   | 311/472 [00:00<00:00, 3322.72it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.bias]         
Loading weights:  66%|██████▌   | 311/472 [00:00<00:00, 3321.44it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.bias]
Loading weights:  66%|██████▌   | 312/472 [00:00<00:00, 3330.17it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.weight]
Loading weights:  66%|██████▌   | 312/472 [00:00<00:00, 3328.96it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.weight]
Loading weights:  66%|██████▋   | 313/472 [00:00<00:00, 3337.67it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.bias]    
Loading weights:  66%|██████▋   | 313/472 [00:00<00:00, 3336.43it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.bias]
Loading weights:  67%|██████▋   | 314/472 [00:00<00:00, 3345.09it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  67%|██████▋   | 314/472 [00:00<00:00, 3343.84it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  67%|██████▋   | 315/472 [00:00<00:00, 3352.56it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.bias]      
Loading weights:  67%|██████▋   | 315/472 [00:00<00:00, 3351.32it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.bias]
Loading weights:  67%|██████▋   | 316/472 [00:00<00:00, 3360.02it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.weight]
Loading weights:  67%|██████▋   | 316/472 [00:00<00:00, 3358.77it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.weight]
Loading weights:  67%|██████▋   | 317/472 [00:00<00:00, 3367.39it/s, Materializing param=text_decoder.cls.predictions.bias]                     
Loading weights:  67%|██████▋   | 317/472 [00:00<00:00, 3365.64it/s, Materializing param=text_decoder.cls.predictions.bias]
Loading weights:  67%|██████▋   | 318/472 [00:00<00:00, 3374.29it/s, Materializing param=text_decoder.cls.predictions.decoder.weight]
Loading weights:  67%|██████▋   | 318/472 [00:00<00:00, 3372.97it/s, Materializing param=text_decoder.cls.predictions.decoder.weight]
Loading weights:  68%|██████▊   | 319/472 [00:00<00:00, 3381.74it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.bias]
Loading weights:  68%|██████▊   | 319/472 [00:00<00:00, 3380.52it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.bias]
Loading weights:  68%|██████▊   | 320/472 [00:00<00:00, 3389.19it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.weight]
Loading weights:  68%|██████▊   | 320/472 [00:00<00:00, 3387.97it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.weight]
Loading weights:  68%|██████▊   | 321/472 [00:00<00:00, 3396.61it/s, Materializing param=text_decoder.cls.predictions.transform.dense.bias]      
Loading weights:  68%|██████▊   | 321/472 [00:00<00:00, 3395.24it/s, Materializing param=text_decoder.cls.predictions.transform.dense.bias]
Loading weights:  68%|██████▊   | 322/472 [00:00<00:00, 3403.99it/s, Materializing param=text_decoder.cls.predictions.transform.dense.weight]
Loading weights:  68%|██████▊   | 322/472 [00:00<00:00, 3402.76it/s, Materializing param=text_decoder.cls.predictions.transform.dense.weight]
Loading weights:  68%|██████▊   | 323/472 [00:00<00:00, 3411.49it/s, Materializing param=vision_model.embeddings.class_embedding]            
Loading weights:  68%|██████▊   | 323/472 [00:00<00:00, 3410.30it/s, Materializing param=vision_model.embeddings.class_embedding]
Loading weights:  69%|██████▊   | 324/472 [00:00<00:00, 3417.32it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  69%|██████▊   | 324/472 [00:00<00:00, 3414.37it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  69%|██████▉   | 325/472 [00:00<00:00, 3421.02it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  69%|██████▉   | 325/472 [00:00<00:00, 3419.51it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  69%|██████▉   | 326/472 [00:00<00:00, 3426.93it/s, Materializing param=vision_model.embeddings.position_embedding]    
Loading weights:  69%|██████▉   | 326/472 [00:00<00:00, 3424.87it/s, Materializing param=vision_model.embeddings.position_embedding]
Loading weights:  69%|██████▉   | 327/472 [00:00<00:00, 3432.57it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  69%|██████▉   | 327/472 [00:00<00:00, 3431.09it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  69%|██████▉   | 328/472 [00:00<00:00, 3439.05it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  69%|██████▉   | 328/472 [00:00<00:00, 3437.79it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  70%|██████▉   | 329/472 [00:00<00:00, 3446.14it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:  70%|██████▉   | 329/472 [00:00<00:00, 3444.81it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]
Loading weights:  70%|██████▉   | 330/472 [00:00<00:00, 3453.22it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  70%|██████▉   | 330/472 [00:00<00:00, 3451.96it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  70%|███████   | 331/472 [00:00<00:00, 3460.39it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:  70%|███████   | 331/472 [00:00<00:00, 3459.12it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:  70%|███████   | 332/472 [00:00<00:00, 3467.54it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  70%|███████   | 332/472 [00:00<00:00, 3466.24it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  71%|███████   | 333/472 [00:00<00:00, 3474.65it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:  71%|███████   | 333/472 [00:00<00:00, 3473.45it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:  71%|███████   | 334/472 [00:00<00:00, 3481.93it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  71%|███████   | 334/472 [00:00<00:00, 3480.77it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  71%|███████   | 335/472 [00:00<00:00, 3489.22it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.bias]
Loading weights:  71%|███████   | 335/472 [00:00<00:00, 3487.95it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.bias]
Loading weights:  71%|███████   | 336/472 [00:00<00:00, 3496.28it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.weight]
Loading weights:  71%|███████   | 336/472 [00:00<00:00, 3495.05it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.weight]
Loading weights:  71%|███████▏  | 337/472 [00:00<00:00, 3503.48it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.bias]         
Loading weights:  71%|███████▏  | 337/472 [00:00<00:00, 3502.24it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.bias]
Loading weights:  72%|███████▏  | 338/472 [00:00<00:00, 3510.67it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.weight]
Loading weights:  72%|███████▏  | 338/472 [00:00<00:00, 3509.39it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.weight]
Loading weights:  72%|███████▏  | 339/472 [00:00<00:00, 3517.91it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]    
Loading weights:  72%|███████▏  | 339/472 [00:00<00:00, 3516.67it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]
Loading weights:  72%|███████▏  | 340/472 [00:00<00:00, 3525.14it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  72%|███████▏  | 340/472 [00:00<00:00, 3523.94it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  72%|███████▏  | 341/472 [00:00<00:00, 3532.40it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]  
Loading weights:  72%|███████▏  | 341/472 [00:00<00:00, 3531.23it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]
Loading weights:  72%|███████▏  | 342/472 [00:00<00:00, 3539.64it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  72%|███████▏  | 342/472 [00:00<00:00, 3538.47it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  73%|███████▎  | 343/472 [00:00<00:00, 3546.94it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:  73%|███████▎  | 343/472 [00:00<00:00, 3545.77it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:  73%|███████▎  | 344/472 [00:00<00:00, 3554.05it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  73%|███████▎  | 344/472 [00:00<00:00, 3552.80it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  73%|███████▎  | 345/472 [00:00<00:00, 3561.18it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:  73%|███████▎  | 345/472 [00:00<00:00, 3559.93it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:  73%|███████▎  | 346/472 [00:00<00:00, 3568.30it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  73%|███████▎  | 346/472 [00:00<00:00, 3567.08it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  74%|███████▎  | 347/472 [00:00<00:00, 3575.51it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.bias]
Loading weights:  74%|███████▎  | 347/472 [00:00<00:00, 3574.33it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.bias]
Loading weights:  74%|███████▎  | 348/472 [00:00<00:00, 3582.61it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.weight]
Loading weights:  74%|███████▎  | 348/472 [00:00<00:00, 3581.42it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.weight]
Loading weights:  74%|███████▍  | 349/472 [00:00<00:00, 3589.79it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.bias]         
Loading weights:  74%|███████▍  | 349/472 [00:00<00:00, 3588.65it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.bias]
Loading weights:  74%|███████▍  | 350/472 [00:00<00:00, 3597.05it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.weight]
Loading weights:  74%|███████▍  | 350/472 [00:00<00:00, 3595.90it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.weight]
Loading weights:  74%|███████▍  | 351/472 [00:00<00:00, 3604.10it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]    
Loading weights:  74%|███████▍  | 351/472 [00:00<00:00, 3602.88it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]
Loading weights:  75%|███████▍  | 352/472 [00:00<00:00, 3611.26it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  75%|███████▍  | 352/472 [00:00<00:00, 3610.11it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  75%|███████▍  | 353/472 [00:00<00:00, 3618.54it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  75%|███████▍  | 353/472 [00:00<00:00, 3617.35it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  75%|███████▌  | 354/472 [00:00<00:00, 3625.75it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  75%|███████▌  | 354/472 [00:00<00:00, 3624.60it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  75%|███████▌  | 355/472 [00:00<00:00, 3632.94it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  75%|███████▌  | 355/472 [00:00<00:00, 3631.79it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  75%|███████▌  | 356/472 [00:00<00:00, 3640.12it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  75%|███████▌  | 356/472 [00:00<00:00, 3638.96it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  76%|███████▌  | 357/472 [00:00<00:00, 3647.28it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  76%|███████▌  | 357/472 [00:00<00:00, 3646.06it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  76%|███████▌  | 358/472 [00:00<00:00, 3654.33it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  76%|███████▌  | 358/472 [00:00<00:00, 3653.09it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  76%|███████▌  | 359/472 [00:00<00:00, 3661.44it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.bias]
Loading weights:  76%|███████▌  | 359/472 [00:00<00:00, 3660.24it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.bias]
Loading weights:  76%|███████▋  | 360/472 [00:00<00:00, 3668.42it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.weight]
Loading weights:  76%|███████▋  | 360/472 [00:00<00:00, 3667.18it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.weight]
Loading weights:  76%|███████▋  | 361/472 [00:00<00:00, 3675.42it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.bias]         
Loading weights:  76%|███████▋  | 361/472 [00:00<00:00, 3674.15it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.bias]
Loading weights:  77%|███████▋  | 362/472 [00:00<00:00, 3682.42it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.weight]
Loading weights:  77%|███████▋  | 362/472 [00:00<00:00, 3681.18it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.weight]
Loading weights:  77%|███████▋  | 363/472 [00:00<00:00, 3688.20it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]    
Loading weights:  77%|███████▋  | 363/472 [00:00<00:00, 3686.33it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  77%|███████▋  | 364/472 [00:00<00:00, 3693.93it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  77%|███████▋  | 364/472 [00:00<00:00, 3692.54it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  77%|███████▋  | 365/472 [00:00<00:00, 3700.55it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  77%|███████▋  | 365/472 [00:00<00:00, 3699.15it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  78%|███████▊  | 366/472 [00:00<00:00, 3707.18it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  78%|███████▊  | 366/472 [00:00<00:00, 3705.84it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  78%|███████▊  | 367/472 [00:00<00:00, 3713.98it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  78%|███████▊  | 367/472 [00:00<00:00, 3712.73it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  78%|███████▊  | 368/472 [00:00<00:00, 3720.70it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 368/472 [00:00<00:00, 3719.13it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  78%|███████▊  | 369/472 [00:00<00:00, 3726.86it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  78%|███████▊  | 369/472 [00:00<00:00, 3724.86it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  78%|███████▊  | 370/472 [00:00<00:00, 3731.83it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  78%|███████▊  | 370/472 [00:00<00:00, 3730.44it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  79%|███████▊  | 371/472 [00:00<00:00, 3738.22it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.bias]
Loading weights:  79%|███████▊  | 371/472 [00:00<00:00, 3736.98it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.bias]
Loading weights:  79%|███████▉  | 372/472 [00:00<00:00, 3744.87it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.weight]
Loading weights:  79%|███████▉  | 372/472 [00:00<00:00, 3743.62it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.weight]
Loading weights:  79%|███████▉  | 373/472 [00:00<00:00, 3750.97it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.bias]         
Loading weights:  79%|███████▉  | 373/472 [00:00<00:00, 3749.34it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.bias]
Loading weights:  79%|███████▉  | 374/472 [00:00<00:00, 3754.60it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.weight]
Loading weights:  79%|███████▉  | 374/472 [00:00<00:00, 3752.27it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.weight]
Loading weights:  79%|███████▉  | 375/472 [00:00<00:00, 3759.33it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]    
Loading weights:  79%|███████▉  | 375/472 [00:00<00:00, 3757.82it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  80%|███████▉  | 376/472 [00:00<00:00, 3765.42it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  80%|███████▉  | 376/472 [00:00<00:00, 3764.02it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  80%|███████▉  | 377/472 [00:00<00:00, 3771.85it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  80%|███████▉  | 377/472 [00:00<00:00, 3770.49it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  80%|████████  | 378/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  80%|████████  | 378/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  80%|████████  | 378/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  80%|████████  | 379/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  80%|████████  | 379/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  81%|████████  | 380/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  81%|████████  | 380/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  81%|████████  | 381/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  81%|████████  | 381/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  81%|████████  | 382/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  81%|████████  | 382/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  81%|████████  | 383/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.bias]
Loading weights:  81%|████████  | 383/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.bias]
Loading weights:  81%|████████▏ | 384/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.weight]
Loading weights:  81%|████████▏ | 384/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.weight]
Loading weights:  82%|████████▏ | 385/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.bias]         
Loading weights:  82%|████████▏ | 385/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.bias]
Loading weights:  82%|████████▏ | 386/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.weight]
Loading weights:  82%|████████▏ | 386/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.weight]
Loading weights:  82%|████████▏ | 387/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]    
Loading weights:  82%|████████▏ | 387/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  82%|████████▏ | 388/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  82%|████████▏ | 388/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  82%|████████▏ | 389/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  82%|████████▏ | 389/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  83%|████████▎ | 390/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  83%|████████▎ | 390/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  83%|████████▎ | 391/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  83%|████████▎ | 391/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  83%|████████▎ | 392/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 392/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  83%|████████▎ | 393/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  83%|████████▎ | 393/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  83%|████████▎ | 394/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 394/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  84%|████████▎ | 395/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.bias]
Loading weights:  84%|████████▎ | 395/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.bias]
Loading weights:  84%|████████▍ | 396/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.weight]
Loading weights:  84%|████████▍ | 396/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.weight]
Loading weights:  84%|████████▍ | 397/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.bias]         
Loading weights:  84%|████████▍ | 397/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.bias]
Loading weights:  84%|████████▍ | 398/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.weight]
Loading weights:  84%|████████▍ | 398/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.weight]
Loading weights:  85%|████████▍ | 399/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]    
Loading weights:  85%|████████▍ | 399/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  85%|████████▍ | 400/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  85%|████████▍ | 400/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  85%|████████▍ | 401/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  85%|████████▍ | 401/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  85%|████████▌ | 402/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  85%|████████▌ | 402/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  85%|████████▌ | 403/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  85%|████████▌ | 403/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  86%|████████▌ | 404/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 404/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  86%|████████▌ | 405/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  86%|████████▌ | 405/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  86%|████████▌ | 406/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  86%|████████▌ | 406/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  86%|████████▌ | 407/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.bias]
Loading weights:  86%|████████▌ | 407/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.bias]
Loading weights:  86%|████████▋ | 408/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.weight]
Loading weights:  86%|████████▋ | 408/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.weight]
Loading weights:  87%|████████▋ | 409/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.bias]         
Loading weights:  87%|████████▋ | 409/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.bias]
Loading weights:  87%|████████▋ | 410/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.weight]
Loading weights:  87%|████████▋ | 410/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.weight]
Loading weights:  87%|████████▋ | 411/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]    
Loading weights:  87%|████████▋ | 411/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  87%|████████▋ | 412/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  87%|████████▋ | 412/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  88%|████████▊ | 413/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  88%|████████▊ | 413/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  88%|████████▊ | 414/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  88%|████████▊ | 414/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  88%|████████▊ | 415/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  88%|████████▊ | 415/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  88%|████████▊ | 416/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  88%|████████▊ | 416/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  88%|████████▊ | 417/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  88%|████████▊ | 417/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  89%|████████▊ | 418/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  89%|████████▊ | 418/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  89%|████████▉ | 419/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.bias]
Loading weights:  89%|████████▉ | 419/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.bias]
Loading weights:  89%|████████▉ | 420/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.weight]
Loading weights:  89%|████████▉ | 420/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.weight]
Loading weights:  89%|████████▉ | 421/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.bias]         
Loading weights:  89%|████████▉ | 421/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.bias]
Loading weights:  89%|████████▉ | 422/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.weight]
Loading weights:  89%|████████▉ | 422/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.weight]
Loading weights:  90%|████████▉ | 423/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]    
Loading weights:  90%|████████▉ | 423/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  90%|████████▉ | 424/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  90%|████████▉ | 424/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  90%|█████████ | 425/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  90%|█████████ | 425/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  90%|█████████ | 426/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  90%|█████████ | 426/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  90%|█████████ | 427/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  90%|█████████ | 427/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  91%|█████████ | 428/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  91%|█████████ | 428/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  91%|█████████ | 429/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  91%|█████████ | 429/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  91%|█████████ | 430/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  91%|█████████ | 430/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  91%|█████████▏| 431/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.bias]
Loading weights:  91%|█████████▏| 431/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.bias]
Loading weights:  92%|█████████▏| 432/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.weight]
Loading weights:  92%|█████████▏| 432/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.weight]
Loading weights:  92%|█████████▏| 433/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.bias]         
Loading weights:  92%|█████████▏| 433/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.bias]
Loading weights:  92%|█████████▏| 434/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.weight]
Loading weights:  92%|█████████▏| 434/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.weight]
Loading weights:  92%|█████████▏| 435/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]    
Loading weights:  92%|█████████▏| 435/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  92%|█████████▏| 436/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  92%|█████████▏| 436/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  93%|█████████▎| 437/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  93%|█████████▎| 437/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  93%|█████████▎| 438/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  93%|█████████▎| 438/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  93%|█████████▎| 439/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  93%|█████████▎| 439/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  93%|█████████▎| 440/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  93%|█████████▎| 440/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  93%|█████████▎| 441/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  93%|█████████▎| 441/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  94%|█████████▎| 442/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  94%|█████████▎| 442/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  94%|█████████▍| 443/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.bias]
Loading weights:  94%|█████████▍| 443/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.bias]
Loading weights:  94%|█████████▍| 444/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.weight]
Loading weights:  94%|█████████▍| 444/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.weight]
Loading weights:  94%|█████████▍| 445/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.bias]         
Loading weights:  94%|█████████▍| 445/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.bias]
Loading weights:  94%|█████████▍| 446/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.weight]
Loading weights:  94%|█████████▍| 446/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.weight]
Loading weights:  95%|█████████▍| 447/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]   
Loading weights:  95%|█████████▍| 447/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  95%|█████████▍| 448/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  95%|█████████▍| 448/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  95%|█████████▌| 449/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  95%|█████████▌| 449/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  95%|█████████▌| 450/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  95%|█████████▌| 450/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  96%|█████████▌| 451/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  96%|█████████▌| 451/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  96%|█████████▌| 452/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  96%|█████████▌| 452/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  96%|█████���███▌| 453/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  96%|█████████▌| 453/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  96%|█████████▌| 454/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  96%|█████████▌| 454/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  96%|█████████▋| 455/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.bias]
Loading weights:  96%|█████████▋| 455/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.bias]
Loading weights:  97%|█████████▋| 456/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.weight]
Loading weights:  97%|█████████▋| 456/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.weight]
Loading weights:  97%|█████████▋| 457/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.bias]         
Loading weights:  97%|█████████▋| 457/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.bias]
Loading weights:  97%|█████████▋| 458/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.weight]
Loading weights:  97%|█████████▋| 458/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.weight]
Loading weights:  97%|█████████▋| 459/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]    
Loading weights:  97%|█████████▋| 459/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  97%|█████████▋| 460/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  97%|█████████▋| 460/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  98%|█████████▊| 461/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  98%|█████████▊| 461/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  98%|█████████▊| 462/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  98%|█████████▊| 462/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  98%|█████████▊| 463/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  98%|█████████▊| 463/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  98%|█████████▊| 464/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 464/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  99%|█████████▊| 465/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  99%|█████████▊| 465/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  99%|█████████▊| 466/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  99%|█████████▊| 466/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 467/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.bias]
Loading weights:  99%|█████████▉| 467/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.bias]
Loading weights:  99%|█████████▉| 468/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.weight]
Loading weights:  99%|█████████▉| 468/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.weight]
Loading weights:  99%|█████████▉| 469/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.bias]         
Loading weights:  99%|█████████▉| 469/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.bias]
Loading weights: 100%|█████████▉| 470/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.weight]
Loading weights: 100%|█████████▉| 470/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.weight]
Loading weights: 100%|█████████▉| 471/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.post_layernorm.bias]                   
Loading weights: 100%|█████████▉| 471/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|██████████| 472/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 472/472 [00:00<00:00, 3778.56it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 472/472 [00:00<00:00, 4352.19it/s, Materializing param=vision_model.post_layernorm.weight]
+The tied weights mapping and config for this model specifies to tie text_decoder.bert.embeddings.word_embeddings.weight to text_decoder.cls.predictions.decoder.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
+BlipForConditionalGeneration LOAD REPORT from: Salesforce/blip-image-captioning-base
+Key                                       | Status     |  | 
+------------------------------------------+------------+--+-
+text_decoder.bert.embeddings.position_ids | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+
Loading weights:   0%|          | 0/788 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/788 [00:00<00:00, 40329.85it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.bias]
Loading weights:   0%|          | 1/788 [00:00<00:00, 2178.86it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.bias] 
Loading weights:   0%|          | 2/788 [00:00<00:00, 2130.17it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.weight]
Loading weights:   0%|          | 2/788 [00:00<00:00, 1940.01it/s, Materializing param=text_decoder.bert.embeddings.LayerNorm.weight]
Loading weights:   0%|          | 3/788 [00:00<00:00, 1844.19it/s, Materializing param=text_decoder.bert.embeddings.position_embeddings.weight]
Loading weights:   0%|          | 3/788 [00:00<00:00, 1668.60it/s, Materializing param=text_decoder.bert.embeddings.position_embeddings.weight]
Loading weights:   1%|          | 4/788 [00:00<00:00, 1763.05it/s, Materializing param=text_decoder.bert.embeddings.word_embeddings.weight]    
Loading weights:   1%|          | 4/788 [00:00<00:00, 1285.02it/s, Materializing param=text_decoder.bert.embeddings.word_embeddings.weight]
Loading weights:   1%|          | 5/788 [00:00<00:00, 1290.40it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   1%|          | 5/788 [00:00<00:00, 1262.36it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   1%|          | 6/788 [00:00<00:00, 1366.82it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   1%|          | 6/788 [00:00<00:00, 1198.37it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   1%|          | 7/788 [00:00<00:00, 1104.30it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.bias]      
Loading weights:   1%|          | 7/788 [00:00<00:00, 1091.70it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.bias]
Loading weights:   1%|          | 8/788 [00:00<00:00, 1105.62it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.weight]
Loading weights:   1%|          | 8/788 [00:00<00:00, 1063.97it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.output.dense.weight]
Loading weights:   1%|          | 9/788 [00:00<00:00, 1062.09it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.bias]      
Loading weights:   1%|          | 9/788 [00:00<00:00, 1016.72it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.bias]
Loading weights:   1%|▏         | 10/788 [00:00<00:00, 1111.99it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.weight]
Loading weights:   1%|▏         | 10/788 [00:00<00:00, 1074.36it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.key.weight]
Loading weights:   1%|▏         | 11/788 [00:00<00:00, 1126.15it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.bias]
Loading weights:   1%|▏         | 11/788 [00:00<00:00, 1107.77it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.bias]
Loading weights:   2%|▏         | 12/788 [00:00<00:00, 1117.12it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.weight]
Loading weights:   2%|▏         | 12/788 [00:00<00:00, 1104.98it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.query.weight]
Loading weights:   2%|▏         | 13/788 [00:00<00:00, 1098.91it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.bias]  
Loading weights:   2%|▏         | 13/788 [00:00<00:00, 1091.06it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.bias]
Loading weights:   2%|▏         | 14/788 [00:00<00:00, 1130.50it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.weight]
Loading weights:   2%|▏         | 14/788 [00:00<00:00, 1121.45it/s, Materializing param=text_decoder.bert.encoder.layer.0.attention.self.value.weight]
Loading weights:   2%|▏         | 15/788 [00:00<00:00, 1113.20it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:   2%|▏         | 15/788 [00:00<00:00, 1088.69it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:   2%|▏         | 16/788 [00:00<00:00, 1081.25it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:   2%|▏         | 16/788 [00:00<00:00, 1050.56it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:   2%|▏         | 17/788 [00:00<00:00, 1081.15it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.bias]      
Loading weights:   2%|▏         | 17/788 [00:00<00:00, 1076.70it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.bias]
Loading weights:   2%|▏         | 18/788 [00:00<00:00, 1092.24it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:   2%|▏         | 18/788 [00:00<00:00, 1082.71it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:   2%|▏         | 19/788 [00:00<00:00, 1032.56it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.bias]      
Loading weights:   2%|▏         | 19/788 [00:00<00:00, 1020.80it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.bias]
Loading weights:   3%|▎         | 20/788 [00:00<00:00, 1027.70it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.weight]
Loading weights:   3%|▎         | 20/788 [00:00<00:00, 1006.76it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.key.weight]
Loading weights:   3%|▎         | 21/788 [00:00<00:00, 1031.65it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.bias]
Loading weights:   3%|▎         | 21/788 [00:00<00:00, 1028.60it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.bias]
Loading weights:   3%|▎         | 22/788 [00:00<00:00, 1032.28it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.weight]
Loading weights:   3%|▎         | 22/788 [00:00<00:00, 1027.56it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.query.weight]
Loading weights:   3%|▎         | 23/788 [00:00<00:00, 1027.84it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.bias]  
Loading weights:   3%|▎         | 23/788 [00:00<00:00, 1008.02it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.bias]
Loading weights:   3%|▎         | 24/788 [00:00<00:00, 1011.04it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.weight]
Loading weights:   3%|▎         | 24/788 [00:00<00:00, 1008.62it/s, Materializing param=text_decoder.bert.encoder.layer.0.crossattention.self.value.weight]
Loading weights:   3%|▎         | 25/788 [00:00<00:00, 1032.34it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.bias]         
Loading weights:   3%|▎         | 25/788 [00:00<00:00, 1028.93it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.bias]
Loading weights:   3%|▎         | 26/788 [00:00<00:00, 1019.93it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.weight]
Loading weights:   3%|▎         | 26/788 [00:00<00:00, 995.25it/s, Materializing param=text_decoder.bert.encoder.layer.0.intermediate.dense.weight] 
Loading weights:   3%|▎         | 27/788 [00:00<00:00, 1012.38it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.bias]   
Loading weights:   3%|▎         | 27/788 [00:00<00:00, 1008.63it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.bias]
Loading weights:   4%|▎         | 28/788 [00:00<00:00, 1012.44it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.weight]
Loading weights:   4%|▎         | 28/788 [00:00<00:00, 998.90it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.LayerNorm.weight] 
Loading weights:   4%|▎         | 29/788 [00:00<00:00, 1005.80it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.bias]     
Loading weights:   4%|▎         | 29/788 [00:00<00:00, 993.23it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.bias] 
Loading weights:   4%|▍         | 30/788 [00:00<00:00, 1004.29it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.weight]
Loading weights:   4%|▍         | 30/788 [00:00<00:00, 979.69it/s, Materializing param=text_decoder.bert.encoder.layer.0.output.dense.weight] 
Loading weights:   4%|▍         | 31/788 [00:00<00:00, 999.20it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:   4%|▍         | 31/788 [00:00<00:00, 997.11it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:   4%|▍         | 32/788 [00:00<00:00, 1000.38it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:   4%|▍         | 32/788 [00:00<00:00, 991.21it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.LayerNorm.weight] 
Loading weights:   4%|▍         | 33/788 [00:00<00:00, 1002.19it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.bias]     
Loading weights:   4%|▍         | 33/788 [00:00<00:00, 989.68it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.bias] 
Loading weights:   4%|▍         | 34/788 [00:00<00:00, 1015.32it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.weight]
Loading weights:   4%|▍         | 34/788 [00:00<00:00, 1005.69it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.output.dense.weight]
Loading weights:   4%|▍         | 35/788 [00:00<00:00, 1009.92it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.bias]      
Loading weights:   4%|▍         | 35/788 [00:00<00:00, 997.50it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.bias] 
Loading weights:   5%|▍         | 36/788 [00:00<00:00, 999.67it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.weight]
Loading weights:   5%|▍         | 36/788 [00:00<00:00, 987.93it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.key.weight]
Loading weights:   5%|▍         | 37/788 [00:00<00:00, 1003.31it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.bias]
Loading weights:   5%|▍         | 37/788 [00:00<00:00, 995.70it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.bias] 
Loading weights:   5%|▍         | 38/788 [00:00<00:00, 1009.92it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.weight]
Loading weights:   5%|▍         | 38/788 [00:00<00:00, 1007.53it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.query.weight]
Loading weights:   5%|▍         | 39/788 [00:00<00:00, 1028.75it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.bias]  
Loading weights:   5%|▍         | 39/788 [00:00<00:00, 1020.54it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.bias]
Loading weights:   5%|▌         | 40/788 [00:00<00:00, 1030.45it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.weight]
Loading weights:   5%|▌         | 40/788 [00:00<00:00, 1023.41it/s, Materializing param=text_decoder.bert.encoder.layer.1.attention.self.value.weight]
Loading weights:   5%|▌         | 41/788 [00:00<00:00, 1042.07it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:   5%|▌         | 41/788 [00:00<00:00, 1033.71it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:   5%|▌         | 42/788 [00:00<00:00, 1053.48it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:   5%|▌         | 42/788 [00:00<00:00, 1043.54it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:   5%|▌         | 43/788 [00:00<00:00, 1044.45it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.bias]      
Loading weights:   5%|▌         | 43/788 [00:00<00:00, 1043.14it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.bias]
Loading weights:   6%|▌         | 44/788 [00:00<00:00, 1038.30it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:   6%|▌         | 44/788 [00:00<00:00, 1021.74it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:   6%|▌         | 45/788 [00:00<00:00, 1029.47it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.bias]      
Loading weights:   6%|▌         | 45/788 [00:00<00:00, 1025.32it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.bias]
Loading weights:   6%|▌         | 46/788 [00:00<00:00, 1038.21it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.weight]
Loading weights:   6%|▌         | 46/788 [00:00<00:00, 1036.09it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.key.weight]
Loading weights:   6%|▌         | 47/788 [00:00<00:00, 1047.66it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.bias]
Loading weights:   6%|▌         | 47/788 [00:00<00:00, 1037.35it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.bias]
Loading weights:   6%|▌         | 48/788 [00:00<00:00, 1044.89it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.weight]
Loading weights:   6%|▌         | 48/788 [00:00<00:00, 1043.46it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.query.weight]
Loading weights:   6%|▌         | 49/788 [00:00<00:00, 1044.18it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.bias]  
Loading weights:   6%|▌         | 49/788 [00:00<00:00, 1035.61it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.bias]
Loading weights:   6%|▋         | 50/788 [00:00<00:00, 1037.35it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.weight]
Loading weights:   6%|▋         | 50/788 [00:00<00:00, 1033.55it/s, Materializing param=text_decoder.bert.encoder.layer.1.crossattention.self.value.weight]
Loading weights:   6%|▋         | 51/788 [00:00<00:00, 1041.97it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.bias]         
Loading weights:   6%|▋         | 51/788 [00:00<00:00, 1035.92it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.bias]
Loading weights:   7%|▋         | 52/788 [00:00<00:00, 1053.92it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.weight]
Loading weights:   7%|▋         | 52/788 [00:00<00:00, 1051.34it/s, Materializing param=text_decoder.bert.encoder.layer.1.intermediate.dense.weight]
Loading weights:   7%|▋         | 53/788 [00:00<00:00, 1060.62it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.bias]    
Loading weights:   7%|▋         | 53/788 [00:00<00:00, 1058.86it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.bias]
Loading weights:   7%|▋         | 54/788 [00:00<00:00, 1076.77it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.weight]
Loading weights:   7%|▋         | 54/788 [00:00<00:00, 1075.23it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.LayerNorm.weight]
Loading weights:   7%|▋         | 55/788 [00:00<00:00, 1071.48it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.bias]      
Loading weights:   7%|▋         | 55/788 [00:00<00:00, 1063.69it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.bias]
Loading weights:   7%|▋         | 56/788 [00:00<00:00, 1073.17it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.weight]
Loading weights:   7%|▋         | 56/788 [00:00<00:00, 1068.15it/s, Materializing param=text_decoder.bert.encoder.layer.1.output.dense.weight]
Loading weights:   7%|▋         | 57/788 [00:00<00:00, 1084.19it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:   7%|▋         | 57/788 [00:00<00:00, 1077.53it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:   7%|▋         | 58/788 [00:00<00:00, 1085.09it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:   7%|▋         | 58/788 [00:00<00:00, 1073.96it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:   7%|▋         | 59/788 [00:00<00:00, 1082.71it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.bias]      
Loading weights:   7%|▋         | 59/788 [00:00<00:00, 1068.07it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.bias]
Loading weights:   8%|▊         | 60/788 [00:00<00:00, 1083.60it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.weight]
Loading weights:   8%|▊         | 60/788 [00:00<00:00, 1080.42it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.output.dense.weight]
Loading weights:   8%|▊         | 61/788 [00:00<00:00, 1089.81it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.bias]      
Loading weights:   8%|▊         | 61/788 [00:00<00:00, 1078.45it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.bias]
Loading weights:   8%|▊         | 62/788 [00:00<00:00, 1079.18it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.weight]
Loading weights:   8%|▊         | 62/788 [00:00<00:00, 1076.28it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.key.weight]
Loading weights:   8%|▊         | 63/788 [00:00<00:00, 1068.72it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.bias]
Loading weights:   8%|▊         | 63/788 [00:00<00:00, 1060.49it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.bias]
Loading weights:   8%|▊         | 64/788 [00:00<00:00, 1066.97it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.weight]
Loading weights:   8%|▊         | 64/788 [00:00<00:00, 1065.88it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.query.weight]
Loading weights:   8%|▊         | 65/788 [00:00<00:00, 1074.10it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.bias]  
Loading weights:   8%|▊         | 65/788 [00:00<00:00, 1072.40it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.bias]
Loading weights:   8%|▊         | 66/788 [00:00<00:00, 1069.50it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.weight]
Loading weights:   8%|▊         | 66/788 [00:00<00:00, 1068.00it/s, Materializing param=text_decoder.bert.encoder.layer.2.attention.self.value.weight]
Loading weights:   9%|▊         | 67/788 [00:00<00:00, 1070.97it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:   9%|▊         | 67/788 [00:00<00:00, 1069.47it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:   9%|▊         | 68/788 [00:00<00:00, 1077.30it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:   9%|▊         | 68/788 [00:00<00:00, 1072.57it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:   9%|▉         | 69/788 [00:00<00:00, 1074.20it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.bias]      
Loading weights:   9%|▉         | 69/788 [00:00<00:00, 1073.15it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.bias]
Loading weights:   9%|▉         | 70/788 [00:00<00:00, 1079.60it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:   9%|▉         | 70/788 [00:00<00:00, 1078.68it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:   9%|▉         | 71/788 [00:00<00:00, 1068.07it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.bias]      
Loading weights:   9%|▉         | 71/788 [00:00<00:00, 1066.65it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.bias]
Loading weights:   9%|▉         | 72/788 [00:00<00:00, 1075.70it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.weight]
Loading weights:   9%|▉         | 72/788 [00:00<00:00, 1065.85it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.key.weight]
Loading weights:   9%|▉         | 73/788 [00:00<00:00, 1078.69it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.bias]
Loading weights:   9%|▉         | 73/788 [00:00<00:00, 1069.62it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.bias]
Loading weights:   9%|▉         | 74/788 [00:00<00:00, 1071.36it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.weight]
Loading weights:   9%|▉         | 74/788 [00:00<00:00, 1070.20it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.query.weight]
Loading weights:  10%|▉         | 75/788 [00:00<00:00, 1080.65it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.bias]  
Loading weights:  10%|▉         | 75/788 [00:00<00:00, 1074.62it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.bias]
Loading weights:  10%|▉         | 76/788 [00:00<00:00, 1085.76it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  10%|▉         | 76/788 [00:00<00:00, 1084.87it/s, Materializing param=text_decoder.bert.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  10%|▉         | 77/788 [00:00<00:00, 1093.46it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.bias]         
Loading weights:  10%|▉         | 77/788 [00:00<00:00, 1087.63it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.bias]
Loading weights:  10%|▉         | 78/788 [00:00<00:00, 1099.63it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.weight]
Loading weights:  10%|▉         | 78/788 [00:00<00:00, 1095.51it/s, Materializing param=text_decoder.bert.encoder.layer.2.intermediate.dense.weight]
Loading weights:  10%|█         | 79/788 [00:00<00:00, 1097.94it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.bias]    
Loading weights:  10%|█         | 79/788 [00:00<00:00, 1064.55it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.bias]
Loading weights:  10%|█         | 80/788 [00:00<00:00, 1064.09it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  10%|█         | 80/788 [00:00<00:00, 1059.03it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  10%|█         | 81/788 [00:00<00:00, 1069.69it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.bias]      
Loading weights:  10%|█         | 81/788 [00:00<00:00, 1068.90it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.bias]
Loading weights:  10%|█         | 82/788 [00:00<00:00, 1070.60it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.weight]
Loading weights:  10%|█         | 82/788 [00:00<00:00, 1066.61it/s, Materializing param=text_decoder.bert.encoder.layer.2.output.dense.weight]
Loading weights:  11%|█         | 83/788 [00:00<00:00, 1074.12it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  11%|█         | 83/788 [00:00<00:00, 1072.49it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  11%|█         | 84/788 [00:00<00:00, 1080.71it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  11%|█         | 84/788 [00:00<00:00, 1076.30it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  11%|█         | 85/788 [00:00<00:00, 1084.95it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.bias]      
Loading weights:  11%|█         | 85/788 [00:00<00:00, 1084.10it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.bias]
Loading weights:  11%|█         | 86/788 [00:00<00:00, 1091.40it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.weight]
Loading weights:  11%|█         | 86/788 [00:00<00:00, 1089.34it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.output.dense.weight]
Loading weights:  11%|█         | 87/788 [00:00<00:00, 1096.13it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.bias]      
Loading weights:  11%|█         | 87/788 [00:00<00:00, 1094.91it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.bias]
Loading weights:  11%|█         | 88/788 [00:00<00:00, 1102.21it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.weight]
Loading weights:  11%|█         | 88/788 [00:00<00:00, 1101.47it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.key.weight]
Loading weights:  11%|█▏        | 89/788 [00:00<00:00, 1111.34it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.bias]
Loading weights:  11%|█▏        | 89/788 [00:00<00:00, 1110.64it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.bias]
Loading weights:  11%|█▏        | 90/788 [00:00<00:00, 1116.24it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.weight]
Loading weights:  11%|█▏        | 90/788 [00:00<00:00, 1111.90it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.query.weight]
Loading weights:  12%|█▏        | 91/788 [00:00<00:00, 1115.76it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.bias]  
Loading weights:  12%|█▏        | 91/788 [00:00<00:00, 1114.74it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.bias]
Loading weights:  12%|█▏        | 92/788 [00:00<00:00, 1117.89it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.weight]
Loading weights:  12%|█▏        | 92/788 [00:00<00:00, 1113.47it/s, Materializing param=text_decoder.bert.encoder.layer.3.attention.self.value.weight]
Loading weights:  12%|█▏        | 93/788 [00:00<00:00, 1119.47it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  12%|█▏        | 93/788 [00:00<00:00, 1114.83it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  12%|█▏        | 94/788 [00:00<00:00, 1120.22it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  12%|█▏        | 94/788 [00:00<00:00, 1114.59it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  12%|█▏        | 95/788 [00:00<00:00, 1113.92it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.bias]      
Loading weights:  12%|█▏        | 95/788 [00:00<00:00, 1111.77it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.bias]
Loading weights:  12%|█▏        | 96/788 [00:00<00:00, 1111.83it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  12%|█▏        | 96/788 [00:00<00:00, 1110.91it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  12%|█▏        | 97/788 [00:00<00:00, 1102.55it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.bias]      
Loading weights:  12%|█▏        | 97/788 [00:00<00:00, 1093.63it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.bias]
Loading weights:  12%|█▏        | 98/788 [00:00<00:00, 1099.42it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  12%|█▏        | 98/788 [00:00<00:00, 1096.28it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  13%|█▎        | 99/788 [00:00<00:00, 1101.75it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  13%|█▎        | 99/788 [00:00<00:00, 1100.45it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  13%|█▎        | 100/788 [00:00<00:00, 1105.50it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  13%|█▎        | 100/788 [00:00<00:00, 1102.20it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  13%|█▎        | 101/788 [00:00<00:00, 1103.26it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.bias]  
Loading weights:  13%|█▎        | 101/788 [00:00<00:00, 1101.12it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.bias]
Loading weights:  13%|█▎        | 102/788 [00:00<00:00, 1105.62it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  13%|█▎        | 102/788 [00:00<00:00, 1105.02it/s, Materializing param=text_decoder.bert.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  13%|█▎        | 103/788 [00:00<00:00, 1112.60it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.bias]         
Loading weights:  13%|█▎        | 103/788 [00:00<00:00, 1111.76it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.bias]
Loading weights:  13%|█▎        | 104/788 [00:00<00:00, 1114.57it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.weight]
Loading weights:  13%|█▎        | 104/788 [00:00<00:00, 1109.67it/s, Materializing param=text_decoder.bert.encoder.layer.3.intermediate.dense.weight]
Loading weights:  13%|█▎        | 105/788 [00:00<00:00, 1115.42it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.bias]    
Loading weights:  13%|█▎        | 105/788 [00:00<00:00, 1114.30it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.bias]
Loading weights:  13%|█▎        | 106/788 [00:00<00:00, 1115.03it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  13%|█▎        | 106/788 [00:00<00:00, 1111.86it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  14%|█▎        | 107/788 [00:00<00:00, 1115.77it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.bias]      
Loading weights:  14%|█▎        | 107/788 [00:00<00:00, 1111.19it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.bias]
Loading weights:  14%|█▎        | 108/788 [00:00<00:00, 1116.80it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.weight]
Loading weights:  14%|█▎        | 108/788 [00:00<00:00, 1113.21it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.weight]
Loading weights:  14%|█▍        | 109/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.3.output.dense.weight]
Loading weights:  14%|█▍        | 109/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  14%|█▍        | 109/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  14%|█▍        | 110/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  14%|█▍        | 110/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  14%|█▍        | 111/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.bias]      
Loading weights:  14%|█▍        | 111/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.bias]
Loading weights:  14%|█▍        | 112/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.weight]
Loading weights:  14%|█▍        | 112/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.output.dense.weight]
Loading weights:  14%|█▍        | 113/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.bias]      
Loading weights:  14%|█▍        | 113/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.bias]
Loading weights:  14%|█▍        | 114/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.weight]
Loading weights:  14%|█▍        | 114/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.key.weight]
Loading weights:  15%|█▍        | 115/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.bias]
Loading weights:  15%|█▍        | 115/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.bias]
Loading weights:  15%|█▍        | 116/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.weight]
Loading weights:  15%|█▍        | 116/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.query.weight]
Loading weights:  15%|█▍        | 117/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.bias]  
Loading weights:  15%|█▍        | 117/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.bias]
Loading weights:  15%|█▍        | 118/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.weight]
Loading weights:  15%|█▍        | 118/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.attention.self.value.weight]
Loading weights:  15%|█▌        | 119/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  15%|█▌        | 119/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  15%|█▌        | 120/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  15%|█▌        | 120/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  15%|█▌        | 121/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.bias]      
Loading weights:  15%|█▌        | 121/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.bias]
Loading weights:  15%|█▌        | 122/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  15%|█▌        | 122/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  16%|█▌        | 123/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.bias]      
Loading weights:  16%|█▌        | 123/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.bias]
Loading weights:  16%|█▌        | 124/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  16%|█▌        | 124/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  16%|█▌        | 125/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  16%|█▌        | 125/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  16%|█▌        | 126/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  16%|█▌        | 126/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  16%|█▌        | 127/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.bias]  
Loading weights:  16%|█▌        | 127/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.bias]
Loading weights:  16%|█▌        | 128/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  16%|█▌        | 128/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  16%|█▋        | 129/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.bias]         
Loading weights:  16%|█▋        | 129/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.bias]
Loading weights:  16%|█▋        | 130/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.weight]
Loading weights:  16%|█▋        | 130/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.intermediate.dense.weight]
Loading weights:  17%|█▋        | 131/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.bias]    
Loading weights:  17%|█▋        | 131/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.bias]
Loading weights:  17%|█▋        | 132/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 132/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 133/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.bias]      
Loading weights:  17%|█▋        | 133/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.bias]
Loading weights:  17%|█▋        | 134/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.weight]
Loading weights:  17%|█▋        | 134/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.4.output.dense.weight]
Loading weights:  17%|█▋        | 135/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  17%|█▋        | 135/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  17%|█▋        | 136/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 136/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  17%|█▋        | 137/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.bias]      
Loading weights:  17%|█▋        | 137/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.bias]
Loading weights:  18%|█▊        | 138/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.weight]
Loading weights:  18%|█▊        | 138/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.output.dense.weight]
Loading weights:  18%|█▊        | 139/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.bias]      
Loading weights:  18%|█▊        | 139/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.bias]
Loading weights:  18%|█▊        | 140/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.weight]
Loading weights:  18%|█▊        | 140/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.key.weight]
Loading weights:  18%|█▊        | 141/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.bias]
Loading weights:  18%|█▊        | 141/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.bias]
Loading weights:  18%|█▊        | 142/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.weight]
Loading weights:  18%|█▊        | 142/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.query.weight]
Loading weights:  18%|█▊        | 143/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.bias]  
Loading weights:  18%|█▊        | 143/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.bias]
Loading weights:  18%|█▊        | 144/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.weight]
Loading weights:  18%|█▊        | 144/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.attention.self.value.weight]
Loading weights:  18%|█▊        | 145/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  18%|█▊        | 145/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  19%|█▊        | 146/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  19%|█▊        | 146/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  19%|█▊        | 147/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.bias]      
Loading weights:  19%|█▊        | 147/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.bias]
Loading weights:  19%|█▉        | 148/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  19%|█▉        | 148/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  19%|█▉        | 149/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.bias]      
Loading weights:  19%|█▉        | 149/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.bias]
Loading weights:  19%|█▉        | 150/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  19%|█▉        | 150/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  19%|█▉        | 151/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  19%|█▉        | 151/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  19%|█▉        | 152/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  19%|█▉        | 152/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  19%|█▉        | 153/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.bias]  
Loading weights:  19%|█▉        | 153/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.bias]
Loading weights:  20%|█▉        | 154/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  20%|█▉        | 154/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  20%|█▉        | 155/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.bias]         
Loading weights:  20%|█▉        | 155/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.bias]
Loading weights:  20%|█▉        | 156/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.weight]
Loading weights:  20%|█▉        | 156/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.intermediate.dense.weight]
Loading weights:  20%|█▉        | 157/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.bias]    
Loading weights:  20%|█▉        | 157/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.bias]
Loading weights:  20%|██        | 158/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  20%|██        | 158/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  20%|██        | 159/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.bias]      
Loading weights:  20%|██        | 159/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.bias]
Loading weights:  20%|██        | 160/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.weight]
Loading weights:  20%|██        | 160/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.5.output.dense.weight]
Loading weights:  20%|██        | 161/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  20%|██        | 161/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  21%|██        | 162/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  21%|██        | 162/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  21%|██        | 163/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.bias]      
Loading weights:  21%|██        | 163/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.bias]
Loading weights:  21%|██        | 164/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.weight]
Loading weights:  21%|██        | 164/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.output.dense.weight]
Loading weights:  21%|██        | 165/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.bias]      
Loading weights:  21%|██        | 165/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.bias]
Loading weights:  21%|██        | 166/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.weight]
Loading weights:  21%|██        | 166/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.key.weight]
Loading weights:  21%|██        | 167/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.bias]
Loading weights:  21%|██        | 167/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.bias]
Loading weights:  21%|██▏       | 168/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.weight]
Loading weights:  21%|██▏       | 168/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.query.weight]
Loading weights:  21%|██▏       | 169/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.bias]  
Loading weights:  21%|██▏       | 169/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.bias]
Loading weights:  22%|██▏       | 170/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.weight]
Loading weights:  22%|██▏       | 170/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.attention.self.value.weight]
Loading weights:  22%|██▏       | 171/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  22%|██▏       | 171/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  22%|██▏       | 172/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  22%|██▏       | 172/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  22%|██▏       | 173/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.bias]      
Loading weights:  22%|██▏       | 173/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.bias]
Loading weights:  22%|██▏       | 174/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  22%|██▏       | 174/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  22%|██▏       | 175/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.bias]      
Loading weights:  22%|██▏       | 175/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.bias]
Loading weights:  22%|██▏       | 176/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  22%|██▏       | 176/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  22%|██▏       | 177/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  22%|██▏       | 177/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  23%|██▎       | 178/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  23%|██▎       | 178/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  23%|██▎       | 179/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.bias]  
Loading weights:  23%|██▎       | 179/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.bias]
Loading weights:  23%|██▎       | 180/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  23%|██▎       | 180/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  23%|██▎       | 181/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.bias]         
Loading weights:  23%|██▎       | 181/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.bias]
Loading weights:  23%|██▎       | 182/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.weight]
Loading weights:  23%|██▎       | 182/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.intermediate.dense.weight]
Loading weights:  23%|██▎       | 183/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.bias]    
Loading weights:  23%|██▎       | 183/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.bias]
Loading weights:  23%|██▎       | 184/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  23%|██▎       | 184/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  23%|██▎       | 185/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.bias]      
Loading weights:  23%|██▎       | 185/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.bias]
Loading weights:  24%|██▎       | 186/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.weight]
Loading weights:  24%|██▎       | 186/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.6.output.dense.weight]
Loading weights:  24%|██▎       | 187/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  24%|██▎       | 187/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  24%|██▍       | 188/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  24%|██▍       | 188/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  24%|██▍       | 189/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.bias]      
Loading weights:  24%|██▍       | 189/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.bias]
Loading weights:  24%|██▍       | 190/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.weight]
Loading weights:  24%|██▍       | 190/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.output.dense.weight]
Loading weights:  24%|██▍       | 191/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.bias]      
Loading weights:  24%|██▍       | 191/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.bias]
Loading weights:  24%|██▍       | 192/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.weight]
Loading weights:  24%|██▍       | 192/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.key.weight]
Loading weights:  24%|██▍       | 193/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.bias]
Loading weights:  24%|██▍       | 193/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.bias]
Loading weights:  25%|██▍       | 194/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.weight]
Loading weights:  25%|██▍       | 194/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.query.weight]
Loading weights:  25%|██▍       | 195/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.bias]  
Loading weights:  25%|██▍       | 195/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.bias]
Loading weights:  25%|██▍       | 196/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.weight]
Loading weights:  25%|██▍       | 196/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.attention.self.value.weight]
Loading weights:  25%|██▌       | 197/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  25%|██▌       | 197/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  25%|██▌       | 198/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  25%|██▌       | 198/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  25%|██▌       | 199/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.bias]      
Loading weights:  25%|██▌       | 199/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.bias]
Loading weights:  25%|██▌       | 200/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  25%|██▌       | 200/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  26%|██▌       | 201/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.bias]      
Loading weights:  26%|██▌       | 201/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.bias]
Loading weights:  26%|██▌       | 202/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  26%|██▌       | 202/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  26%|██▌       | 203/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  26%|██▌       | 203/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  26%|██▌       | 204/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  26%|██▌       | 204/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  26%|██▌       | 205/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.bias]  
Loading weights:  26%|██▌       | 205/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.bias]
Loading weights:  26%|██▌       | 206/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  26%|██▌       | 206/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  26%|██▋       | 207/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.bias]         
Loading weights:  26%|██▋       | 207/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.bias]
Loading weights:  26%|██▋       | 208/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.weight]
Loading weights:  26%|██▋       | 208/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.intermediate.dense.weight]
Loading weights:  27%|██▋       | 209/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.bias]    
Loading weights:  27%|██▋       | 209/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.bias]
Loading weights:  27%|██▋       | 210/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  27%|██▋       | 210/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  27%|██▋       | 211/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.bias]      
Loading weights:  27%|██▋       | 211/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.bias]
Loading weights:  27%|██▋       | 212/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.weight]
Loading weights:  27%|██▋       | 212/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.7.output.dense.weight]
Loading weights:  27%|██▋       | 213/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  27%|██▋       | 213/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  27%|██▋       | 214/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  27%|██▋       | 214/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  27%|██▋       | 215/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.bias]      
Loading weights:  27%|██▋       | 215/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.bias]
Loading weights:  27%|██▋       | 216/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.weight]
Loading weights:  27%|██▋       | 216/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.output.dense.weight]
Loading weights:  28%|██▊       | 217/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.bias]      
Loading weights:  28%|██▊       | 217/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.bias]
Loading weights:  28%|██▊       | 218/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.weight]
Loading weights:  28%|██▊       | 218/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.key.weight]
Loading weights:  28%|██▊       | 219/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.bias]
Loading weights:  28%|██▊       | 219/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.bias]
Loading weights:  28%|██▊       | 220/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.weight]
Loading weights:  28%|██▊       | 220/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.query.weight]
Loading weights:  28%|██▊       | 221/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.bias]  
Loading weights:  28%|██▊       | 221/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.bias]
Loading weights:  28%|██▊       | 222/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.weight]
Loading weights:  28%|██▊       | 222/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.attention.self.value.weight]
Loading weights:  28%|██▊       | 223/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  28%|██▊       | 223/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  28%|██▊       | 224/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  28%|██▊       | 224/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  29%|██▊       | 225/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.bias]      
Loading weights:  29%|██▊       | 225/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.bias]
Loading weights:  29%|██▊       | 226/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  29%|██▊       | 226/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  29%|██▉       | 227/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.bias]      
Loading weights:  29%|██▉       | 227/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.bias]
Loading weights:  29%|██▉       | 228/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  29%|██▉       | 228/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  29%|██▉       | 229/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  29%|██▉       | 229/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  29%|██▉       | 230/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  29%|██▉       | 230/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  29%|██▉       | 231/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.bias]  
Loading weights:  29%|██▉       | 231/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.bias]
Loading weights:  29%|██▉       | 232/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  29%|██▉       | 232/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  30%|██▉       | 233/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.bias]         
Loading weights:  30%|██▉       | 233/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.bias]
Loading weights:  30%|██▉       | 234/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.weight]
Loading weights:  30%|██▉       | 234/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.intermediate.dense.weight]
Loading weights:  30%|██▉       | 235/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.bias]    
Loading weights:  30%|██▉       | 235/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.bias]
Loading weights:  30%|██▉       | 236/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  30%|██▉       | 236/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  30%|███       | 237/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.bias]      
Loading weights:  30%|███       | 237/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.bias]
Loading weights:  30%|███       | 238/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.weight]
Loading weights:  30%|███       | 238/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.8.output.dense.weight]
Loading weights:  30%|███       | 239/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  30%|███       | 239/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  30%|███       | 240/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  30%|███       | 240/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  31%|███       | 241/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.bias]      
Loading weights:  31%|███       | 241/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.bias]
Loading weights:  31%|███       | 242/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.weight]
Loading weights:  31%|███       | 242/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.output.dense.weight]
Loading weights:  31%|███       | 243/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.bias]      
Loading weights:  31%|███       | 243/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.bias]
Loading weights:  31%|███       | 244/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.weight]
Loading weights:  31%|███       | 244/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.key.weight]
Loading weights:  31%|███       | 245/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.bias]
Loading weights:  31%|███       | 245/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.bias]
Loading weights:  31%|███       | 246/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.weight]
Loading weights:  31%|███       | 246/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.query.weight]
Loading weights:  31%|███▏      | 247/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.bias]  
Loading weights:  31%|███▏      | 247/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.bias]
Loading weights:  31%|███▏      | 248/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.weight]
Loading weights:  31%|███▏      | 248/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.attention.self.value.weight]
Loading weights:  32%|███▏      | 249/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  32%|███▏      | 249/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  32%|███▏      | 250/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  32%|███▏      | 250/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  32%|███▏      | 251/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.bias]      
Loading weights:  32%|███▏      | 251/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.bias]
Loading weights:  32%|███▏      | 252/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  32%|███▏      | 252/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  32%|███▏      | 253/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.bias]      
Loading weights:  32%|███▏      | 253/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.bias]
Loading weights:  32%|███▏      | 254/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  32%|███▏      | 254/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  32%|███▏      | 255/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  32%|███▏      | 255/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  32%|███▏      | 256/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  32%|███▏      | 256/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  33%|███▎      | 257/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.bias]  
Loading weights:  33%|███▎      | 257/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.bias]
Loading weights:  33%|███▎      | 258/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  33%|███▎      | 258/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  33%|███▎      | 259/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.bias]         
Loading weights:  33%|███▎      | 259/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.bias]
Loading weights:  33%|███▎      | 260/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.weight]
Loading weights:  33%|███▎      | 260/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.intermediate.dense.weight]
Loading weights:  33%|███▎      | 261/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.bias]    
Loading weights:  33%|███▎      | 261/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.bias]
Loading weights:  33%|███▎      | 262/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  33%|███▎      | 262/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  33%|███▎      | 263/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.bias]      
Loading weights:  33%|███▎      | 263/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.bias]
Loading weights:  34%|███▎      | 264/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.weight]
Loading weights:  34%|███▎      | 264/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.9.output.dense.weight]
Loading weights:  34%|███▎      | 265/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  34%|███▎      | 265/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  34%|███▍      | 266/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  34%|███▍      | 266/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  34%|███▍      | 267/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.bias]      
Loading weights:  34%|███▍      | 267/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.bias]
Loading weights:  34%|███▍      | 268/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.weight]
Loading weights:  34%|███▍      | 268/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.output.dense.weight]
Loading weights:  34%|███▍      | 269/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.bias]      
Loading weights:  34%|███▍      | 269/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.bias]
Loading weights:  34%|███▍      | 270/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.weight]
Loading weights:  34%|███▍      | 270/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.key.weight]
Loading weights:  34%|███▍      | 271/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.bias]
Loading weights:  34%|███▍      | 271/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.bias]
Loading weights:  35%|███▍      | 272/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.weight]
Loading weights:  35%|███▍      | 272/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.query.weight]
Loading weights:  35%|███▍      | 273/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.bias]  
Loading weights:  35%|███▍      | 273/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.bias]
Loading weights:  35%|███▍      | 274/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.weight]
Loading weights:  35%|███▍      | 274/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.attention.self.value.weight]
Loading weights:  35%|███▍      | 275/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  35%|███▍      | 275/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  35%|███▌      | 276/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  35%|███▌      | 276/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  35%|███▌      | 277/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.bias]      
Loading weights:  35%|███▌      | 277/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.bias]
Loading weights:  35%|███▌      | 278/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  35%|███▌      | 278/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  35%|███▌      | 279/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.bias]      
Loading weights:  35%|███▌      | 279/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.bias]
Loading weights:  36%|███▌      | 280/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  36%|███▌      | 280/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  36%|███▌      | 281/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  36%|███▌      | 281/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  36%|███▌      | 282/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  36%|███▌      | 282/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  36%|███▌      | 283/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.bias]  
Loading weights:  36%|███▌      | 283/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.bias]
Loading weights:  36%|███▌      | 284/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  36%|███▌      | 284/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  36%|███▌      | 285/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.bias]         
Loading weights:  36%|███▌      | 285/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.bias]
Loading weights:  36%|███▋      | 286/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.weight]
Loading weights:  36%|███▋      | 286/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.intermediate.dense.weight]
Loading weights:  36%|███▋      | 287/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.bias]    
Loading weights:  36%|███▋      | 287/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.bias]
Loading weights:  37%|███▋      | 288/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  37%|███▋      | 288/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  37%|███▋      | 289/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.bias]      
Loading weights:  37%|███▋      | 289/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.bias]
Loading weights:  37%|███▋      | 290/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.weight]
Loading weights:  37%|███▋      | 290/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.10.output.dense.weight]
Loading weights:  37%|███▋      | 291/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  37%|███▋      | 291/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  37%|███▋      | 292/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  37%|███▋      | 292/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  37%|███▋      | 293/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.bias]      
Loading weights:  37%|███▋      | 293/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.bias]
Loading weights:  37%|███▋      | 294/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.weight]
Loading weights:  37%|███▋      | 294/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.output.dense.weight]
Loading weights:  37%|███▋      | 295/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.bias]      
Loading weights:  37%|███▋      | 295/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.bias]
Loading weights:  38%|███▊      | 296/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.weight]
Loading weights:  38%|███▊      | 296/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.key.weight]
Loading weights:  38%|███▊      | 297/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.bias]
Loading weights:  38%|███▊      | 297/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.bias]
Loading weights:  38%|███▊      | 298/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.weight]
Loading weights:  38%|███▊      | 298/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.query.weight]
Loading weights:  38%|███▊      | 299/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.bias]  
Loading weights:  38%|███▊      | 299/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.bias]
Loading weights:  38%|███▊      | 300/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.weight]
Loading weights:  38%|███▊      | 300/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.attention.self.value.weight]
Loading weights:  38%|███▊      | 301/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  38%|███▊      | 301/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  38%|███▊      | 302/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  38%|███▊      | 302/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  38%|███▊      | 303/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.bias]      
Loading weights:  38%|███▊      | 303/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.bias]
Loading weights:  39%|███▊      | 304/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  39%|███▊      | 304/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  39%|███▊      | 305/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.bias]      
Loading weights:  39%|███▊      | 305/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.bias]
Loading weights:  39%|███▉      | 306/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  39%|███▉      | 306/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  39%|███▉      | 307/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  39%|███▉      | 307/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  39%|███▉      | 308/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  39%|███▉      | 308/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  39%|███▉      | 309/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.bias]  
Loading weights:  39%|███▉      | 309/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.bias]
Loading weights:  39%|███▉      | 310/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  39%|███▉      | 310/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  39%|███▉      | 311/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.bias]         
Loading weights:  39%|███▉      | 311/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.bias]
Loading weights:  40%|███▉      | 312/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.weight]
Loading weights:  40%|███▉      | 312/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.intermediate.dense.weight]
Loading weights:  40%|███▉      | 313/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.bias]    
Loading weights:  40%|███▉      | 313/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.bias]
Loading weights:  40%|███▉      | 314/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  40%|███▉      | 314/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  40%|███▉      | 315/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.bias]      
Loading weights:  40%|███▉      | 315/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.bias]
Loading weights:  40%|████      | 316/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.weight]
Loading weights:  40%|████      | 316/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.bert.encoder.layer.11.output.dense.weight]
Loading weights:  40%|████      | 317/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.bias]                     
Loading weights:  40%|███���      | 317/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.bias]
Loading weights:  40%|████      | 318/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.decoder.weight]
Loading weights:  40%|████      | 318/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.decoder.weight]
Loading weights:  40%|████      | 319/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.bias]
Loading weights:  40%|████      | 319/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.bias]
Loading weights:  41%|████      | 320/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.weight]
Loading weights:  41%|████      | 320/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.LayerNorm.weight]
Loading weights:  41%|████      | 321/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.dense.bias]      
Loading weights:  41%|████      | 321/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.dense.bias]
Loading weights:  41%|████      | 322/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.dense.weight]
Loading weights:  41%|████      | 322/788 [00:00<00:00, 1089.00it/s, Materializing param=text_decoder.cls.predictions.transform.dense.weight]
Loading weights:  41%|████      | 323/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.LayerNorm.bias]             
Loading weights:  41%|████      | 323/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.LayerNorm.bias]
Loading weights:  41%|████      | 324/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.LayerNorm.weight]
Loading weights:  41%|████      | 324/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.LayerNorm.weight]
Loading weights:  41%|████      | 325/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.position_embeddings.weight]
Loading weights:  41%|████      | 325/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.position_embeddings.weight]
Loading weights:  41%|████▏     | 326/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.word_embeddings.weight]    
Loading weights:  41%|████▏     | 326/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.embeddings.word_embeddings.weight]
Loading weights:  41%|████▏     | 327/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:  41%|████▏     | 327/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:  42%|████▏     | 328/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:  42%|████▏     | 328/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:  42%|████▏     | 329/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.dense.bias]      
Loading weights:  42%|████▏     | 329/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.dense.bias]
Loading weights:  42%|████▏     | 330/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.dense.weight]
Loading weights:  42%|████▏     | 330/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.output.dense.weight]
Loading weights:  42%|████▏     | 331/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.key.bias]      
Loading weights:  42%|████▏     | 331/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.key.bias]
Loading weights:  42%|████▏     | 332/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.key.weight]
Loading weights:  42%|████▏     | 332/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.key.weight]
Loading weights:  42%|████▏     | 333/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.query.bias]
Loading weights:  42%|████▏     | 333/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.query.bias]
Loading weights:  42%|████▏     | 334/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.query.weight]
Loading weights:  42%|████▏     | 334/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.query.weight]
Loading weights:  43%|████▎     | 335/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.value.bias]  
Loading weights:  43%|████▎     | 335/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.value.bias]
Loading weights:  43%|████▎     | 336/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.value.weight]
Loading weights:  43%|████▎     | 336/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.attention.self.value.weight]
Loading weights:  43%|████▎     | 337/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:  43%|████▎     | 337/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.LayerNorm.bias]
Loading weights:  43%|████▎     | 338/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:  43%|████▎     | 338/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.LayerNorm.weight]
Loading weights:  43%|████▎     | 339/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.dense.bias]      
Loading weights:  43%|████▎     | 339/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.dense.bias]
Loading weights:  43%|████▎     | 340/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:  43%|████▎     | 340/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.output.dense.weight]
Loading weights:  43%|████▎     | 341/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.key.bias]      
Loading weights:  43%|████▎     | 341/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.key.bias]
Loading weights:  43%|████▎     | 342/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.key.weight]
Loading weights:  43%|████▎     | 342/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.key.weight]
Loading weights:  44%|████▎     | 343/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.query.bias]
Loading weights:  44%|████▎     | 343/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.query.bias]
Loading weights:  44%|████▎     | 344/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.query.weight]
Loading weights:  44%|████▎     | 344/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.query.weight]
Loading weights:  44%|████▍     | 345/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.value.bias]  
Loading weights:  44%|████▍     | 345/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.value.bias]
Loading weights:  44%|████▍     | 346/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.value.weight]
Loading weights:  44%|████▍     | 346/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.crossattention.self.value.weight]
Loading weights:  44%|████▍     | 347/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.intermediate.dense.bias]         
Loading weights:  44%|████▍     | 347/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.intermediate.dense.bias]
Loading weights:  44%|████▍     | 348/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.intermediate.dense.weight]
Loading weights:  44%|████▍     | 348/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.intermediate.dense.weight]
Loading weights:  44%|████▍     | 349/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.LayerNorm.bias]    
Loading weights:  44%|████▍     | 349/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.LayerNorm.bias]
Loading weights:  44%|████▍     | 350/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.LayerNorm.weight]
Loading weights:  44%|████▍     | 350/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.LayerNorm.weight]
Loading weights:  45%|████▍     | 351/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.dense.bias]      
Loading weights:  45%|████▍     | 351/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.dense.bias]
Loading weights:  45%|████▍     | 352/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.dense.weight]
Loading weights:  45%|████▍     | 352/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.0.output.dense.weight]
Loading weights:  45%|████▍     | 353/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:  45%|████▍     | 353/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:  45%|████▍     | 354/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:  45%|████▍     | 354/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:  45%|████▌     | 355/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.dense.bias]      
Loading weights:  45%|████▌     | 355/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.dense.bias]
Loading weights:  45%|████▌     | 356/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.dense.weight]
Loading weights:  45%|████▌     | 356/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.output.dense.weight]
Loading weights:  45%|████▌     | 357/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.key.bias]      
Loading weights:  45%|████▌     | 357/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.key.bias]
Loading weights:  45%|████▌     | 358/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.key.weight]
Loading weights:  45%|████▌     | 358/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.key.weight]
Loading weights:  46%|████▌     | 359/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.query.bias]
Loading weights:  46%|████▌     | 359/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.query.bias]
Loading weights:  46%|████▌     | 360/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.query.weight]
Loading weights:  46%|████▌     | 360/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.query.weight]
Loading weights:  46%|████▌     | 361/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.value.bias]  
Loading weights:  46%|████▌     | 361/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.value.bias]
Loading weights:  46%|████▌     | 362/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.value.weight]
Loading weights:  46%|████▌     | 362/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.attention.self.value.weight]
Loading weights:  46%|████▌     | 363/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:  46%|████▌     | 363/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.LayerNorm.bias]
Loading weights:  46%|████▌     | 364/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:  46%|████▌     | 364/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.LayerNorm.weight]
Loading weights:  46%|████▋     | 365/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.dense.bias]      
Loading weights:  46%|████▋     | 365/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.dense.bias]
Loading weights:  46%|████▋     | 366/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:  46%|██��█▋     | 366/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.output.dense.weight]
Loading weights:  47%|████▋     | 367/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.key.bias]      
Loading weights:  47%|████▋     | 367/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.key.bias]
Loading weights:  47%|████▋     | 368/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.key.weight]
Loading weights:  47%|████▋     | 368/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.key.weight]
Loading weights:  47%|████▋     | 369/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.query.bias]
Loading weights:  47%|████▋     | 369/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.query.bias]
Loading weights:  47%|████▋     | 370/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.query.weight]
Loading weights:  47%|████▋     | 370/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.query.weight]
Loading weights:  47%|████▋     | 371/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.value.bias]  
Loading weights:  47%|████▋     | 371/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.value.bias]
Loading weights:  47%|████▋     | 372/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.value.weight]
Loading weights:  47%|████▋     | 372/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.crossattention.self.value.weight]
Loading weights:  47%|████▋     | 373/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.intermediate.dense.bias]         
Loading weights:  47%|████▋     | 373/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.intermediate.dense.bias]
Loading weights:  47%|████▋     | 374/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.intermediate.dense.weight]
Loading weights:  47%|████▋     | 374/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.intermediate.dense.weight]
Loading weights:  48%|████▊     | 375/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.LayerNorm.bias]    
Loading weights:  48%|████▊     | 375/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.LayerNorm.bias]
Loading weights:  48%|████▊     | 376/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.LayerNorm.weight]
Loading weights:  48%|████▊     | 376/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.LayerNorm.weight]
Loading weights:  48%|████▊     | 377/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.dense.bias]      
Loading weights:  48%|████▊     | 377/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.dense.bias]
Loading weights:  48%|████▊     | 378/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.dense.weight]
Loading weights:  48%|████▊     | 378/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.1.output.dense.weight]
Loading weights:  48%|████▊     | 379/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  48%|████▊     | 379/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  48%|████▊     | 380/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  48%|████▊     | 380/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  48%|████▊     | 381/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.dense.bias]      
Loading weights:  48%|████▊     | 381/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.dense.bias]
Loading weights:  48%|████▊     | 382/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.dense.weight]
Loading weights:  48%|████▊     | 382/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.output.dense.weight]
Loading weights:  49%|████▊     | 383/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.key.bias]      
Loading weights:  49%|████▊     | 383/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.key.bias]
Loading weights:  49%|████▊     | 384/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.key.weight]
Loading weights:  49%|████▊     | 384/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.key.weight]
Loading weights:  49%|████▉     | 385/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.query.bias]
Loading weights:  49%|████▉     | 385/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.query.bias]
Loading weights:  49%|████▉     | 386/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.query.weight]
Loading weights:  49%|████▉     | 386/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.query.weight]
Loading weights:  49%|████▉     | 387/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.value.bias]  
Loading weights:  49%|████▉     | 387/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.value.bias]
Loading weights:  49%|████▉     | 388/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.value.weight]
Loading weights:  49%|████▉     | 388/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.attention.self.value.weight]
Loading weights:  49%|████▉     | 389/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:  49%|████▉     | 389/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.LayerNorm.bias]
Loading weights:  49%|████▉     | 390/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:  49%|████▉     | 390/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.LayerNorm.weight]
Loading weights:  50%|████▉     | 391/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.dense.bias]      
Loading weights:  50%|████▉     | 391/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.dense.bias]
Loading weights:  50%|████▉     | 392/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:  50%|████▉     | 392/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.output.dense.weight]
Loading weights:  50%|████▉     | 393/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.key.bias]      
Loading weights:  50%|████▉     | 393/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.key.bias]
Loading weights:  50%|█████     | 394/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.key.weight]
Loading weights:  50%|█████     | 394/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.key.weight]
Loading weights:  50%|█████     | 395/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.query.bias]
Loading weights:  50%|█████     | 395/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.query.bias]
Loading weights:  50%|█████     | 396/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.query.weight]
Loading weights:  50%|█████     | 396/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.query.weight]
Loading weights:  50%|█████     | 397/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.value.bias]  
Loading weights:  50%|█████     | 397/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.value.bias]
Loading weights:  51%|█████     | 398/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  51%|█████     | 398/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.crossattention.self.value.weight]
Loading weights:  51%|█████     | 399/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.intermediate.dense.bias]         
Loading weights:  51%|█████     | 399/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.intermediate.dense.bias]
Loading weights:  51%|█████     | 400/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.intermediate.dense.weight]
Loading weights:  51%|█████     | 400/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.intermediate.dense.weight]
Loading weights:  51%|█████     | 401/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.LayerNorm.bias]    
Loading weights:  51%|█████     | 401/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.LayerNorm.bias]
Loading weights:  51%|█████     | 402/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  51%|█████     | 402/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.LayerNorm.weight]
Loading weights:  51%|█████     | 403/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.dense.bias]      
Loading weights:  51%|█████     | 403/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.dense.bias]
Loading weights:  51%|█████▏    | 404/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.dense.weight]
Loading weights:  51%|█████▏    | 404/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.2.output.dense.weight]
Loading weights:  51%|█████▏    | 405/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  51%|█████▏    | 405/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  52%|█████▏    | 406/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  52%|█████▏    | 406/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  52%|█████▏    | 407/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.dense.bias]      
Loading weights:  52%|█████▏    | 407/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.dense.bias]
Loading weights:  52%|█████▏    | 408/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 408/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.output.dense.weight]
Loading weights:  52%|█████▏    | 409/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.key.bias]      
Loading weights:  52%|█████▏    | 409/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.key.bias]
Loading weights:  52%|█████▏    | 410/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.key.weight]
Loading weights:  52%|█████▏    | 410/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.key.weight]
Loading weights:  52%|█████▏    | 411/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.query.bias]
Loading weights:  52%|█████▏    | 411/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.query.bias]
Loading weights:  52%|█████▏    | 412/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.query.weight]
Loading weights:  52%|█████▏    | 412/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.query.weight]
Loading weights:  52%|█████▏    | 413/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.value.bias]  
Loading weights:  52%|█████▏    | 413/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.value.bias]
Loading weights:  53%|█████▎    | 414/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.value.weight]
Loading weights:  53%|█████▎    | 414/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.attention.self.value.weight]
Loading weights:  53%|█████▎    | 415/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  53%|█████▎    | 415/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.LayerNorm.bias]
Loading weights:  53%|█████▎    | 416/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  53%|█████▎    | 416/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.LayerNorm.weight]
Loading weights:  53%|█████▎    | 417/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.dense.bias]      
Loading weights:  53%|█████▎    | 417/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.dense.bias]
Loading weights:  53%|█████▎    | 418/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  53%|█████▎    | 418/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.output.dense.weight]
Loading weights:  53%|█████▎    | 419/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.key.bias]      
Loading weights:  53%|█████▎    | 419/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.key.bias]
Loading weights:  53%|█████▎    | 420/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  53%|█████▎    | 420/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.key.weight]
Loading weights:  53%|█████▎    | 421/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  53%|█████▎    | 421/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.query.bias]
Loading weights:  54%|█████▎    | 422/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  54%|█████▎    | 422/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.query.weight]
Loading weights:  54%|█████▎    | 423/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.value.bias]  
Loading weights:  54%|█████▎    | 423/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.value.bias]
Loading weights:  54%|█████▍    | 424/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  54%|█████▍    | 424/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.crossattention.self.value.weight]
Loading weights:  54%|█████▍    | 425/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.intermediate.dense.bias]         
Loading weights:  54%|█████▍    | 425/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.intermediate.dense.bias]
Loading weights:  54%|█████▍    | 426/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.intermediate.dense.weight]
Loading weights:  54%|█████▍    | 426/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.intermediate.dense.weight]
Loading weights:  54%|█████▍    | 427/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.LayerNorm.bias]    
Loading weights:  54%|█████▍    | 427/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.LayerNorm.bias]
Loading weights:  54%|█████▍    | 428/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  54%|█████▍    | 428/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.LayerNorm.weight]
Loading weights:  54%|█████▍    | 429/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.dense.bias]      
Loading weights:  54%|█████▍    | 429/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.dense.bias]
Loading weights:  55%|█████▍    | 430/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.dense.weight]
Loading weights:  55%|█████▍    | 430/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.3.output.dense.weight]
Loading weights:  55%|█████▍    | 431/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  55%|█████▍    | 431/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  55%|█████▍    | 432/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  55%|█████▍    | 432/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  55%|█████▍    | 433/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.dense.bias]      
Loading weights:  55%|█████▍    | 433/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.dense.bias]
Loading weights:  55%|█████▌    | 434/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.dense.weight]
Loading weights:  55%|█████▌    | 434/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.output.dense.weight]
Loading weights:  55%|█████▌    | 435/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.key.bias]      
Loading weights:  55%|█████▌    | 435/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.key.bias]
Loading weights:  55%|█████▌    | 436/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.key.weight]
Loading weights:  55%|█████▌    | 436/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.key.weight]
Loading weights:  55%|█████▌    | 437/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.query.bias]
Loading weights:  55%|█████▌    | 437/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.query.bias]
Loading weights:  56%|█████▌    | 438/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.query.weight]
Loading weights:  56%|█████▌    | 438/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.query.weight]
Loading weights:  56%|█████▌    | 439/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.value.bias]  
Loading weights:  56%|█████▌    | 439/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.value.bias]
Loading weights:  56%|█████▌    | 440/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.value.weight]
Loading weights:  56%|█████▌    | 440/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.attention.self.value.weight]
Loading weights:  56%|█████▌    | 441/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  56%|█████▌    | 441/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.LayerNorm.bias]
Loading weights:  56%|█████▌    | 442/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  56%|█████▌    | 442/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.LayerNorm.weight]
Loading weights:  56%|█████▌    | 443/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.dense.bias]      
Loading weights:  56%|█████▌    | 443/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.dense.bias]
Loading weights:  56%|█████▋    | 444/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  56%|█████▋    | 444/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.output.dense.weight]
Loading weights:  56%|█████▋    | 445/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.key.bias]      
Loading weights:  56%|█████▋    | 445/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.key.bias]
Loading weights:  57%|█████▋    | 446/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  57%|█████▋    | 446/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.key.weight]
Loading weights:  57%|█████▋    | 447/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  57%|█████▋    | 447/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.query.bias]
Loading weights:  57%|█████▋    | 448/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  57%|█████▋    | 448/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.query.weight]
Loading weights:  57%|█████▋    | 449/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.value.bias]  
Loading weights:  57%|█████▋    | 449/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.value.bias]
Loading weights:  57%|█████▋    | 450/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  57%|█████▋    | 450/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.crossattention.self.value.weight]
Loading weights:  57%|█████▋    | 451/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.intermediate.dense.bias]         
Loading weights:  57%|█████▋    | 451/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.intermediate.dense.bias]
Loading weights:  57%|█████▋    | 452/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.intermediate.dense.weight]
Loading weights:  57%|█████▋    | 452/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.intermediate.dense.weight]
Loading weights:  57%|█████▋    | 453/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.LayerNorm.bias]    
Loading weights:  57%|█████▋    | 453/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.LayerNorm.bias]
Loading weights:  58%|█████▊    | 454/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  58%|█████▊    | 454/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.LayerNorm.weight]
Loading weights:  58%|█████▊    | 455/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.dense.bias]      
Loading weights:  58%|█████▊    | 455/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.dense.bias]
Loading weights:  58%|█████▊    | 456/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.dense.weight]
Loading weights:  58%|█████▊    | 456/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.4.output.dense.weight]
Loading weights:  58%|█████▊    | 457/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  58%|█████▊    | 457/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  58%|█████▊    | 458/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  58%|█████▊    | 458/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  58%|█████▊    | 459/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.dense.bias]      
Loading weights:  58%|█████▊    | 459/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.dense.bias]
Loading weights:  58%|█████▊    | 460/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.dense.weight]
Loading weights:  58%|█████▊    | 460/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.output.dense.weight]
Loading weights:  59%|█████▊    | 461/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.key.bias]      
Loading weights:  59%|█████▊    | 461/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.key.bias]
Loading weights:  59%|█████▊    | 462/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.key.weight]
Loading weights:  59%|█████▊    | 462/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.key.weight]
Loading weights:  59%|█████▉    | 463/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.query.bias]
Loading weights:  59%|█████▉    | 463/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.query.bias]
Loading weights:  59%|█████▉    | 464/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.query.weight]
Loading weights:  59%|█████▉    | 464/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.query.weight]
Loading weights:  59%|█████▉    | 465/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.value.bias]  
Loading weights:  59%|█████▉    | 465/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.value.bias]
Loading weights:  59%|█████▉    | 466/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.value.weight]
Loading weights:  59%|█████▉    | 466/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.attention.self.value.weight]
Loading weights:  59%|█████▉    | 467/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  59%|█████▉    | 467/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.LayerNorm.bias]
Loading weights:  59%|█████▉    | 468/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  59%|█████▉    | 468/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.LayerNorm.weight]
Loading weights:  60%|█████▉    | 469/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.dense.bias]      
Loading weights:  60%|█████▉    | 469/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.dense.bias]
Loading weights:  60%|█████▉    | 470/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  60%|█████▉    | 470/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.output.dense.weight]
Loading weights:  60%|█████▉    | 471/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.key.bias]      
Loading weights:  60%|█████▉    | 471/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.key.bias]
Loading weights:  60%|█████▉    | 472/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  60%|█████▉    | 472/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.key.weight]
Loading weights:  60%|██████    | 473/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  60%|██████    | 473/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.query.bias]
Loading weights:  60%|██████    | 474/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  60%|██████    | 474/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.query.weight]
Loading weights:  60%|██████    | 475/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.value.bias]  
Loading weights:  60%|██████    | 475/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.value.bias]
Loading weights:  60%|██████    | 476/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  60%|██████    | 476/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.crossattention.self.value.weight]
Loading weights:  61%|██████    | 477/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.intermediate.dense.bias]         
Loading weights:  61%|██████    | 477/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.intermediate.dense.bias]
Loading weights:  61%|██████    | 478/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.intermediate.dense.weight]
Loading weights:  61%|██████    | 478/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.intermediate.dense.weight]
Loading weights:  61%|██████    | 479/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.LayerNorm.bias]    
Loading weights:  61%|██████    | 479/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.LayerNorm.bias]
Loading weights:  61%|██████    | 480/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  61%|██████    | 480/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.LayerNorm.weight]
Loading weights:  61%|██████    | 481/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.dense.bias]      
Loading weights:  61%|██████    | 481/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.dense.bias]
Loading weights:  61%|██████    | 482/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.dense.weight]
Loading weights:  61%|██████    | 482/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.5.output.dense.weight]
Loading weights:  61%|██████▏   | 483/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  61%|██████▏   | 483/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.LayerNorm.bias]
Loading weights:  61%|██████▏   | 484/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  61%|██████▏   | 484/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.LayerNorm.weight]
Loading weights:  62%|██████▏   | 485/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.dense.bias]      
Loading weights:  62%|██████▏   | 485/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.dense.bias]
Loading weights:  62%|██████▏   | 486/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.dense.weight]
Loading weights:  62%|██████▏   | 486/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.output.dense.weight]
Loading weights:  62%|██████▏   | 487/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.key.bias]      
Loading weights:  62%|██████▏   | 487/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.key.bias]
Loading weights:  62%|██████▏   | 488/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.key.weight]
Loading weights:  62%|██████▏   | 488/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.key.weight]
Loading weights:  62%|██████▏   | 489/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.query.bias]
Loading weights:  62%|██████▏   | 489/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.query.bias]
Loading weights:  62%|██████▏   | 490/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.query.weight]
Loading weights:  62%|██████▏   | 490/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.query.weight]
Loading weights:  62%|██████▏   | 491/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.value.bias]  
Loading weights:  62%|██████▏   | 491/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.value.bias]
Loading weights:  62%|██████▏   | 492/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.value.weight]
Loading weights:  62%|██████▏   | 492/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.attention.self.value.weight]
Loading weights:  63%|██████▎   | 493/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  63%|██████▎   | 493/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.LayerNorm.bias]
Loading weights:  63%|██████▎   | 494/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  63%|██████▎   | 494/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.LayerNorm.weight]
Loading weights:  63%|██████▎   | 495/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.dense.bias]      
Loading weights:  63%|██████▎   | 495/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.dense.bias]
Loading weights:  63%|██████▎   | 496/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  63%|██████▎   | 496/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.output.dense.weight]
Loading weights:  63%|██████▎   | 497/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.key.bias]      
Loading weights:  63%|██████▎   | 497/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.key.bias]
Loading weights:  63%|██████▎   | 498/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  63%|██████▎   | 498/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.key.weight]
Loading weights:  63%|██████▎   | 499/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  63%|██████▎   | 499/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.query.bias]
Loading weights:  63%|██████▎   | 500/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  63%|██████▎   | 500/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.query.weight]
Loading weights:  64%|██████▎   | 501/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.value.bias]  
Loading weights:  64%|██████▎   | 501/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.value.bias]
Loading weights:  64%|██████▎   | 502/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  64%|██████▎   | 502/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.crossattention.self.value.weight]
Loading weights:  64%|██████▍   | 503/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.intermediate.dense.bias]         
Loading weights:  64%|██████▍   | 503/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.intermediate.dense.bias]
Loading weights:  64%|██████▍   | 504/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.intermediate.dense.weight]
Loading weights:  64%|██████▍   | 504/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.intermediate.dense.weight]
Loading weights:  64%|██████▍   | 505/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.LayerNorm.bias]    
Loading weights:  64%|██████▍   | 505/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.LayerNorm.bias]
Loading weights:  64%|██████▍   | 506/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  64%|██████▍   | 506/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.LayerNorm.weight]
Loading weights:  64%|██████▍   | 507/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.dense.bias]      
Loading weights:  64%|██████▍   | 507/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.dense.bias]
Loading weights:  64%|██████▍   | 508/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.dense.weight]
Loading weights:  64%|██████▍   | 508/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.6.output.dense.weight]
Loading weights:  65%|██████▍   | 509/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  65%|██████▍   | 509/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.LayerNorm.bias]
Loading weights:  65%|██████▍   | 510/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  65%|██████▍   | 510/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.LayerNorm.weight]
Loading weights:  65%|██████▍   | 511/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.dense.bias]      
Loading weights:  65%|██████▍   | 511/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.dense.bias]
Loading weights:  65%|██████▍   | 512/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.dense.weight]
Loading weights:  65%|██████▍   | 512/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.output.dense.weight]
Loading weights:  65%|██████▌   | 513/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.key.bias]      
Loading weights:  65%|██████▌   | 513/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.key.bias]
Loading weights:  65%|██████▌   | 514/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.key.weight]
Loading weights:  65%|██████▌   | 514/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.key.weight]
Loading weights:  65%|██████▌   | 515/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.query.bias]
Loading weights:  65%|██████▌   | 515/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.query.bias]
Loading weights:  65%|██████▌   | 516/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.query.weight]
Loading weights:  65%|██████▌   | 516/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.query.weight]
Loading weights:  66%|██████▌   | 517/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.value.bias]  
Loading weights:  66%|██████▌   | 517/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.value.bias]
Loading weights:  66%|██████▌   | 518/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.value.weight]
Loading weights:  66%|██████▌   | 518/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.attention.self.value.weight]
Loading weights:  66%|██████▌   | 519/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  66%|██████▌   | 519/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.LayerNorm.bias]
Loading weights:  66%|██████▌   | 520/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  66%|██████▌   | 520/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.LayerNorm.weight]
Loading weights:  66%|██████▌   | 521/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.dense.bias]      
Loading weights:  66%|██████▌   | 521/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.dense.bias]
Loading weights:  66%|██████▌   | 522/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  66%|██████▌   | 522/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.output.dense.weight]
Loading weights:  66%|██████▋   | 523/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.key.bias]      
Loading weights:  66%|██████▋   | 523/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.key.bias]
Loading weights:  66%|██████▋   | 524/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  66%|██████▋   | 524/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.key.weight]
Loading weights:  67%|██████▋   | 525/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  67%|██████▋   | 525/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.query.bias]
Loading weights:  67%|██████▋   | 526/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  67%|██████▋   | 526/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.query.weight]
Loading weights:  67%|██████▋   | 527/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.value.bias]  
Loading weights:  67%|██████▋   | 527/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.value.bias]
Loading weights:  67%|██████▋   | 528/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  67%|██████▋   | 528/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.crossattention.self.value.weight]
Loading weights:  67%|██████▋   | 529/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.intermediate.dense.bias]         
Loading weights:  67%|██████▋   | 529/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.intermediate.dense.bias]
Loading weights:  67%|██████▋   | 530/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.intermediate.dense.weight]
Loading weights:  67%|██████▋   | 530/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.intermediate.dense.weight]
Loading weights:  67%|██████▋   | 531/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.LayerNorm.bias]    
Loading weights:  67%|██████▋   | 531/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.LayerNorm.bias]
Loading weights:  68%|██████▊   | 532/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  68%|██████▊   | 532/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.LayerNorm.weight]
Loading weights:  68%|██████▊   | 533/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.dense.bias]      
Loading weights:  68%|██████▊   | 533/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.dense.bias]
Loading weights:  68%|██████▊   | 534/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.dense.weight]
Loading weights:  68%|██████▊   | 534/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.7.output.dense.weight]
Loading weights:  68%|██████▊   | 535/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  68%|██████▊   | 535/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.LayerNorm.bias]
Loading weights:  68%|██████▊   | 536/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  68%|██████▊   | 536/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.LayerNorm.weight]
Loading weights:  68%|██████▊   | 537/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.dense.bias]      
Loading weights:  68%|██████▊   | 537/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.dense.bias]
Loading weights:  68%|██████▊   | 538/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.dense.weight]
Loading weights:  68%|██████▊   | 538/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.output.dense.weight]
Loading weights:  68%|██████▊   | 539/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.key.bias]      
Loading weights:  68%|██████▊   | 539/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.key.bias]
Loading weights:  69%|██████▊   | 540/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.key.weight]
Loading weights:  69%|██████▊   | 540/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.key.weight]
Loading weights:  69%|██████▊   | 541/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.query.bias]
Loading weights:  69%|██████▊   | 541/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.query.bias]
Loading weights:  69%|██████▉   | 542/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.query.weight]
Loading weights:  69%|██████▉   | 542/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.query.weight]
Loading weights:  69%|██████▉   | 543/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.value.bias]  
Loading weights:  69%|██████▉   | 543/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.value.bias]
Loading weights:  69%|██████▉   | 544/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.value.weight]
Loading weights:  69%|██████▉   | 544/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.attention.self.value.weight]
Loading weights:  69%|██████▉   | 545/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  69%|██████▉   | 545/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.LayerNorm.bias]
Loading weights:  69%|██████▉   | 546/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  69%|██████▉   | 546/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.LayerNorm.weight]
Loading weights:  69%|██████▉   | 547/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.dense.bias]      
Loading weights:  69%|██████▉   | 547/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.dense.bias]
Loading weights:  70%|██████▉   | 548/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  70%|██████▉   | 548/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.output.dense.weight]
Loading weights:  70%|██████▉   | 549/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.key.bias]      
Loading weights:  70%|██████▉   | 549/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.key.bias]
Loading weights:  70%|██████▉   | 550/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  70%|██████▉   | 550/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.key.weight]
Loading weights:  70%|██████▉   | 551/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  70%|██████▉   | 551/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.query.bias]
Loading weights:  70%|███████   | 552/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  70%|███████   | 552/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.query.weight]
Loading weights:  70%|███████   | 553/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.value.bias]  
Loading weights:  70%|███████   | 553/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.value.bias]
Loading weights:  70%|███████   | 554/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  70%|███████   | 554/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.crossattention.self.value.weight]
Loading weights:  70%|███████   | 555/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.intermediate.dense.bias]         
Loading weights:  70%|███████   | 555/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.intermediate.dense.bias]
Loading weights:  71%|███████   | 556/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.intermediate.dense.weight]
Loading weights:  71%|███████   | 556/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.intermediate.dense.weight]
Loading weights:  71%|███████   | 557/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.LayerNorm.bias]    
Loading weights:  71%|███████   | 557/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.LayerNorm.bias]
Loading weights:  71%|███████   | 558/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  71%|███████   | 558/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.LayerNorm.weight]
Loading weights:  71%|███████   | 559/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.dense.bias]      
Loading weights:  71%|███████   | 559/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.dense.bias]
Loading weights:  71%|███████   | 560/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.dense.weight]
Loading weights:  71%|███████   | 560/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.8.output.dense.weight]
Loading weights:  71%|███████   | 561/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  71%|███████   | 561/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.LayerNorm.bias]
Loading weights:  71%|███████▏  | 562/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  71%|███████▏  | 562/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.LayerNorm.weight]
Loading weights:  71%|███████▏  | 563/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.dense.bias]      
Loading weights:  71%|███████▏  | 563/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.dense.bias]
Loading weights:  72%|███████▏  | 564/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.dense.weight]
Loading weights:  72%|███████▏  | 564/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.output.dense.weight]
Loading weights:  72%|███████▏  | 565/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.key.bias]      
Loading weights:  72%|███████▏  | 565/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.key.bias]
Loading weights:  72%|███████▏  | 566/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.key.weight]
Loading weights:  72%|███████▏  | 566/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.key.weight]
Loading weights:  72%|███████▏  | 567/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.query.bias]
Loading weights:  72%|███████▏  | 567/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.query.bias]
Loading weights:  72%|███████▏  | 568/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.query.weight]
Loading weights:  72%|███████▏  | 568/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.query.weight]
Loading weights:  72%|███████▏  | 569/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.value.bias]  
Loading weights:  72%|███████▏  | 569/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.value.bias]
Loading weights:  72%|███████▏  | 570/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.value.weight]
Loading weights:  72%|███████▏  | 570/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.attention.self.value.weight]
Loading weights:  72%|███████▏  | 571/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  72%|███████▏  | 571/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.LayerNorm.bias]
Loading weights:  73%|███████▎  | 572/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  73%|███████▎  | 572/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.LayerNorm.weight]
Loading weights:  73%|███████▎  | 573/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.dense.bias]      
Loading weights:  73%|███████▎  | 573/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.dense.bias]
Loading weights:  73%|███████▎  | 574/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  73%|███████▎  | 574/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.output.dense.weight]
Loading weights:  73%|███████▎  | 575/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.key.bias]      
Loading weights:  73%|███████▎  | 575/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.key.bias]
Loading weights:  73%|███████▎  | 576/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  73%|███████▎  | 576/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.key.weight]
Loading weights:  73%|███████▎  | 577/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  73%|███████▎  | 577/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.query.bias]
Loading weights:  73%|███████▎  | 578/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  73%|███████▎  | 578/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.query.weight]
Loading weights:  73%|███████▎  | 579/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.value.bias]  
Loading weights:  73%|███████▎  | 579/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.value.bias]
Loading weights:  74%|███████▎  | 580/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  74%|███████▎  | 580/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.crossattention.self.value.weight]
Loading weights:  74%|███████▎  | 581/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.intermediate.dense.bias]         
Loading weights:  74%|███████▎  | 581/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.intermediate.dense.bias]
Loading weights:  74%|███████▍  | 582/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.intermediate.dense.weight]
Loading weights:  74%|███████▍  | 582/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.intermediate.dense.weight]
Loading weights:  74%|███████▍  | 583/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.LayerNorm.bias]    
Loading weights:  74%|███████▍  | 583/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.LayerNorm.bias]
Loading weights:  74%|███████▍  | 584/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  74%|███████▍  | 584/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.LayerNorm.weight]
Loading weights:  74%|███████▍  | 585/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.dense.bias]      
Loading weights:  74%|███████▍  | 585/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.dense.bias]
Loading weights:  74%|███████▍  | 586/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.dense.weight]
Loading weights:  74%|███████▍  | 586/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.9.output.dense.weight]
Loading weights:  74%|███████▍  | 587/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  74%|███████▍  | 587/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.LayerNorm.bias]
Loading weights:  75%|███████▍  | 588/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  75%|███████▍  | 588/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.LayerNorm.weight]
Loading weights:  75%|███████▍  | 589/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.dense.bias]      
Loading weights:  75%|███████▍  | 589/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.dense.bias]
Loading weights:  75%|███████▍  | 590/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.dense.weight]
Loading weights:  75%|███████▍  | 590/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.output.dense.weight]
Loading weights:  75%|███████▌  | 591/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.key.bias]      
Loading weights:  75%|███████▌  | 591/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.key.bias]
Loading weights:  75%|███████▌  | 592/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.key.weight]
Loading weights:  75%|███████▌  | 592/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.key.weight]
Loading weights:  75%|███████▌  | 593/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.query.bias]
Loading weights:  75%|███████▌  | 593/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.query.bias]
Loading weights:  75%|███████▌  | 594/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.query.weight]
Loading weights:  75%|███████▌  | 594/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.query.weight]
Loading weights:  76%|███████▌  | 595/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.value.bias]  
Loading weights:  76%|███████▌  | 595/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.value.bias]
Loading weights:  76%|███████▌  | 596/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.value.weight]
Loading weights:  76%|███████▌  | 596/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.attention.self.value.weight]
Loading weights:  76%|███████▌  | 597/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  76%|███████▌  | 597/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.LayerNorm.bias]
Loading weights:  76%|███████▌  | 598/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  76%|███████▌  | 598/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.LayerNorm.weight]
Loading weights:  76%|███████▌  | 599/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.dense.bias]      
Loading weights:  76%|███████▌  | 599/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.dense.bias]
Loading weights:  76%|███████▌  | 600/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  76%|███████▌  | 600/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.output.dense.weight]
Loading weights:  76%|███████▋  | 601/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.key.bias]      
Loading weights:  76%|███████▋  | 601/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.key.bias]
Loading weights:  76%|███████▋  | 602/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  76%|███████▋  | 602/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.key.weight]
Loading weights:  77%|███████▋  | 603/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  77%|███████▋  | 603/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.query.bias]
Loading weights:  77%|███████▋  | 604/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  77%|███████▋  | 604/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.query.weight]
Loading weights:  77%|███████▋  | 605/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.value.bias]  
Loading weights:  77%|███████▋  | 605/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.value.bias]
Loading weights:  77%|███████▋  | 606/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  77%|███████▋  | 606/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.crossattention.self.value.weight]
Loading weights:  77%|███████▋  | 607/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.intermediate.dense.bias]         
Loading weights:  77%|███████▋  | 607/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.intermediate.dense.bias]
Loading weights:  77%|███████▋  | 608/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.intermediate.dense.weight]
Loading weights:  77%|███████▋  | 608/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.intermediate.dense.weight]
Loading weights:  77%|███████▋  | 609/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.LayerNorm.bias]    
Loading weights:  77%|███████▋  | 609/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.LayerNorm.bias]
Loading weights:  77%|███████▋  | 610/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  77%|███████▋  | 610/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.LayerNorm.weight]
Loading weights:  78%|███████▊  | 611/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.dense.bias]      
Loading weights:  78%|███████▊  | 611/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.dense.bias]
Loading weights:  78%|███████▊  | 612/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.dense.weight]
Loading weights:  78%|███████▊  | 612/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.10.output.dense.weight]
Loading weights:  78%|███████▊  | 613/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  78%|███████▊  | 613/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.LayerNorm.bias]
Loading weights:  78%|███████▊  | 614/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  78%|███████▊  | 614/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.LayerNorm.weight]
Loading weights:  78%|███████▊  | 615/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.dense.bias]      
Loading weights:  78%|███████▊  | 615/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.dense.bias]
Loading weights:  78%|███████▊  | 616/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.dense.weight]
Loading weights:  78%|███████▊  | 616/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.output.dense.weight]
Loading weights:  78%|███████▊  | 617/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.key.bias]      
Loading weights:  78%|███████▊  | 617/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.key.bias]
Loading weights:  78%|███████▊  | 618/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.key.weight]
Loading weights:  78%|███████▊  | 618/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.key.weight]
Loading weights:  79%|███████▊  | 619/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.query.bias]
Loading weights:  79%|███████▊  | 619/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.query.bias]
Loading weights:  79%|███████▊  | 620/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.query.weight]
Loading weights:  79%|███████▊  | 620/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.query.weight]
Loading weights:  79%|███████▉  | 621/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.value.bias]  
Loading weights:  79%|███████▉  | 621/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.value.bias]
Loading weights:  79%|███████▉  | 622/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.value.weight]
Loading weights:  79%|███████▉  | 622/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.attention.self.value.weight]
Loading weights:  79%|███████▉  | 623/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  79%|███████▉  | 623/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.LayerNorm.bias]
Loading weights:  79%|███████▉  | 624/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  79%|███████▉  | 624/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.LayerNorm.weight]
Loading weights:  79%|███████▉  | 625/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.dense.bias]      
Loading weights:  79%|███████▉  | 625/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.dense.bias]
Loading weights:  79%|███████▉  | 626/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  79%|███████▉  | 626/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.output.dense.weight]
Loading weights:  80%|███████▉  | 627/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.key.bias]      
Loading weights:  80%|███████▉  | 627/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.key.bias]
Loading weights:  80%|███████▉  | 628/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  80%|███████▉  | 628/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.key.weight]
Loading weights:  80%|███████▉  | 629/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  80%|███████▉  | 629/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.query.bias]
Loading weights:  80%|███████▉  | 630/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  80%|███████▉  | 630/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.query.weight]
Loading weights:  80%|████████  | 631/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.value.bias]  
Loading weights:  80%|████████  | 631/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.value.bias]
Loading weights:  80%|████████  | 632/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  80%|████████  | 632/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.crossattention.self.value.weight]
Loading weights:  80%|████████  | 633/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.intermediate.dense.bias]         
Loading weights:  80%|████████  | 633/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.intermediate.dense.bias]
Loading weights:  80%|████████  | 634/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.intermediate.dense.weight]
Loading weights:  80%|████████  | 634/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.intermediate.dense.weight]
Loading weights:  81%|████████  | 635/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.LayerNorm.bias]    
Loading weights:  81%|████████  | 635/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.LayerNorm.bias]
Loading weights:  81%|████████  | 636/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  81%|████████  | 636/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.LayerNorm.weight]
Loading weights:  81%|████████  | 637/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.dense.bias]      
Loading weights:  81%|████████  | 637/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.dense.bias]
Loading weights:  81%|████████  | 638/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.dense.weight]
Loading weights:  81%|████████  | 638/788 [00:00<00:00, 1089.00it/s, Materializing param=text_encoder.encoder.layer.11.output.dense.weight]
Loading weights:  81%|████████  | 639/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.class_embedding]          
Loading weights:  81%|████████  | 639/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.class_embedding]
Loading weights:  81%|████████  | 640/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  81%|████████  | 640/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.patch_embedding.bias]
Loading weights:  81%|████████▏ | 641/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  81%|████████▏ | 641/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.patch_embedding.weight]
Loading weights:  81%|████████▏ | 642/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.position_embedding]    
Loading weights:  81%|████████▏ | 642/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.embeddings.position_embedding]
Loading weights:  82%|████████▏ | 643/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  82%|████████▏ | 643/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.bias]
Loading weights:  82%|████████▏ | 644/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  82%|████████▏ | 644/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm1.weight]
Loading weights:  82%|████████▏ | 645/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]  
Loading weights:  82%|████████▏ | 645/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.bias]
Loading weights:  82%|████████▏ | 646/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  82%|████████▏ | 646/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.layer_norm2.weight]
Loading weights:  82%|████████▏ | 647/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]      
Loading weights:  82%|████████▏ | 647/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.bias]
Loading weights:  82%|████████▏ | 648/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 648/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc1.weight]
Loading weights:  82%|████████▏ | 649/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]  
Loading weights:  82%|████████▏ | 649/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.bias]
Loading weights:  82%|████████▏ | 650/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  82%|████████▏ | 650/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.mlp.fc2.weight]
Loading weights:  83%|████████▎ | 651/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.bias]
Loading weights:  83%|████████▎ | 651/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.bias]
Loading weights:  83%|████████▎ | 652/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.weight]
Loading weights:  83%|████████▎ | 652/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.projection.weight]
Loading weights:  83%|████████▎ | 653/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.bias]         
Loading weights:  83%|████████▎ | 653/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.bias]
Loading weights:  83%|████████▎ | 654/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.weight]
Loading weights:  83%|████████▎ | 654/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.0.self_attn.qkv.weight]
Loading weights:  83%|████████▎ | 655/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]    
Loading weights:  83%|████████▎ | 655/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.bias]
Loading weights:  83%|████████▎ | 656/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  83%|████████▎ | 656/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm1.weight]
Loading weights:  83%|████████▎ | 657/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]  
Loading weights:  83%|████████▎ | 657/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.bias]
Loading weights:  84%|████████▎ | 658/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  84%|████████▎ | 658/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.layer_norm2.weight]
Loading weights:  84%|████████▎ | 659/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]      
Loading weights:  84%|████████▎ | 659/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.bias]
Loading weights:  84%|████████▍ | 660/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  84%|████████▍ | 660/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc1.weight]
Loading weights:  84%|████████▍ | 661/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]  
Loading weights:  84%|████████▍ | 661/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.bias]
Loading weights:  84%|████████▍ | 662/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  84%|████████▍ | 662/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.mlp.fc2.weight]
Loading weights:  84%|████████▍ | 663/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.bias]
Loading weights:  84%|████████▍ | 663/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.bias]
Loading weights:  84%|████████▍ | 664/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.weight]
Loading weights:  84%|████████▍ | 664/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.projection.weight]
Loading weights:  84%|████████▍ | 665/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.bias]         
Loading weights:  84%|████████▍ | 665/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.bias]
Loading weights:  85%|████████▍ | 666/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.weight]
Loading weights:  85%|████████▍ | 666/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.1.self_attn.qkv.weight]
Loading weights:  85%|████████▍ | 667/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]    
Loading weights:  85%|████████▍ | 667/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.bias]
Loading weights:  85%|████████▍ | 668/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  85%|████████▍ | 668/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm1.weight]
Loading weights:  85%|████████▍ | 669/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]  
Loading weights:  85%|████████▍ | 669/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.bias]
Loading weights:  85%|████████▌ | 670/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  85%|████████▌ | 670/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.layer_norm2.weight]
Loading weights:  85%|████████▌ | 671/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]      
Loading weights:  85%|████████▌ | 671/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.bias]
Loading weights:  85%|████████▌ | 672/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  85%|████████▌ | 672/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc1.weight]
Loading weights:  85%|████████▌ | 673/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]  
Loading weights:  85%|████████▌ | 673/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.bias]
Loading weights:  86%|████████▌ | 674/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  86%|████████▌ | 674/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.mlp.fc2.weight]
Loading weights:  86%|████████▌ | 675/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.bias]
Loading weights:  86%|████████▌ | 675/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.bias]
Loading weights:  86%|████████▌ | 676/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.weight]
Loading weights:  86%|████████▌ | 676/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.projection.weight]
Loading weights:  86%|████████▌ | 677/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.bias]         
Loading weights:  86%|████████▌ | 677/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.bias]
Loading weights:  86%|████████▌ | 678/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.weight]
Loading weights:  86%|████████▌ | 678/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.2.self_attn.qkv.weight]
Loading weights:  86%|████████▌ | 679/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]    
Loading weights:  86%|████████▌ | 679/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.bias]
Loading weights:  86%|████████▋ | 680/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  86%|████████▋ | 680/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm1.weight]
Loading weights:  86%|████████▋ | 681/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]  
Loading weights:  86%|████████▋ | 681/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.bias]
Loading weights:  87%|████████▋ | 682/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  87%|████████▋ | 682/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.layer_norm2.weight]
Loading weights:  87%|████████▋ | 683/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]      
Loading weights:  87%|████████▋ | 683/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.bias]
Loading weights:  87%|████████▋ | 684/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 684/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc1.weight]
Loading weights:  87%|████████▋ | 685/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]  
Loading weights:  87%|████████▋ | 685/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.bias]
Loading weights:  87%|████████▋ | 686/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 686/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.mlp.fc2.weight]
Loading weights:  87%|████████▋ | 687/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.bias]
Loading weights:  87%|████████▋ | 687/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.bias]
Loading weights:  87%|████████▋ | 688/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.weight]
Loading weights:  87%|████████▋ | 688/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.projection.weight]
Loading weights:  87%|████████▋ | 689/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.bias]         
Loading weights:  87%|████████▋ | 689/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.bias]
Loading weights:  88%|████████▊ | 690/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.weight]
Loading weights:  88%|████████▊ | 690/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.3.self_attn.qkv.weight]
Loading weights:  88%|████████▊ | 691/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]    
Loading weights:  88%|████████▊ | 691/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.bias]
Loading weights:  88%|████████▊ | 692/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  88%|████████▊ | 692/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm1.weight]
Loading weights:  88%|████████▊ | 693/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]  
Loading weights:  88%|████████▊ | 693/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.bias]
Loading weights:  88%|████████▊ | 694/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  88%|████████▊ | 694/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.layer_norm2.weight]
Loading weights:  88%|████████▊ | 695/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]      
Loading weights:  88%|████████▊ | 695/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.bias]
Loading weights:  88%|████████▊ | 696/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  88%|████████▊ | 696/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc1.weight]
Loading weights:  88%|████████▊ | 697/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]  
Loading weights:  88%|████████▊ | 697/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.bias]
Loading weights:  89%|████████▊ | 698/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  89%|████████▊ | 698/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.mlp.fc2.weight]
Loading weights:  89%|████████▊ | 699/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.bias]
Loading weights:  89%|████████▊ | 699/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.bias]
Loading weights:  89%|████████▉ | 700/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.weight]
Loading weights:  89%|████████▉ | 700/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.projection.weight]
Loading weights:  89%|████████▉ | 701/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.bias]         
Loading weights:  89%|████████▉ | 701/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.bias]
Loading weights:  89%|████████▉ | 702/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.weight]
Loading weights:  89%|████████▉ | 702/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.4.self_attn.qkv.weight]
Loading weights:  89%|████████▉ | 703/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]    
Loading weights:  89%|████████▉ | 703/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.bias]
Loading weights:  89%|████████▉ | 704/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  89%|████████▉ | 704/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm1.weight]
Loading weights:  89%|████████▉ | 705/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]  
Loading weights:  89%|████████▉ | 705/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.bias]
Loading weights:  90%|████████▉ | 706/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  90%|████████▉ | 706/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.layer_norm2.weight]
Loading weights:  90%|████████▉ | 707/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]      
Loading weights:  90%|████████▉ | 707/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.bias]
Loading weights:  90%|████████▉ | 708/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  90%|████████▉ | 708/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc1.weight]
Loading weights:  90%|████████▉ | 709/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]  
Loading weights:  90%|████████▉ | 709/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.bias]
Loading weights:  90%|█████████ | 710/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  90%|█████████ | 710/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.mlp.fc2.weight]
Loading weights:  90%|█████████ | 711/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.bias]
Loading weights:  90%|█████████ | 711/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.bias]
Loading weights:  90%|█████████ | 712/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.weight]
Loading weights:  90%|█████████ | 712/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.projection.weight]
Loading weights:  90%|█████████ | 713/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.bias]         
Loading weights:  90%|█████████ | 713/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.bias]
Loading weights:  91%|█████████ | 714/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.weight]
Loading weights:  91%|█████████ | 714/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.5.self_attn.qkv.weight]
Loading weights:  91%|█████████ | 715/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]    
Loading weights:  91%|█████████ | 715/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.bias]
Loading weights:  91%|█████████ | 716/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  91%|█████████ | 716/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm1.weight]
Loading weights:  91%|█████████ | 717/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]  
Loading weights:  91%|█████████ | 717/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.bias]
Loading weights:  91%|█████████ | 718/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  91%|█████████ | 718/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.layer_norm2.weight]
Loading weights:  91%|█████████ | 719/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]      
Loading weights:  91%|█████████ | 719/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.bias]
Loading weights:  91%|█████████▏| 720/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  91%|█████████▏| 720/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc1.weight]
Loading weights:  91%|█████████▏| 721/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]  
Loading weights:  91%|█████████▏| 721/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.bias]
Loading weights:  92%|█████████▏| 722/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  92%|█████████▏| 722/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.mlp.fc2.weight]
Loading weights:  92%|█████████▏| 723/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.bias]
Loading weights:  92%|█████████▏| 723/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.bias]
Loading weights:  92%|█████████▏| 724/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.weight]
Loading weights:  92%|█████████▏| 724/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.projection.weight]
Loading weights:  92%|█████████▏| 725/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.bias]         
Loading weights:  92%|█████████▏| 725/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.bias]
Loading weights:  92%|█████████▏| 726/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.weight]
Loading weights:  92%|█████████▏| 726/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.6.self_attn.qkv.weight]
Loading weights:  92%|█████████▏| 727/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]    
Loading weights:  92%|█████████▏| 727/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.bias]
Loading weights:  92%|█████████▏| 728/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  92%|█████████▏| 728/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm1.weight]
Loading weights:  93%|█████████▎| 729/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]  
Loading weights:  93%|█████████▎| 729/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.bias]
Loading weights:  93%|█████████▎| 730/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  93%|█████████▎| 730/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.layer_norm2.weight]
Loading weights:  93%|█████████▎| 731/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]      
Loading weights:  93%|█████████▎| 731/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.bias]
Loading weights:  93%|█████████▎| 732/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  93%|█████████▎| 732/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc1.weight]
Loading weights:  93%|█████████▎| 733/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]  
Loading weights:  93%|█████████▎| 733/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.bias]
Loading weights:  93%|█████████▎| 734/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  93%|█████████▎| 734/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.mlp.fc2.weight]
Loading weights:  93%|█████████▎| 735/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.bias]
Loading weights:  93%|█████████▎| 735/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.bias]
Loading weights:  93%|█████████▎| 736/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.weight]
Loading weights:  93%|█████████▎| 736/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.projection.weight]
Loading weights:  94%|█████████▎| 737/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.bias]         
Loading weights:  94%|█████████▎| 737/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.bias]
Loading weights:  94%|█████████▎| 738/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.weight]
Loading weights:  94%|█████████▎| 738/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.7.self_attn.qkv.weight]
Loading weights:  94%|█████████▍| 739/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]    
Loading weights:  94%|█████████▍| 739/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.bias]
Loading weights:  94%|█████████▍| 740/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  94%|█████████▍| 740/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm1.weight]
Loading weights:  94%|█████████▍| 741/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]  
Loading weights:  94%|█████████▍| 741/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.bias]
Loading weights:  94%|█████████▍| 742/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  94%|█████████▍| 742/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.layer_norm2.weight]
Loading weights:  94%|█████████▍| 743/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]      
Loading weights:  94%|█████████▍| 743/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.bias]
Loading weights:  94%|█████████▍| 744/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  94%|█████████▍| 744/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc1.weight]
Loading weights:  95%|█████████▍| 745/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]  
Loading weights:  95%|█████████▍| 745/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.bias]
Loading weights:  95%|█████████▍| 746/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 746/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.mlp.fc2.weight]
Loading weights:  95%|█████████▍| 747/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.bias]
Loading weights:  95%|█████████▍| 747/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.bias]
Loading weights:  95%|█████████▍| 748/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.weight]
Loading weights:  95%|█████████▍| 748/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.projection.weight]
Loading weights:  95%|█████████▌| 749/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.bias]         
Loading weights:  95%|█████████▌| 749/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.bias]
Loading weights:  95%|█████████▌| 750/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.weight]
Loading weights:  95%|█████████▌| 750/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.8.self_attn.qkv.weight]
Loading weights:  95%|█████████▌| 751/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]    
Loading weights:  95%|█████████▌| 751/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.bias]
Loading weights:  95%|█████████▌| 752/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  95%|█████████▌| 752/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm1.weight]
Loading weights:  96%|█████████▌| 753/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]  
Loading weights:  96%|█████████▌| 753/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.bias]
Loading weights:  96%|█████████▌| 754/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  96%|█████████▌| 754/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.layer_norm2.weight]
Loading weights:  96%|█████████▌| 755/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]      
Loading weights:  96%|█████████▌| 755/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.bias]
Loading weights:  96%|█████████▌| 756/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  96%|█████████▌| 756/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc1.weight]
Loading weights:  96%|█████████▌| 757/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]  
Loading weights:  96%|█████████▌| 757/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.bias]
Loading weights:  96%|█████████▌| 758/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  96%|█████████▌| 758/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.mlp.fc2.weight]
Loading weights:  96%|█████████▋| 759/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.bias]
Loading weights:  96%|█████████▋| 759/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.bias]
Loading weights:  96%|█████████▋| 760/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.weight]
Loading weights:  96%|█████████▋| 760/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.projection.weight]
Loading weights:  97%|█████████▋| 761/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.bias]         
Loading weights:  97%|█████████▋| 761/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.bias]
Loading weights:  97%|█████████▋| 762/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.weight]
Loading weights:  97%|█████████▋| 762/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.9.self_attn.qkv.weight]
Loading weights:  97%|█████████▋| 763/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]   
Loading weights:  97%|█████████▋| 763/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.bias]
Loading weights:  97%|█████████▋| 764/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  97%|█████████▋| 764/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm1.weight]
Loading weights:  97%|█████████▋| 765/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]  
Loading weights:  97%|█████████▋| 765/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.bias]
Loading weights:  97%|█████████▋| 766/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  97%|█████████▋| 766/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.layer_norm2.weight]
Loading weights:  97%|█████████▋| 767/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]      
Loading weights:  97%|█████████▋| 767/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.bias]
Loading weights:  97%|█████████▋| 768/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  97%|█████████▋| 768/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc1.weight]
Loading weights:  98%|█████████▊| 769/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]  
Loading weights:  98%|█████████▊| 769/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.bias]
Loading weights:  98%|█████████▊| 770/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  98%|█████████▊| 770/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.mlp.fc2.weight]
Loading weights:  98%|█████████▊| 771/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.bias]
Loading weights:  98%|█████████▊| 771/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.bias]
Loading weights:  98%|█████████▊| 772/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.weight]
Loading weights:  98%|█████████▊| 772/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.projection.weight]
Loading weights:  98%|█████████▊| 773/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.bias]         
Loading weights:  98%|█████████▊| 773/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.bias]
Loading weights:  98%|█████████▊| 774/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.weight]
Loading weights:  98%|█████████▊| 774/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.10.self_attn.qkv.weight]
Loading weights:  98%|█████████▊| 775/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]    
Loading weights:  98%|█████████▊| 775/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.bias]
Loading weights:  98%|█████████▊| 776/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  98%|█████████▊| 776/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm1.weight]
Loading weights:  99%|█████████▊| 777/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]  
Loading weights:  99%|█████████▊| 777/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.bias]
Loading weights:  99%|█████████▊| 778/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  99%|█████████▊| 778/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.layer_norm2.weight]
Loading weights:  99%|█████████▉| 779/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]      
Loading weights:  99%|█████████▉| 779/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.bias]
Loading weights:  99%|█████████▉| 780/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 780/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc1.weight]
Loading weights:  99%|█████████▉| 781/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]  
Loading weights:  99%|█████████▉| 781/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.bias]
Loading weights:  99%|█████████▉| 782/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 782/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.mlp.fc2.weight]
Loading weights:  99%|█████████▉| 783/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.bias]
Loading weights:  99%|█████████▉| 783/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.bias]
Loading weights:  99%|█████████▉| 784/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.weight]
Loading weights:  99%|█████████▉| 784/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.projection.weight]
Loading weights: 100%|█████████▉| 785/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.bias]         
Loading weights: 100%|█████████▉| 785/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.bias]
Loading weights: 100%|█████████▉| 786/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.weight]
Loading weights: 100%|█████████▉| 786/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.encoder.layers.11.self_attn.qkv.weight]
Loading weights: 100%|█████████▉| 787/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.post_layernorm.bias]                   
Loading weights: 100%|█████████▉| 787/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.post_layernorm.bias]
Loading weights: 100%|██████████| 788/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 788/788 [00:00<00:00, 1089.00it/s, Materializing param=vision_model.post_layernorm.weight]
Loading weights: 100%|██████████| 788/788 [00:00<00:00, 4171.99it/s, Materializing param=vision_model.post_layernorm.weight]
+The tied weights mapping and config for this model specifies to tie text_decoder.bert.embeddings.word_embeddings.weight to text_decoder.cls.predictions.decoder.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
+BlipForQuestionAnswering LOAD REPORT from: Salesforce/blip-vqa-base
+Key                                       | Status     |  | 
+------------------------------------------+------------+--+-
+text_encoder.embeddings.position_ids      | UNEXPECTED |  | 
+text_decoder.bert.embeddings.position_ids | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+  ✓ BLIP (captioning + VQA)
+  Loaded 100 COCO samples
+
+📦 COCO Detection Benchmark
+----------------------------------------
+  Progress: 0/100
+  [Adapter] Creating vision adapter: 1792 -> 2048
+  Progress: 50/100
+  Mean IoU: 0.1759
+  Accuracy (IoU>0.5 + correct class): 0.0272
+  Loaded 0 counting samples
+
+🔢 Counting Benchmark
+----------------------------------------
+  Exact Accuracy: 0.00%
+  Within-1 Accuracy: 0.00%
+  Mean Absolute Error: 0.00
+  Loaded 0 VQA samples
+
+❓ VQA Benchmark
+----------------------------------------
+  Accuracy: 0.00% (0/0)
+
+======================================================================
+📊 BENCHMARK SUMMARY
+======================================================================
+
+coco_detection:
+  mean_iou: 0.1759
+  accuracy: 0.0272
+  num_samples: 100
+
+counting:
+  exact_accuracy: 0.0000
+  within_one_accuracy: 0.0000
+  mae: 0.0000
+  total: 0
+
+vqa:
+  accuracy: 0.0000
+  correct: 0
+  total: 0
+
+💾 Results saved to: checkpoints/oculus_detection_v2/final/benchmark_results.json