Train a CausalLM for machine translation

[John6666]

Maybe unresolved issue? Or maybe it hasn’t been made into an issue on the transformers github.

There were missing keys in the checkpoint model loaded: ['proj_out.weight']....

Unable to load checkpoint after finetuning

Hi, So I’m running into trouble when trying to load the model weights after finetuning a pretrained base model, steps I’m taking: model= BertForSequenceClassification.from_pretrained("model/repo") model.classifier = torch.nn.Sequential( torch.nn.Dropout(0.4), torch.nn.Linear(model.config.hidden_size, 256), torch.nn.ReLU(), torch.nn.Dropout(0.4), torch.nn.Linear(256, 2), ) getting a warning here which I guess is fine for the moment: Some weights of the model checkpoint at…

Resuming training: There were missing keys in the checkpoint model loaded: ['lm_head.weight']

Hey all, Trying to learn a bit about training AI models, so I figured I would try to kind of recreate the “Nothing, Forever” AI but with a whatsapp friend group of mine as training set. However, the text set that i’m using is large (100k lines) so training is taking a while and I would like to interrupt it at times. It all seems to work fine, except when i resume a checkpoint I always see this message: There were missing keys in the checkpoint model loaded: ['lm_head.weight']. Should I be wo…