bert模型训练,加载保存的模型Can‘t load tokenizer for ‘/content/drive/MyDrive/Colab Notebooks/classification_mode

# 5.保存训练好的模型参数
import os
model.save_pretrained("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/")
tokenizer.save_pretrained("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/")

torch.save(args,os.path.join("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/","training_args.bin"))

# 1. 载入训练好的模型
args_eval={"model_name_or_path": "/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/",
    "config_name": "/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/",
    "tokenizer_name": "/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/",
      }

config_class, tokenizer_class = MODEL_CLASSES["bert"]
model_class=BertForClassification


config = config_class.from_pretrained(
    args_eval["config_name"],
    finetuning_task="", 
    cache_dir=None,
)
tokenizer = tokenizer_class.from_pretrained(
    args_eval["tokenizer_name"],
    do_lower_case=True,
    cache_dir=None,
)
model = model_class.from_pretrained(
    args_eval["model_name_or_path"],
    from_tf=bool(".ckpt" in args_eval["model_name_or_path"]),
    config=config,
    cache_dir=None,
)


model.to(device)


OSError                                   Traceback (most recent call last)
<ipython-input-57-4658f3d30e04> in <module>()
     17     args_eval["tokenizer_name"],
     18     do_lower_case=True,
---> 19     cache_dir=None,
     20 )
     21 model = model_class.from_pretrained(

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1766         if all(full_file_name is None for full_file_name in resolved_vocab_files.values()):
   1767             raise EnvironmentError(
-> 1768                 f"Can't load tokenizer for '{pretrained_model_name_or_path}'. If you were trying to load it from "
   1769                 "'https://huggingface.co/models', make sure you don't have a local directory with the same name. "
   1770                 f"Otherwise, make sure '{pretrained_model_name_or_path}' is the correct path to a directory "

OSError: Can't load tokenizer for '/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/' is the correct path to a directory containing all relevant files for a BertTokenizer tokenizer.

哈喽!

我用Bert预测评论分数,训练好模型保存到文件夹后,再一次加载它出现了上述错误,不太明白为什么,请教各位!

这是第二次训练这模型,也就是说我训练了一次之后,再把第一次训练的模型用新的数据训练,提高它的精确度,是可以的把,为什么第一次训练没有报错,第二次就报错了呢,我只改了保存的路径

文章出处登录后可见!

已经登录?立即刷新

共计人评分,平均

到目前为止还没有投票!成为第一位评论此文章。

(0)
青葱年少的头像青葱年少普通用户
上一篇 2023年7月6日
下一篇 2023年7月6日

相关推荐