NISHIO Hirokazu[Translate]
TPUからGoogle ColabのローカルFSには書き込めない
from 日本語BERTのfine-tuning
BERTのfine-tuningが少し走って死んだ:

エラーメッセージ: Unsuccessful TensorSliceReader constructor: Failed to get matching files on /content/gdrive/My Drive/bert-wiki-ja/model.ckpt-1400000: Unimplemented: File system scheme '[local]' not implemented

TPUからGoogle ColabのローカルFSには書き込めないことが原因
>Local file system is not supported on TPU. You need to use Google Storage.
https://github.com/google-research/bert/issues/445

結果の書き出し先としてGCP Bucketが必要

Related Pages

  • →松尾研サマースクール2023大規模言語モデル×please_provide_an_offload_folder×熊野古道2023×1gpu_+_device_map='auto'_の罠×device_map='auto'×hf_device_map×lightblue/openorca×thebloke/stablebeluga2-70b-gptq×xlsum×nvidia_l4×google_colab×runpod→
  • 松尾研LLMサマースクールのコンペ
"Engineer's way of creating knowledge" the English version of my book is now available on [Engineer's way of creating knowledge]
(C)NISHIO Hirokazu / Converted from [Scrapbox] at [Edit]