Skip to content

[Question]: 关于使用昇腾npu运行paddlenlp文档中零样本分类示例代码时遇到错误 #6728

Description

@Walllllllllllllll

请提出你的问题

[ma-user work]$python3 paddlenlp_test.py
I0815 11:45:03.235680 6616 init.cc:239] ENV [CUSTOM_DEVICE_ROOT]=/home/ma-user/python3.9.17/lib/python3.9/site-packages/paddle_custom_device
I0815 11:45:03.235747 6616 init.cc:145] Try loading custom device libs from: [/home/ma-user/python3.9.17/lib/python3.9/site-packages/paddle_custom_device]
I0815 11:45:03.786727 6616 custom_device.cc:1112] Successed in loading custom runtime in lib: /home/ma-user/python3.9.17/lib/python3.9/site-packages/paddle_custom_device/libpaddle-custom-npu.so
I0815 11:45:03.804697 6616 custom_kernel.cc:76] Successed in loading 325 custom kernel(s) from loaded lib(s), will be used like native ones.
I0815 11:45:03.804880 6616 init.cc:157] Finished in LoadCustomDevice with libs_path: [/home/ma-user/python3.9.17/lib/python3.9/site-packages/paddle_custom_device]
I0815 11:45:03.804915 6616 init.cc:245] CustomDevice: npu, visible devices count: 1
/home/ma-user/python3.9.17/lib/python3.9/site-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
[2023-08-15 11:45:09,169] [ INFO] - Downloading model_state.pdparams from https://paddlenlp.bj.bcebos.com/taskflow/zero_shot_text_classification/utc-base/model_state.pdparams
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 450M/450M [00:27<00:00, 17.1MB/s]
[2023-08-15 11:45:38,717] [ INFO] - Downloading config.json from https://paddlenlp.bj.bcebos.com/taskflow/zero_shot_text_classification/utc-base/config.json
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 553/553 [00:00<00:00, 1.48MB/s]
[2023-08-15 11:45:38,899] [ INFO] - Downloading vocab.txt from https://paddlenlp.bj.bcebos.com/taskflow/zero_shot_text_classification/utc-base/vocab.txt
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 182k/182k [00:00<00:00, 770kB/s]
[2023-08-15 11:45:39,371] [ INFO] - Downloading special_tokens_map.json from https://paddlenlp.bj.bcebos.com/taskflow/zero_shot_text_classification/utc-base/special_tokens_map.json
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 414kB/s]
[2023-08-15 11:45:39,578] [ INFO] - Downloading tokenizer_config.json from https://paddlenlp.bj.bcebos.com/taskflow/zero_shot_text_classification/utc-base/tokenizer_config.json
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 198/198 [00:00<00:00, 343kB/s]
[2023-08-15 11:45:39,744] [ INFO] - We are using <class 'paddlenlp.transformers.ernie.tokenizer.ErnieTokenizer'> to load '/home/ma-user/work/paddle_model/taskflow/zero_shot_text_classification/utc-base'.
[2023-08-15 11:45:39,805] [ INFO] - Assigning ['[O-MASK]'] to the additional_special_tokens key of the tokenizer
[2023-08-15 11:45:39,809] [ INFO] - Loading configuration file /home/ma-user/work/paddle_model/taskflow/zero_shot_text_classification/utc-base/config.json
[2023-08-15 11:45:39,811] [ INFO] - Loading weights file /home/ma-user/work/paddle_model/taskflow/zero_shot_text_classification/utc-base/model_state.pdparams
[2023-08-15 11:45:40,878] [ INFO] - Loaded weights file from disk, setting weights to model.
[2023-08-15 11:46:05,027] [ INFO] - All model checkpoint weights were used when initializing UTC.

[2023-08-15 11:46:05,028] [ INFO] - All the weights of UTC were initialized from the model checkpoint at /home/ma-user/work/paddle_model/taskflow/zero_shot_text_classification/utc-base.
If your task is similar to the task the model of the checkpoint was trained on, you can already use UTC for predictions without further training.
[2023-08-15 11:46:05,046] [ INFO] - Converting to the inference model cost a little time.
Segmentation fault (core dumped)
[ma-user work]$Process ForkServerProcess-2:
Traceback (most recent call last):
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/ma-user/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 61, in wrapper
raise exp
File "/home/ma-user/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 58, in wrapper
func(*args, **kwargs)
File "/home/ma-user/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 268, in task_distribute
key, func_name, detail = resource_proxy[TASK_QUEUE].get()
File "", line 2, in get
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/managers.py", line 810, in _callmethod
kind, result = conn.recv()
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
/home/ma-user/python3.9.17/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 91 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

我的paddle框架是在官网给出的PaddleCustomDevice编译步骤编译的(python3.9.17,cann6.0.1),paddle
nlp是pip3安装的develop版本,然后运行环境是在华为ModelArts平台上,请问这个问题该怎么处理呢?

Metadata

Metadata

Assignees

Labels

questionFurther information is requestedtriage

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions