Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
357 views
in Technique[技术] by (71.8m points)

python 3.x - Loading the pre-trained model of torch and sentence_transformers when running in a docker container failing

I am getting below error while loading the pre-trained model of torch and sentence_transformers("distilbert-base-nli-stsb-mean-tokens") when trying to run in a docker container.

Error: Invalid value for '-A' / '--app': 
 Unable to load celery application.
 While trying to load the module app.celery the following error occurred:
 Traceback (most recent call last):
   File "/usr/local/lib/python3.8/site-packages/celery/bin/celery.py", line 53, in convert
     return find_app(value)
   File "/usr/local/lib/python3.8/site-packages/celery/app/utils.py", line 384, in find_app
     sym = symbol_by_name(app, imp=imp)
   File "/usr/local/lib/python3.8/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
     module = imp(module_name, package=package, **kwargs)
   File "/usr/local/lib/python3.8/site-packages/celery/utils/imports.py", line 100, in import_from_cwd
     return imp(module, package=package)
   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
   File "<frozen importlib._bootstrap_external>", line 783, in exec_module
   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
   File "/code/app.py", line 997, in <module>
     load_model()
   File "/code/app.py", line 255, in load_model
     embedder = SentenceTransformer('distilbert-base-nli-stsb-mean-tokens')
   File "/usr/local/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 48, in __init__
     os.makedirs(model_path, exist_ok=True)
   File "/usr/local/lib/python3.8/os.py", line 213, in makedirs
     makedirs(head, exist_ok=exist_ok)
   File "/usr/local/lib/python3.8/os.py", line 213, in makedirs
     makedirs(head, exist_ok=exist_ok)
   File "/usr/local/lib/python3.8/os.py", line 213, in makedirs
     makedirs(head, exist_ok=exist_ok)
   [Previous line repeated 1 more time]
   File "/usr/local/lib/python3.8/os.py", line 223, in makedirs
     mkdir(name, mode)
 PermissionError: [Errno 13] Permission denied: '/nonexistent'

Here it is saying permission denied error while creating the folder. But I have tried providing USER root in the Dockerfile. Stuck with this issue for long time. Please anyone help me here.

Updated: My Dockerfile:

FROM python:3.8.5-slim

WORKDIR /code

ENV ENVIRONMENT='LOCAL'
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN apt-get update && apt-get install -y sudo netcat apt-utils
RUN apt-get install -y python3-dev  build-essential python3-pip

COPY ./requirements_local.txt /code/requirements_local.txt
RUN pip install -r /code/requirements_local.txt

EXPOSE 8000
COPY . /code/

CMD [ "gunicorn", "app:app", "-b", "0.0.0.0:8000","--timeout","7200"]

Docker-compose:

services:
  web:
    build: 
      context: .
      dockerfile: ./Dockerfile.prod
    hostname: flaskapp
    env_file:
      - ./.env.prod
    links:
      - redis
      - celery
    depends_on:
      - redis
    volumes:
      - data:/code
      - type: bind
        source: /home/ubuntu/models
        target: /mnt/models

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...