Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
181 views
in Technique[技术] by (71.8m points)

Python MemoryError when uploading big video through YouTube Data API

Getting MemoryError while trying to upload a large video file to YouTube. When uploading smaller file it is okay but when the file is bigger I get MemoryError. I guess it's because code tries to load video file to RAM. Is there any way to upload a video without loading it to RAM.

import os
from googleapiclient.http import MediaFileUpload
import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors

scopes = ["https://www.googleapis.com/auth/youtube.upload"]


def main():
    os.environ["OAUTHLIB_INSECURE_TRANSPORT"] = "0"

    client_secret_file = 'client_credentials.json'
    api_name = 'youtube'
    api_version = 'v3'

    flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(client_secret_file, scopes)
    credentials = flow.run_console()
    youtube = googleapiclient.discovery.build(api_name, api_version, credentials=credentials)

    request = youtube.videos().insert(
        part='snippet,status',
        body={
            'snippet': {
                'categoryId': "20",
                'title': 'test title',
                'tags': ['tag1', 'tag2']
            },
            'status': {
                'privacyStatus': 'private'
            }
        },
        media_body=MediaFileUpload(File Path)
    )

    response = request.execute()

    print(response)


if __name__ == '__main__':
    main()
question from:https://stackoverflow.com/questions/65833755/python-memoryerror-when-uploading-big-video-through-youtube-data-api

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The reason for your MemoryError exception is the following: your input video file gets loaded in memory in its entirety prior to be processed for to be sent off to the remote service:

def createMethod(methodName, methodDesc, rootDesc, schema):
    ...
    def method(self, **kwargs):
        ...
        if media_filename:
            ...
            if media_upload.resumable():
                ...
            else:
                # A non-resumable upload
                if body is None:
                    ...
                else:
                    ...
                    payload = media_upload.getbytes(0, media_upload.size())
                    ...

Instead of uploading videos in one go, I'd very much recommend to use resumable uploads by means of the time-tested public Google script upload_video.py. (This script has an official documentation too. To obtain a self-explanatory usage information page from the script just issue it with the command line option --help.)


If you're using Python 3 (your code suggests that) then you have to convert that script to Python 3, because it's written for Python 2. For this, see the section Patching upload_video.py of one answer of mine.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...