python - 404 Error using compose for Google Cloud Storage in Google Cloud Function

Question

I have several files I want to upload to Google Cloud Storage and then combine using compose. I am able to upload the files but cannot get compose to work. I originally tried it with Nodejs and got weird errors and now tried to use compose in a cloud function written in python. Here is the code I am using:

from google.cloud import storage

def compose_file(event, context):
    bucket_name = 'bucket-name-appspot.com'
    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)

    blob1 = bucket.blob('composettest/compose1.txt')
    blob2 = bucket.blob('composettest/compose2.txt')

    sources = [blob1, blob2]
    destination_blob_name = 'compose3.txt'
    print(blob1)
    print(blob2)

    destination = bucket.blob(destination_blob_name)
    print(destination)
    destination.content_type = "text/plain"
    destination.compose(sources)
    return destination

And this is the error I keep getting. I know the file exists because I can print it as a blob. Any ideas? Is there possibly some sort of permission error? I doubt it's that because I can create and download files no problem. I just can't get compose to work with either python or the nodejs client libraries.

  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 449, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 268, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 265, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 26, in compose_file
    destination.compose(sources)
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 3070, in compose
    retry=retry,
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/_http.py", line 63, in api_request
    return call()
  File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 438, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.NotFound: 404 POST https://storage.googleapis.com/storage/v1/b/bucket-name-appspot.com/o/compose3.txt/compose?prettyPrint=false: Not Found

question from:https://stackoverflow.com/questions/65851993/404-error-using-compose-for-google-cloud-storage-in-google-cloud-function

Answer 1

You can't compose in a new file. You take a file and you compose it with another one. In your case, either you first create a compose3.txt empty in GCS and then you can create your command. Or, you use the compose1 as destination and only the compose2 as source, and both are composed together.