javascript - Upload file chunks sequentially using forkjoin and wait for response before firing next api call

Question

I have the following piece of code, basically i am trying to upload chunks of a large file to signed urls returned from a service..

I need to call the uploadcompleted endpoint only after all the PUT requests in the returned forkjoin have been completed, but the issue i am facing is that I am not able to wait for the chunks to be uploaded to the signed url.

What am i doing wrong here? Any help will greatly be appreciated. Thanks in advance.

filechunks(chunk: any, uploadUrls: string[], index) {
    let observableBatch = [];

    const headers: HttpHeaders = new HttpHeaders({
      'content-type': 'application/json'
    });
    observableBatch.push(this._http.put<any>(uploadUrls[index], chunk, { headers }))
    return forkJoin(observableBatch).pipe(delay(1000));
    // const createRequest = () => of(this._http.put<any>(uploadUrls[index], chunk, { headers }));
    // return forkJoin(createRequest).pipe(delay(1000));
  }

uploadfileinchunks(uniqueFileId: string, uploadId: string, chunks, uploadUrls): Observable<any> {
    let response = [];

    chunks.map( (chunk, i) => {
      let response = this.filechunks(chunk, uploadUrls, i).toPromise();
      console.log(response) 
    });


    const params = {
      some-id: some-id
    };

    return this._http.post<FileData>(
      `${somurl}/uploadcompleted`,
      {},
      {  {}, params }
    );
  }

question from:https://stackoverflow.com/questions/65946372/upload-file-chunks-sequentially-using-forkjoin-and-wait-for-response-before-firi

Answer 1

Interpretation:

Your code doesn't really make sense, so it's hard to figure out what you're even trying to do

filechunks(chunk: any, uploadUrls: string[], index) {
  const headers: HttpHeaders = new HttpHeaders({
    'content-type': 'application/json'
  });

  // observableBatch is an array of size 1. Why bother?
  const observableBatch = [];  
  observableBatch.push(
    this._http.put<any>(
      uploadUrls[index], 
      chunk, 
      { headers }
    )
  );

  // You're runnung forkJoin on a single observable? Again, why?
  return forkJoin(observableBatch).pipe(delay(1000));
}

uploadfileinchunks(uniqueFileId: string, uploadId: string, chunks, uploadUrls): Observable<any> {
  
  // Here, response is declared, but it's never used. You can delete this.
  let response = [];

  // chunks#map is a mapping function, but you're mapping every element to
  // null. Maybe what you mean is forEach?
  chunks.map( (chunk, i) => {
    // This a new response, this one is a promise. If you want the result
    // returned from filechunks, you'd need to resolve this promise
    // with response.then(LAMBDA) or await response
    // Also, mixing observables and promises is often a code smell.
    let response = this.filechunks(chunk, uploadUrls, i).toPromise();
    // You're logging a promise object here, not its resolved value
    console.log(response) 
  });

  // Not sure what this is?
  const params = {
    some-id: some-id
  };

  // call the uploadcompleted endpoint, but you haven't waited 
  // for your other calls to complete. That's what this question is about.
  return this._http.post<FileData>(
    `${somurl}/uploadcompleted`,
    {},
    {  {}, params }
  );
}

My best guess at what you meant to do:

I obviously can't test this and it's incomplete anyway as your question is pretty vague, but maybe this will lead you down the right path?

Remember that uploadfileinchunks returns an Observable and observables don't do anything until you subscribe. So wherever this code is called, you'll need to be sure to do that.

filechunks(chunk: any, uploadUrls: string[], index) {
  return this._http.put<any>(
    uploadUrls[index], 
    chunk, 
    { 
      headers: new HttpHeaders({
        'content-type': 'application/json'
      })
    }
  ).pipe(
    delay(1000)
  );
}

uploadfileinchunks(uniqueFileId: string, uploadId: string, chunks, uploadUrls): Observable<any> {

  return forkJoin(chunks.map(
    (chunk, i) => this.filechunks(chunk, uploadUrls, i)
  ).pipe(
    map(response => ({
      some-id: "some-id"
    })),
    switchMap(params => this._http.post<FileData>(
      `${somurl}/uploadcompleted`,
      {},
      {  {}, params }
    ))
  );
}