video - FFmpeg splitting large files

Question

I need to split a large video file into multiple pieces quickly and without files with errors. The basic idea is, I have a 2GB video file which I want to change to multiple formats. I have read some encoders can not make use of more than 1 core when encoding to a different format. So I had the idea to split the large file (which is very quick) into 4/8 pieces depending on how many cores I have available on individual servers. re-encode each piece to a new format and use these to display video in sequence.

so

testfile.mp4

becomes

1. testfile_piece_1.mp4
2. testfile_piece_2.mp4
3. testfile_piece_3.mp4
4. testfile_piece_4.mp4

these can then be individually be converted.

The solution should not be format specific. However I have found issues with mp4 files.

I have tried the command below, which works really well and fast but creates files with errors.

ffmpeg -i testfile.mp4 -ss 00:00:00 -t 00:20:00 -c copy testfile_piece_1.mp4

When I play the testfile_piece_1.mp4 on VLC it works fine. An issue arises when converting the split file to a different height and width mp4 file. I would get an error similar to "moov atom not found"

I tried adding -movflags faststart with no luck

I then came across this library https://code.google.com/archive/p/moovrelocator/ which fixed the moov issue but I would then get an error with regards to aac "Error while opening encoder for output stream #0.0 - maybe incorrect parameters such as bit_rate, rate, width or height"

The other way of splitting the files is useless but involved re-encoding the file. not too bad for smaller file size but the 2GB file would probably take days to complete.

Is there a way to split the largr file quickly without producing files with errors? I have been working on it for days with no luck.

Console output for comment - FFmpeg splitting large files

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/usr/share/nginx/html/uploads/testfile01.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.36.100
  Duration: 00:05:02.08, start: 302.120000, bitrate: 3254 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709)                                  , 1920x1080 [SAR 1:1 DAR 16:9], 3252 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (de                                  fault)
    Metadata:
      handler_name    : VideoHandler
[libx264 @ 0x165ffc0] width not divisible by 2 (853x480)
Output #0, mp4, to '/usr/share/nginx/html/uploads/testfile01_480.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.36.100
    Stream #0:0(und): Video: h264, none, q=2-31, 128 kb/s, SAR 2560:2559 DAR 0:0                                  , 25 fps (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.41.100 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters                                   such as bit_rate, rate, width or height

console output 2

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/usr/share/nginx/html/uploads/testfile.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    creation_time   : 2016-01-24 04:26:37
  Duration: 01:15:58.08, start: 0.000000, bitrate: 3163 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 3161 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2016-01-24 04:26:37
      handler_name    : VideoHandler
[segment @ 0x1197060] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, segment, to '/usr/share/nginx/html/uploads/testfile%02d.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    encoder         : Lavf56.36.100
    Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 3161 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2016-01-24 04:26:37
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers

Answer 1

Use the segment muxer to break the input into segments:

ffmpeg -i testfile.mp4 -c copy -f segment -segment_time 1200 testfile_piece_%02d.mp4

This will split the source at keyframes, so segments may not be exactly 1200 seconds long. And the timestamps aren't reset, so some players will fail to play the 2nd and latter segments. If playability is needed, insert -reset_timestamps 1.

After the parallel encoding, you can stitch the generated segments by first creating a text file seg.txt like this

file 'encoded_testfile_piece_00.mp4'
file 'encoded_testfile_piece_01.mp4'
file 'encoded_testfile_piece_02.mp4'
file 'encoded_testfile_piece_03.mp4'

And then running

ffmpeg -f concat -i seg.txt -c copy -fflags +genpts encoded_full.mp4