ios - AVAssetExportSession export fails non-deterministically with error: "Operation Stopped, NSLocalizedFailureReason=The video could not be composed."

Question

We add subtitles to a video recorded by the user, but the export by our AVAssetExportSession object fails non-deterministically: sometimes it works, and sometimes it doesn't. It's unclear even how to reproduce the error.

We noticed the asset tracks seem to get lost during export.

Before exporting, there are two tracks (one for audio, one for video) as expected. But checking the number of tracks for the same file URL in exportDidFinish shows 0 tracks. So something seems wrong with the export process.

Update: Commenting out exporter.videoComposition = mutableComposition fixes the error, but of course no transforms are applied to the video. So the problem seems to lie in creating AVMutableVideoComposition, which causes problems downstream during export. Documentation and tutorials on AVMutableVideoComposition are sparse, so even if you don't have a solution but could recommend sources for reference beyond Apple, that would be helpful.

Error:

Error Domain=AVFoundationErrorDomain Code=-11841 "Operation Stopped" UserInfo=0x170676e80 {NSLocalizedDescription=Operation Stopped, NSLocalizedFailureReason=The video could not be composed.}

Code:

    let videoAsset = AVURLAsset(URL: fileUrl, options: nil)
    let mixComposition = AVMutableComposition()
    let videoTrack = mixComposition.addMutableTrackWithMediaType(AVMediaTypeVideo, preferredTrackID: CMPersistentTrackID(kCMPersistentTrackID_Invalid))
    let audioTrack = mixComposition.addMutableTrackWithMediaType(AVMediaTypeAudio, preferredTrackID: CMPersistentTrackID(kCMPersistentTrackID_Invalid))

    let sourceVideoTrack = videoAsset.tracksWithMediaType(AVMediaTypeVideo)[0] as! AVAssetTrack
    let sourceAudioTrack = videoAsset.tracksWithMediaType(AVMediaTypeAudio)[0] as! AVAssetTrack
    videoTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), ofTrack: sourceVideoTrack, atTime: kCMTimeZero, error: nil)
    audioTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), ofTrack: sourceAudioTrack, atTime: kCMTimeZero, error: nil)

    // Create something mutable???
    // -- Create instruction
    let instruction = AVMutableVideoCompositionInstruction()
    instruction.timeRange = CMTimeRangeMake(kCMTimeZero, videoAsset.duration)
    let videoLayerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: sourceVideoTrack)
    instruction.layerInstructions = [videoLayerInstruction]

    let mutableComposition = AVMutableVideoComposition()
    //mutableComposition.renderSize = videoTrack.naturalSize
    mutableComposition.renderSize = CGSize(width: 320, height: 320)
    mutableComposition.frameDuration = CMTimeMake(1, 60)
    mutableComposition.instructions = [instruction]

    // Animate
    mutableComposition.animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videoLayer, inLayer: parentLayer)

    // -- Get path
    let fileName = "/editedVideo-(arc4random() % 10000).mp4"
    let allPaths = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)
    let docsPath = allPaths[0] as! NSString
    let exportPath = docsPath.stringByAppendingFormat(fileName)
    let exportUrl = NSURL.fileURLWithPath(exportPath as String)!

    println("Tracks before export: (mixComposition.tracks.count). File URL: (exportUrl)")

    // -- Remove old video?
    if NSFileManager.defaultManager().fileExistsAtPath(exportPath as String) {
        println("Deleting existing file
")
        NSFileManager.defaultManager().removeItemAtPath(exportPath as String, error: nil)
    }

    // -- Create exporter
    let exporter = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetHighestQuality)
    exporter.videoComposition = mutableComposition
    exporter.outputFileType = AVFileTypeMPEG4
    exporter.outputURL = exportUrl
    exporter.shouldOptimizeForNetworkUse = true

    // -- Export video
    exporter.exportAsynchronouslyWithCompletionHandler({
        self.exportDidFinish(exporter)
    })


func exportDidFinish(exporter: AVAssetExportSession) {
    println("Exported video with status: (getExportStatus(exporter))")

    // Save video to photo album
    let assetLibrary = ALAssetsLibrary()
    assetLibrary.writeVideoAtPathToSavedPhotosAlbum(exporter.outputURL, completionBlock: {(url: NSURL!, error: NSError!) in
        println("Saved video to album (exporter.outputURL)")
        if (error != nil) {
            println("Error saving video")
        }
    })

    // Check asset tracks
    let asset = AVAsset.assetWithURL(exporter.outputURL) as? AVAsset
    println("Tracks after export: (asset!.tracks.count). File URL: (exporter.outputURL)")
}

Questions:

1) What's causing the problem, and what's the solution?

2) Suggestions on how to reproduce the error consistently, which hopefully helps debug the problem?

Answer 1

What seems to be the cure is making sure the assetTrack parameter in AVMutableVideoCompositionLayerInstruction is not from the AVURLAsset object, but from the video object returned by addMutableTrackWithMediaType.

In other words, this line:

let videoLayerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: sourceVideoTrack)

Should be:

let videoLayerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack)

Argh. Hours of endless frustration because sometimes the first line worked, and sometimes it didn't.

Still would like to award the bounty to someone.

If you can explain why the first line failed non-deterministically, instead of every time, or provide a deeper tutorial into AVMutableComposition and its related classes -- for the purposes of adding text overlays to user-recorded videos -- the bounty is all yours. :)