actionscript 3 - AS3 NetStream AppendBytes Seek issue

Question

I'm having trouble with NetStream in AS3. The project I am working on allows users to browse a video (locally) and play it back. The issue I am having is that netStream.seek(0); from what I can tell it doesn't do anything, although I get inside a NetStatusEvent function and NetStream.Seek.Notify is triggered. I'm using NativeProcess and the following function is this makes any difference.

public function ProgressEventOutputHandler(e:ProgressEvent):void {
    videoByteArray = new ByteArray();
    nativeProcess.standardOutput.readBytes(videoByteArray, 0, nativeProcess.standardOutput.bytesAvailable);
    netStream.appendBytes(videoByteArray);
}

Am I missing something here? I am pausing netStream before using netStream.seek(0);.

EDIT:

In an attempt to fix this issue I followed the instructions by VC.One I've done the following:

• Moved videoByteArray = new ByteArray(); to my init function and also created tempVideoByteArray = new ByteArray(); in this function.

• Update my ProgressEventOutputHandler function so that it no longer created a new ByteArray for videoByteArray and changed this line - nativeProcess.standardOutput.readBytes(videoByteArray, videoByteArray.length, nativeProcess.standardOutput.bytesAvailable);

I have changed nothing else and now the video will not load. If I allow a new ByteArray to be created inside the ProgressEventOutputHandler function the video does load again.

Answer 1

Short Version :

Try the code I pasted here: Github Snippet link

Long version :

this one's kinda long but hope it helps once and for all... Don't worry about the brick wall thing, walls are made to be smashed. To keep you inspired, check out some in-house demos from the VC:One labs using appendBytes :

• MP4 Seeking Experiment : research for appendBytes frame data access and time/seek handling. Real-time frame bytes convert from MP4 to FLV format using only AS3 code.
• Speed Adjust of Audio & Video : for real-time MP3 audio in video separation & effecting experiment. Requires MP4/FLV file with MP3 data in the audio track.
• Synchronised Video Frames : for multiple videos displaying by the same frame number.

PS: I'll be using URLStream method as that's a more useful answer to those loading local or online files. You could change from urlstream.progressEvent to your usual nativeProcess.progressEvent.
I know FFMPEG but only used AIR for making Android apps. So for this AIR/FFMPEG connection you know more than me.

Also this answer assumes you're using FLV with MPEG H.264 video & MP3 or AAC audio.

ffmpeg -i input.mp4 -c:v copy -c:a mp3 -b:a 128k -ac 2 -ar 44100 FLV_with_MP3.flv

This assumption matters because it affects what kind of bytes we look for. In the case of the above FLV with a H.264 video and AAC or MP3 audio we can expect the following (when seeking) :

• Since this is MPEG the first video tag will hold AVC Decoder Config bytes and the first audio tag holds the Audio Specific Config bytes. This data is not actual media frames but simply packaged like an audio/video tag. These are needed for MPEG playback. The same bytes can be found in the STSD metadata entry (MOOV atom) inside an MP4 container. Now the next found video tag will (or should) be the video's actual first frame.
• Video keyframe : begins 0x09 and next 11th byte is 0x17 & 12th byte is 0x01
• Audio TAG AAC : begins 0x08 and next 11th byte is 0xAF & 12th byte is 0x01
• Audio TAG MP3 : begins 0x08 and next 11th byte is 0x2F & 12th byte is 0xFF

1) Bytes copying and checking values :

You are looking for bytes that represent a video "tag". Apart from the Metadata tag, you can now expect "tag" to mean a container of an audio or video frame. There are two ways to get tag bytes into your "temporary byte array" (we'll name it as temp_BA ).

• ReadBytes (slow) : extracts the individual byte values within a start/end range in source_BA
• WriteBytes (fast) : instant duplication of a start/end range of bytes from source_BA

Readbytes explained : tells Source to read its bytes into Target. Source will read forwards up to the length from its current offset (position). Go to correct Source position before reading onwards...

source_BA.readBytes( into Target_BA, Pos within Target_BA, length of bytes required );

After the above line executes, Source position will now have moved forward to account for the new length travelled. (formula : Source new Pos = previousPos + BytesLengthRequired).

Writebytes explained : tells Target to duplicate a range of bytes from Source. Is fast since copying from already-known information (from Source). Target writes onwards from its current position...

target_BA.writeBytes( from source_BA, Pos within source_BA, length of bytes required );

After the above line executes, note that both Source and Target positions are unchanged.

Use above methods to get required tag bytes into temp_BA from a specific source_BA.position = x.

To check any byte (its value), use the methods below to update some variable of int type:

• Read a one-byte value : use my_Integer = source_BA.readByte();
• Read a two-byte value : use my_Integer = source_BA.readUnsignedShort();
• Read a four-byte value : use my_Integer = source_BA.readUnsignedInt();
• variable Number for eight-byte value : use my_Number = source_BA.readDouble();

note : Don't confuse .readByte(); which extracts a numerical value (of byte) with the similar sounding .readBytes() which copies a chunk of bytes to another byte array.

2) Finding A Video KeyFrame (or I-frame) :

[ illustration image of Video TAG with Keyframe H264/AAC ]

To find a video keyframe

• From a starting offset, use a while loop to now travel [forward] through the bytes searching each byte for a one-byte value of "9" ( hex: 0x09), when found we check further ahead bytes to confirm that indeed it's a true keyframe and not just a random occurence of "9".
• In the case of H.264 video codec, at the correct "9" byte position (xPos) we expect the 11th & 12th bytes ahead always to be "17" and "01" respectively.
• If that is == true then we check the three Tag Size bytes and add 15 to this integer for the total length of bytes expected to be written from Source into Target ( temp_BA). We have added 15 to account for the 11 bytes before and also the 4 bytes after expected TAG DATA. These 4 bytes at tag ending are "Previous Tag Size" and this amount actually includes the 11 front bytes but not counting these end 4 bytes themselves.
• We tell temp_BA to write bytes of Source (your videoByteArray) starting from pos of "9" byte (xPos) for a length of "Tag Size" + 15. You have now extracted an MPEG keyframe.
  example : temp_BA.writeBytes( videoByteArray, int (xPos), int (TAG_size) );
• This temp_BA with tag of a Keyframe can now be appended using:
  example : netStream.appendBytes( temp_BA ); //displays a single frame

note : For reading 3 bytes of Tag Size I will show a custom converting bytes_toInt() function (since processors read either 1, 2 or 4 bytes at once for integers, reading 3 bytes here is an akward request).

Searching tip : Tags always follow each other in a trail. We can seek faster by also checking if bytes are for a non-keyframe (P frame) video tag or even some audio tag. If so then we check that particular tag size and now increment our xPos to jump this new length. This way we can skip by whole tag sizes not just by single bytes. Stopping only when we have a keyframe tag.

3) Playback From A Video KeyFrame :

When you think about it, play is simply like an auto-seek going on a frame by frame basis. Where the expected speed of getting each next frame is defined by the video's encoded framerate.

So your playback function can simply be a Timer that gets X-amount of video tags (frames) every second (or 1000 milisecs). You do that as example my_Timer = new Timer ( video_FPS ). When the timer runs and reaches each FPS slice of a second it will run the append_PLAY(); function which in turn runs a get_frame_Tag(); function.

• NS.seek(0) : Puts NetStream into "seek mode". (the number doesn't matter but must exist in the command). Any "ahead frames" buffer is cleared and they'll be no (image) frame updates until..
• RESET_SEEK : Ends the "seek mode" and now allows image updates. The first tag you append after using the RESET_SEEK command must be a tag with a video keyframe. (for audio-only this can be any tag since technically all audio tags are audio keyframes)
• END_SEQUENCE : (for MPEG H.264) Plays out any remaining "ahead frames" (drains the buffer). Once drained you can now append any type of video tag. Remember H.264 expects forward-moving timestamps, If you see f**ked up pixels then your next tag timestamp is wrong (too high or too low). If you appending just one frame (poster image?) you could use END_SEQUEMCE to drain the buffer and display that one frame (without waiting for buffer to fill up to x-amount of frames first)...

The play function acts as a middle-man function to manage things without cluttering the get frame function with If statements etc. Managing things means for example checking that there are enough bytes downloaded to even begin getting a frame according to Tag Size.

4) Source Code For A Working Example :

Code is too long.. see this link below: https://gist.github.com/Valerio-Charles-VC1/657054b773dba9ba1cbc

Hope it helps. VC