Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Post an Article
Welcome To Ask or Share your Answers For Others

Categories

javascript - How to use "segments" mode at SourceBuffer of MediaSource to render same result at Chomium, Chorme and Firefox?

0 votes
174 views
in Technique[技术] by (71.8m points)

javascript - How to use "segments" mode at SourceBuffer of MediaSource to render same result at Chomium, Chorme and Firefox?

After further development of the code at OP at How to use Blob URL, MediaSource or other methods to play concatenated Blobs of media fragments? have been able to achieve requirement of recording discrete media fragments using MediaRecorder, adding cues to the resulting webm file using ts-ebml and recording the discrete media fragments as a single media file using MediaSource with .mode of SourceBuffer set to "sequence" at both Chromium and Firefox browsers .

The Chromium issue at Monitor and potentially deprecate support for multitrack SourceBuffer support of 'sequence' AppendMode discusses "sequence" mode is being considered for deprecation for multitrack SourceBuffer objects. When asked OP of issue how to implement the code using "segments" .mode (default AppendMode of SourceBuffer) the response was essentially that "segments" mode also supports multitrack input at SourceBuffer.

However, when trying code with .mode of SourceBuffer set to "segments" Chromium 60 only plays approximately one second, the first buffer of multiple appended buffers, of an expected ten second playback of recorded media fragments having cues set at webm file which is converted to ArrayBuffer and passed to .appendBuffer(), while Firefox renders same result when .mode is set to either "sequence" and "segments".

Code which renders expected result at both Chromium and Firefox. Note, Firefox does not play .mp4 at <video> element if multipleUrls is tried, though Firefox does support playing .mp4 at MediaSource when proper media codec is set.

<!DOCTYPE html>
<html>
<!-- recordMediaFragments.js demo https://github.com/guest271314/recordMediaFragments/tree/master/demos 2017 guest271314 -->

<head>
  <!-- https://github.com/guest271314/recordMediaFragments/ts-ebml -->
</head>

<body>
  <video width="320" height="280" controls="true"></video>
  <script>
    (async() => {
      let request = await fetch("https://raw.githubusercontent.com/guest271314/recordMediaFragments/master/ts-ebml/ts-ebml-min.js");

      let blob = await request.blob();

      const script = document.createElement("script");

      document.head.appendChild(script);

      script.src = URL.createObjectURL(blob);

      script.onload = () => {

        const tsebml = require("ts-ebml");

        const video = document.querySelector("video");

        const videoStream = document.createElement("video");

        // `MediaSource`
        const mediaSource = new MediaSource();
        // for firefox 
        // see https://bugzilla.mozilla.org/show_bug.cgi?id=1259788
        const hasCaptureStream = HTMLMediaElement.prototype.hasOwnProperty("captureStream");

        // handle firefox and chromium
        const captureStream = mediaElement =>
          !!mediaElement.mozCaptureStream 
          ? mediaElement.mozCaptureStream() 
          : mediaElement.captureStream();

        let currentFragmentURL, currentBlobURL, fragments;

        videoStream.width = video.width;

        videoStream.height = video.height;

        const mimeCodec = "video/webm;codecs=vp8,opus";
        // set to `.currentTime` of `videoStream` at `pause`
        // to set next media fragment starting `.currentTime`
        // if URL to be set at `.src` has same origin and pathname
        let cursor = 0;

        // https://gist.github.com/jsturgis/3b19447b304616f18657
        // https://www.w3.org/2010/05/video/mediaevents.html
        const multipleUrls = [
          "https://media.w3.org/2010/05/sintel/trailer.mp4#t=0,5",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=55,60",
          "https://raw.githubusercontent.com/w3c/web-platform-tests/master/media-source/mp4/test.mp4#t=0,5",
          "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4#t=0,5",
          "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerJoyrides.mp4#t=0,5",
          "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerMeltdowns.mp4#t=0,6",
          "https://media.w3.org/2010/05/video/movie_300.mp4#t=30,36"
        ];

        const singleUrl = [
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=0,1",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=1,2",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=2,3",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=3,4",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=4,5",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=5,6",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=6,7",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=7,8",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=8,9",
          "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=9,10"
        ];

        const geckoUrl = [
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=10,11",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=11,12",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=12,13",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=13,14",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=14,15",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=15,16",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=16,17",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=17,18",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=18,19",
          "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=19,20"

        ];

        const mediaFragmentRecorder = async(urls) => {
          // `ts-ebml`
          const tsebmlTools = async() => ({
            decoder: new tsebml.Decoder(),
            encoder: new tsebml.Encoder(),
            reader: new tsebml.Reader(),
            tools: tsebml.tools
          });
          // create `ArrayBuffer` from `Blob`
          const readAsArrayBuffer = (blob) => {
            return new Promise((resolve, reject) => {
              const fr = new FileReader();
              fr.readAsArrayBuffer(blob);
              fr.onloadend = () => {
                resolve(fr.result);
              };
              fr.onerror = (ev) => {
                reject(ev.error);
              };
            });
          }
          // `urls`: string or array of URLs
          // record each media fragment
          const recordMediaFragments = async(video, mimeCodec, decoder, encoder, reader, tools, ...urls) => {
            urls = [].concat(...urls);
            const media = [];
            for (let url of urls) {
              await new Promise(async(resolve) => {

                let mediaStream, recorder;

                videoStream.onprogress = e => {
                  videoStream.onprogress = null;
                  console.log("loading " + url)
                }

                videoStream.oncanplay = async(e) => {

                  videoStream.oncanplay = null;
                  videoStream.play();

                  mediaStream = captureStream(videoStream);
                  console.log(mediaStream);

                  recorder = new MediaRecorder(mediaStream, {
                    mimeType: mimeCodec
                  });

                  recorder.ondataavailable = async(e) => {
                    // set metadata of recorded media fragment `Blob`
                    const mediaBlob = await setMediaMetadata(e.data);
                    // create `ArrayBuffer` of `Blob` of recorded media fragment
                    const mediaBuffer = await readAsArrayBuffer(mediaBlob);
                    const mediaDuration = videoStream.played.end(0) - videoStream.played.start(0);
                    const mediaFragmentId = currentFragmentURL || new URL(url);
                    const mediaFileName = mediaFragmentId.pathname.split("/").pop() + mediaFragmentId.hash;
                    const mediaFragmentType = "singleMediaFragment";
                    if (currentBlobURL) {
                      URL.revokeObjectURL(currentBlobURL);
                    }
                    media.push({
                      mediaBlob,
                      mediaBuffer,
                      mediaDuration,
                      mediaFragmentType,
                      mediaFileName
                    });
                    resolve();

                  }
                  recorder.start();
                }
                videoStream.onpause = e => {
                  videoStream.onpause = null;
                  cursor = videoStream.currentTime;
                  recorder.stop();
                  // stop `MediaStreamTrack`s
                  for (let track of mediaStream.getTracks()) {
                    track.stop();
                  }
                }
                currentFragmentURL = new URL(url);
                // for firefox to load cross origin media without silence 
                if (!hasCaptureStream) {
                  console.log(currentFragmentURL);
                  request = new Request(currentFragmentURL.href);
                  blob = await fetch(request).then(response => response.blob());
                  console.log(blob);
                  currentBlobURL = URL.createObjectURL(blob);
                  // set next media fragment URL to `.currentTime` at `pause` event
                  // of previous media fragment if `url` has same `origin` and `pathname`
                  if (urls.indexOf(currentFragmentURL.href) > 0 
                  && new URL(urls[urls.indexOf(currentFragmentURL.href) - 1]).origin === currentFragmentURL.origin 
                  && new URL(urls[urls.indexOf(currentFragmentURL.href) - 1]).pathname === currentFragmentURL.pathname) {
                    if (cursor > 0) {
                      url = url = currentBlobURL + currentFragmentURL.hash.replace(/=d+/, "=" + cursor);
                      console.log(url)
                    }
                  } else {
                    url = currentBlobURL + currentFragmentURL.hash;
                  }
                } else {
                  if (cursor > 0 
                  && new URL(urls[urls.indexOf(url)

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界，一个免费开放的程序员编程交流平台！开放，进步，分享！让技术改变生活，让极客改变未来！ Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

Just Browsing Browsing

1.4m articles

1.4m replys

5 comments

57.0k users

Most popular tags

Snow Theme by Q2A Market
Powered by Question2Answer
...