c# - Play 360 Stereoscopic video with VideoPlayer

Question

I want to play a stereo 360 degree video in virtual reality in Unity on an Android. So far I have been doing some research and I have two cameras for the right and left eye with each a sphere around them. I also need a custom shader to make the image render on the inside of the sphere. I have the upper half of the image showing on one sphere by setting the y-tiling to 0.5 and the lower half shows on the other sphere with y-tiling 0.5 and y-offset 0.5. With this I can show a 3D 360 degree image already correct. The whole idea is from this tutorial.

Now for video, I need control over the Video speed so it turned out I need the VideoPlayer from the new Unity 5.6 beta. Now my setup so far would require the Video Player to play the video on both spheres with one sphere playing the upper part (one eye) and the other video playing the lower part (other eye).

Here is my problem: I don't know how to get the video Player to play the same video on two different materials (since they have different tiling values). Is there a way to do that?

I got a hint that I could use the same material and achieve the tiling effect via UV, but I don't know how that works and I haven't even got the video player to play the video on two objects using the same material on both of them. I have a screenshot of that here. The Right sphere just has the material videoMaterial. No tiling since I'd have to do that via UV.

Which way to go and how to do it? Am I on the right way here?

Answer 1

Am I on the right way here?

Almost but you are currently using Renderer and Material instead of RenderTexture and Material.

Which way to go and how to do it?

You need to use RenderTexture for this. Basically, you render the Video to RenderTexture then you assign that Texture to the material of both Spheres.

1.Create a RenderTexture and assign it to the VideoPlayer.

2.Create two materials for the spheres.

3.Set VideoPlayer.renderMode to VideoRenderMode.RenderTexture;

4.Set the Texture of both Spheres to the Texture from the RenderTexture

5.Prepare and Play Video.

The code below is doing that exact thing. It should work out of the box. The only thing you need to do is to modify the tiling and offset of each material to your needs.

You should also comment out:

leftSphere = createSphere("LeftEye", new Vector3(-5f, 0f, 0f), new Vector3(4f, 4f, 4f));
rightSphere = createSphere("RightEye", new Vector3(5f, 0f, 0f), new Vector3(4f, 4f, 4f));

then use a Sphere imported from any 3D application. That line of code is only there for testing purposes and it's not a good idea to play video with Unity's sphere because the spheres don't have enough details to make the video smooth.

using UnityEngine;
using UnityEngine.Video;

public class StereoscopicVideoPlayer : MonoBehaviour
{
    RenderTexture renderTexture;

    Material leftSphereMat;
    Material rightSphereMat;

    public GameObject leftSphere;
    public GameObject rightSphere;

    private VideoPlayer videoPlayer;

    //Audio
    private AudioSource audioSource;

    void Start()
    {
        //Create Render Texture
        renderTexture = createRenderTexture();

        //Create Left and Right Sphere Materials
        leftSphereMat = createMaterial();
        rightSphereMat = createMaterial();

        //Create the Left and Right Sphere Spheres
        leftSphere = createSphere("LeftEye", new Vector3(-5f, 0f, 0f), new Vector3(4f, 4f, 4f));
        rightSphere = createSphere("RightEye", new Vector3(5f, 0f, 0f), new Vector3(4f, 4f, 4f));

        //Assign material to the Spheres
        leftSphere.GetComponent<MeshRenderer>().material = leftSphereMat;
        rightSphere.GetComponent<MeshRenderer>().material = rightSphereMat;

        //Add VideoPlayer to the GameObject
        videoPlayer = gameObject.AddComponent<VideoPlayer>();

        //Add AudioSource
        audioSource = gameObject.AddComponent<AudioSource>();

        //Disable Play on Awake for both Video and Audio
        videoPlayer.playOnAwake = false;
        audioSource.playOnAwake = false;

        // We want to play from url
        videoPlayer.source = VideoSource.Url;
        videoPlayer.url = "http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4";

        //Set Audio Output to AudioSource
        videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;

        //Assign the Audio from Video to AudioSource to be played
        videoPlayer.EnableAudioTrack(0, true);
        videoPlayer.SetTargetAudioSource(0, audioSource);

        //Set the mode of output to be RenderTexture
        videoPlayer.renderMode = VideoRenderMode.RenderTexture;

        //Set the RenderTexture to store the images to
        videoPlayer.targetTexture = renderTexture;

        //Set the Texture of both Spheres to the Texture from the RenderTexture
        assignTextureToSphere();

        //Prepare Video to prevent Buffering
        videoPlayer.Prepare();

        //Subscribe to prepareCompleted event
        videoPlayer.prepareCompleted += OnVideoPrepared;
    }


    RenderTexture createRenderTexture()
    {

        RenderTexture rd = new RenderTexture(1024, 1024, 16, RenderTextureFormat.ARGB32);
        rd.Create();
        return rd;
    }

    Material createMaterial()
    {
        return new Material(Shader.Find("Specular"));
    }

    void assignTextureToSphere()
    {
        //Set the Texture of both Spheres to the Texture from the RenderTexture
        leftSphereMat.mainTexture = renderTexture;
        rightSphereMat.mainTexture = renderTexture;
    }

    GameObject createSphere(string name, Vector3 spherePos, Vector3 sphereScale)
    {
        GameObject sphere = GameObject.CreatePrimitive(PrimitiveType.Sphere);
        sphere.transform.position = spherePos;
        sphere.transform.localScale = sphereScale;
        sphere.name = name;
        return sphere;
    }

    void OnVideoPrepared(VideoPlayer source)
    {
        Debug.Log("Done Preparing Video");

        //Play Video
        videoPlayer.Play();

        //Play Sound
        audioSource.Play();

        //Change Play Speed
        if (videoPlayer.canSetPlaybackSpeed)
        {
            videoPlayer.playbackSpeed = 1f;
        }
    }
}

There is also Unity tutorial on how to do this with a special shader but this does not work for me and some other people. I suggest you use the method above until VR support is added to the VideoPlayer API.