android - Non-blocking loading and copying of large Texture2D's in C# for Unity

Question

I'm building a Unity app for Android which deals with loading a lot of large textures dynamically (all images are over 6MB in size as png's). These textures can either come from an Amazon S3 server, in which case they arrive as a stream, or from the user's device itself.

In both cases I'm able to get hold of the raw data or texture asynchronously without a problem. In the first I query the server and get a callback with the stream of data, and in the second I use the WWW class to get hold of the texture making use of the "file://" protocol.

The problem happens as soon as I want to copy this data into a Texture2D to some place I can make use of, such as onto a Texture2D private member.

With the stream I convert it into a byte[] and try calling LoadImage(), and with the WWW class I simply try copying it with myTexture = www.texture. Both times I get a massive frame out as the texture is loaded or copied. I want to eradicate this frame out because the App is simply un-shippable with it.

using (var stream = responseStream)
{
   byte[] myBinary = ToByteArray(stream);
   m_myTexture.LoadImage(myBinary);  // Commenting this line removes frame out
}

...

WWW www = new WWW("file://" + filePath);
yield return www;
m_myTexture = www.texture;  // Commenting this line removes frame out

Unfortunately Unity doesn't seem to like running these operations on a separate thread from the main thread and throws an exception when I try.

Is there any way to perhaps chunk up these operations so that it takes multiple frames? Or do some sort of fast memcopy operation that won't stall the main thread?

Thanks in advance!

PS: I've created a working example of the problem in the following repo: https://github.com/NeoSouldier/Texture2DTest/

Answer 1

The www.texture is known to cause hiccups when large Texture is downloaded.

Things you should try:

1.Use the WWW's LoadImageIntoTexture function which replaces the contents of an existing Texture2D with an image from the downloaded data. Keep reading if problem is still not solved.

WWW www = new WWW("file://" + filePath);
yield return www;
///////m_myTexture = www.texture;  // Commenting this line removes frame out
www.LoadImageIntoTexture(m_myTexture);

2.Use the www.textureNonReadable variable

Using www.textureNonReadable instead of www.texture can also speed up your loading time. I'be seen instances of this happening from time to time.

3.Use the function Graphics.CopyTexture to copy from one Texture to another. This should be fast. Continue reading if problem is still not solved.

//Create new Empty texture with size that matches source info
m_myTexture = new Texture2D(www.texture.width, www.texture.height, www.texture.format, false);
Graphics.CopyTexture(www.texture, m_myTexture);

4.Use Unity's UnityWebRequest API. This replaced the WWW class. You must have Unity 5.2 and above in order to use this. It has GetTexture function that is optimized for downloading textures.

using (UnityWebRequest www = UnityWebRequest.GetTexture("http://www.my-server.com/image.png"))
{
    yield return www.Send();
    if (www.isError)
    {
        Debug.Log(www.error);
    }
    else
    {
        m_myTexture = DownloadHandlerTexture.GetContent(www);
    }
}

If the three options above did not solve the freezing problem, another solution is copying the pixels one by one in a coroutine function with the GetPixel and SetPixel functions. You add a counter and set when you want it to wait. It spaced the Texture copying over time.

5.Copy Texture2D pixels one by one with the GetPixel and SetPixel functions. The example code includes 8K texture from Nasa for testing purposes. It won't block while copying the Texture. If it does, decrease the value of the LOOP_TO_WAIT variable in the copyTextureAsync function. You also have option to provide a function that will be called when this is done copying the Texture.

public Texture2D m_myTexture;

void Start()
{
    //Application.runInBackground = true;
    StartCoroutine(downloadTexture());
}

IEnumerator downloadTexture()
{
    //http://visibleearth.nasa.gov/view.php?id=79793
    //http://eoimages.gsfc.nasa.gov/images/imagerecords/79000/79793/city_lights_africa_8k.jpg

    string url = "http://eoimages.gsfc.nasa.gov/images/imagerecords/79000/79793/city_lights_africa_8k.jpg";
    //WWW www = new WWW("file://" + filePath);
    WWW www = new WWW(url);
    yield return www;

    //m_myTexture = www.texture;  // Commenting this line removes frame out

    Debug.Log("Downloaded Texture. Now copying it");

    //Copy Texture to m_myTexture WITHOUT callback function
    //StartCoroutine(copyTextureAsync(www.texture));

    //Copy Texture to m_myTexture WITH callback function
    StartCoroutine(copyTextureAsync(www.texture, false, finishedCopying));
}


IEnumerator copyTextureAsync(Texture2D source, bool useMipMap = false, System.Action callBack = null)
{

    const int LOOP_TO_WAIT = 400000; //Waits every 400,000 loop, Reduce this if still freezing
    int loopCounter = 0;

    int heightSize = source.height;
    int widthSize = source.width;

    //Create new Empty texture with size that matches source info
    m_myTexture = new Texture2D(widthSize, heightSize, source.format, useMipMap);

    for (int y = 0; y < heightSize; y++)
    {
        for (int x = 0; x < widthSize; x++)
        {
            //Get color/pixel at x,y pixel from source Texture
            Color tempSourceColor = source.GetPixel(x, y);

            //Set color/pixel at x,y pixel to destintaion Texture
            m_myTexture.SetPixel(x, y, tempSourceColor);

            loopCounter++;

            if (loopCounter % LOOP_TO_WAIT == 0)
            {
                //Debug.Log("Copying");
                yield return null; //Wait after every LOOP_TO_WAIT 
            }
        }
    }
    //Apply changes to the Texture
    m_myTexture.Apply();

    //Let our optional callback function know that we've done copying Texture
    if (callBack != null)
    {
        callBack.Invoke();
    }
}

void finishedCopying()
{
    Debug.Log("Finished Copying Texture");
    //Do something else
}