c# - How to convert new Canvas ui image position to linerenderer position?

Question

I have an image UI in a canvas with Screen Space - Camera render mode. What I like to do is move my LineRenderer to the image vertical position by looping through all the LineRenderer positions and changing its y axis. My problem is I cant get the correct position of the image that the LineRenderer can understand. I've tried using ViewportToWorldPoint and ScreenToWorldPoint but its not the same position.

    Vector3 val = Camera.main.ViewportToWorldPoint(new Vector3(image.transform.position.x, image.transform.position.y, Camera.main.nearClipPlane));

    for (int i = 0; i < newListOfPoints.Count; i++)
    {
        line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
    }

Screenshot result using Vector3 val = Camera.main.ScreenToWorldPoint(new Vector3(image.transform.localPosition.x, image.transform.localPosition.y, -10));

The green LineRenderer is the result of changing the y position. It should be at the bottom of the square image.

Answer 1

Wow, this was annoying and complicated.

Here's the code I ended up with. The code in your question is the bottom half of the Update() function. The only thing I changed is what was passed into the ScreenToWorldPoint() method. That value is calculated in the upper half of the Update() function.

The RectTransformToScreenSpace() function was adapted from this Unity Answer post¹ about getting the screen space coordinates of a RectTransform (which is exactly what we want in order to convert from screen space coordinates back into world space!) The only difference is that I was getting inverse Y values, so I changed from Screen.height - transform.position.y to just transform.position.y which did the trick perfectly.

After that it was just a matter of grabbing that rectangle's lower left corner, making it a Vector3 instead of a Vector2, and passing it back into ScreenToWorldPoint(). The only trick there was because of the perspective camera, I needed to know how far away the line was from the camera originally in order to maintain that same distance (otherwise the line moves up and down the screen faster than the image). For an orthographic camera, this value can be anything.

void Update () {
    //the new bits:
    float dist = (Camera.main.transform.position - newListOfPoints[0]).magnitude;
    Rect r = RectTransformToScreenSpace((RectTransform)image.transform);
    Vector3 v3 = new Vector3(r.xMin, r.yMin, dist);

    //more or less original code:
    Vector3 val = Camera.main.ScreenToWorldPoint(v3);
    for(int i = 0; i < newListOfPoints.Count; i++) {
        line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
    }
}

//helper function:
public static Rect RectTransformToScreenSpace(RectTransform transform) {
    Vector2 size = Vector2.Scale(transform.rect.size, transform.lossyScale);
    Rect rect = new Rect(transform.position.x, transform.position.y, size.x, size.y);
    rect.x -= (transform.pivot.x * size.x);
    rect.y -= ((1.0f - transform.pivot.y) * size.y);
    return rect;
}

¹And finding that post from a generalized search on "how do I get the screen coordinates of a UI object" was not easy. A bunch of other posts came up and had some code, but none of it did what I wanted (including converting screen space coordinates back into world space coordinates of the UI object which was stupid easy and not reversibe, thanks RectTransformUtility!)