Is there an alternative to ScreenToWorldPoint() specific for RGB camera?

Hi everyone, I’m using the Nreal Light RGB camera to get frames and find objects in them using YOLO. Once I get the 2D points corresponding to the positions of objects in a particular frame, I want to tranform this point back in Unity 3D space. I’m currently doing so with the method ScreenToWorldPoint(), but the x and y coordinates position are not so accurate (the z is estimated by me). Does NRSDK provide an alternative way?
Thank you in advance!

1 Like

Hi, developer. Maybe the attached post can be helpful to you.

Hi, thanks for the response. Unfortunately this doesn’t seem to fix my problem. More precisely, I have 2 variable x and y, both are normalized in the range (0,1). This coordinates are associated to the center position of one object that was recognized in screen space. I want to track it in the 3D world. This is what I tried so far:

transform.position = Camera.main.ViewportToWorldPoint(new Vector3(x, y, z));

I previously used ScreenToWorldPoint() (after getting those coordinates in pixels), but its the same result.

Maybe there is some sort of matrix that I could use, provided by NRSDK as utility. If so could you please explain to me I to use it?

Dont use the Camera.main
For the Camera from the glasses you might want to use CameraCenter

Join us on discord <3

The RGB cam corresponds to the far plane of the camera, so converting your viewport to world point you use that for the z axis, to augment objects on any detected yolo object.