I am developing a MR application for the Nreal. Here is what I want to do:
I am detecting a real object relative to the camera image that I get from the RGB camera. This works great. But now I want to overlay a virtual object at the right position in Unity. Therefore I need the camera transformation matrix. Can anyone help me with this? Do I have to use the Eye pose or the head pose for this? How can I convert this pose to a 4x4 camera matrix? (extrinsic parameters).
Thanks a lot in advance!
I suggest to use the search function but here:
Hi thanks, but I know how to get the intrinsic paramters - I already detected the object relative to the camera right. What I need is to place in in the right position in Unity. Do you know how to do that?
I am trying the same but its not accurate yet. Will let you know if I managed it properly but its quite hard. Would you let me know if you managed it, please?
I am also working with these APIs… I was wondering how you are using the values from GetRGBCameraIntrinsicMatrix to get a projection matrix? The numbers do not look like focal lengths… I ended up calling GetEyeProjectMatrix to get the RGB projection matrix. But everything is a little off.
As for NRFrame.EyePoseFromHead.RGBEyePose - this does seem a little off, it returns:
pos:(0.0, 0.0, 0.0), rot:(0.1, 0.0, 0.0, 1.0) - which looks like a temp value.
There is a function listed in their docs though to convert it to a 4x4 transform:
NRSDK Scripting API — NRSDK 1.6.0 documentation (nrealsdkdoc.readthedocs.io)
Not tried it yet.
Sorry for delayed reply. You could check the “NRSDK Scripting API/Device Parameters” in NRSDK document.
The interface of NRFrame.GetEyeProjectMatrix“ could get leftDisplay、rightDisplay、RGBCamera’s PRojectMatrix.
Thanks @nreal-dev - could you share what the values from the call mean:
// rgb camera intrinsic matrix NativeMat3f rgb_in_mat = NRFrame.GetRGBCameraIntrinsicMatrix();
The returned values are:
1143.524,0,650.8565, 0,1142.766,386.9736, 0,0,1
It looks to me like the 650 and 386 values are cx and cy, however, are the larger numbers fx and fy?
Don’t worry @nreal-dev - I did some reading and found out how to turn these values into a projection matrix:
var intrinsic = NRFrame.GetRGBCameraIntrinsicMatrix(); var fx = intrinsic * SensorWidth / Width; var fy = intrinsic * SensorHeight / Height; var cx = ((Width / 2f) - intrinsic) / Width * 2f; var cy = (intrinsic - (Height / 2f)) / Height * 2f;
I am putting SensorWidth and SensorHeight as 2mm - do you have an actual number for this?
hey, any updates on this? been a year!