Tango 开发指南 — 参考帧

原文链接：https://developers.google.com/tango/overview/frames-of-reference

当我们描述某样东西（比如 Tango 设备）的位置和方向时，说清楚你使用的参考帧是很重要的。

下面举个例子来帮助理解参考帧：比如“玛丽站在三米远的地方”并不能真正说明她的位置。如果你想确定她的位置，你首先得解决这个问题“离谁三英尺远？”。如果你说“玛丽站在离自由女神像入口前面三英尺的地方”，那就能够确定了，因为你在用自由女神像入作为参考帧，然后给出了相对于参考帧的距离和方向。

但是，玛丽并不是 3D 空间中一个简单的点，即她还有方向，相对于参考帧的方向。换句话，和其他 3D 物体一样，玛丽面朝某个方向。确定玛丽的位置和方向（这两种数据的集合叫 pose）的完整表述是“玛丽站在离自由女神像入口前面三英尺的地方，面朝自由女神像”。现在完整的表述里面有了关于方向的信息，如果玛丽向右转了，你可以说“玛丽现在旋转了90度”。这是描述方向的另一种表述。

现实生活如此，Tango 设备是什么样呢？运动追踪功能运行时，设备会报告它相对于参考帧的 pose（位置和方向）数据，参考帧固定于 3D 空间。例如，设备会说“相对于我开始运动追踪功能的地方，我往前进了三英尺，升高了一英尺，向右 30 度”。这样，设备就告诉你了足够的确定它位置的信息：相对于起始位置，向前三英尺，向上一英尺。除此之外还告诉你了方向信息：相对于起始位置，向右旋转了 30 度。

要使用运动追踪功能，你必须做到如下几点：

• 选择基础帧，这是你测量的参考点。如上所述，这是一个固定于 3D 空间的位置，比如自由女神像。举例：COORDINATE_FRAME_START_OF_SERVICE。
• 选择目标帧，这是你测量的目标点。对于运动追踪，目标值通常是 COORDINATE_FRAME_DEVICE，表示设备在 3D 空间移动时某个时刻的 pose。当设备移动时，目标帧的 pose 跟着改变，最大频率是 100 次/秒，但是基础帧是不变的。pose 数据流就构成了运动追踪的轨迹。

相对于基础帧的目标帧的 pose 这一量化指标回答了这个问题“相对于参考帧，设备的位置和分析怎么描述？”。

下面的章节，我们会讨论在运动追踪中使用的服务开始帧（start-of-service frame），区域描述帧（area description frame）和设备 pose 帧。对于某些应用，可能需要选择合适的帧对，才能做到精确对齐来自设备某部件的数据源。稍后我们会讨论这些帧对的类型。

关于用于参考帧的坐标系的展示，请参考坐标系转换。

坐标帧对

Tango 提供了若干帧对用于运动追踪：

目标帧 基础帧 COORDINATE_FRAME_DEVICE COORDINATE_FRAME_START_OF_SERVICE COORDINATE_FRAME_DEVICE COORDINATE_FRAME_AREA_DESCRIPTION COORDINATE_FRAME_START_OF_SERVICE COORDINATE_FRAME_AREA_DESCRIPTION

我们来看一个常见的应用案例：
问题：你的应用在一个完全虚拟的环境中控制照相机。你需要的是设备相对于服务开启时的原始位置的 pose 数据。
方案：目标帧选 COORDINATE_FRAME_DEVICE，基础帧选 COORDINATE_FRAME_START_OF_SERVICE。

下面是我们在 demo 工程 cpp_hello_motion_tracking_example 例子中使用的帧对：

 TangoCoordinateFramePair pair;   pair.base = TANGO_COORDINATE_FRAME_START_OF_SERVICE;   pair.target = TANGO_COORDINATE_FRAME_DEVICE;   if (TangoService_connectOnPoseAvailable(1, &pair, onPoseAvailable) !=       TANGO_SUCCESS) {     LOGE("TangoHandler::OnResume, connectOnPoseAvailable error.");     std::exit(EXIT_SUCCESS);   }

我们来详细的看下每一个帧对。

目标帧 基础帧 COORDINATE_FRAME_DEVICE COORDINATE_FRAME_START_OF_SERVICE

这个帧对提供的是设备相对于运动追踪服务首次初始化成功时的 pose 数据。这种模式将从服务开启时持续记录设备的运动轨迹。运动追踪服务还能检测自身是否出现故障。在发生故障的时间段，系统会上报 INVALID 类型的 pose 数据。此时如果调用了 TangoService_resetMotionTracking() 方法或 服务配置中开启了 auto-reset 选项，系统将尝试重新初始化该服务。重新初始化成功后，系统将努力恢复最近一次可用的 pose 数据，and pick up where it left off。关于该问题的更多信息，请查看 pose 状态的生命周期。该帧对不包括漂移修正和定位。如果你的应用不使用漂移修正或定位，你可以关闭区域学习模式以及不装载 ADF，以此 lower processing requirements。

---

目标帧 基础帧 COORDINATE_FRAME_DEVICE COORDINATE_FRAME_AREA_DESCRIPTION

该针对提供相对于装载的区域描述的 pose，且包括漂移修正。该模式需要开启区域学习模式或装载预先创建好的 ADF 文件。如果你开启了区域学习模式但是未装载 ADF，区域描述基础帧将会被初始化为服务开启帧。如果你装载了 ADF，不管你有没有开启区域学习模式，区域描述基础帧都将是存在 ADF 中起始地点，而且只有当设备定位（localize）后才会拿到 pose 数据。根据配置的不同，这种模式并不总是可用的。更多信息，请查看使用区域学习模式和装载 ADF。如果你想在 COORDINATE_FRAME_DEVICE & COORDINATE_FRAME_AREA_DESCRIPTION 帧对变成 VALID 状态之前使用，可以临时使用 COORDINATE_FRAME_START_OF_SERVICE 作为基础帧。

---

原文：

When describing the position and orientation of something (for example, your Tango device), it is important to indicate the frame of reference you are using to base your description on.

To help understand frames of reference, consider the following: Saying “Mary is standing three feet away” does not really tell you much. If you want to know Mary’s position, you must also address the question “three feet from what?” If you say “Mary is standing three feet in front of the entrance to the Statue of Liberty,” you can now establish Mary’s position because you are using the Statue of Liberty as your frame of reference and you can measure the distance and directon of Mary relative to the Statue.

But Mary isn’t simply a point with a position in 3D space—she also has an orientation, which is described in terms of some type of rotation relative to the frame of reference. In other words, Mary, like all 3D objects, faces a certain direction. A full description of Mary’s position and orientation (we call this combination a pose) in 3D space would be something like this: “Mary is standing three feet in front of the entrance to the Statue of Liberty, and she is directly facing it.” Now you have provided information about her orientation. If Mary turned to her right, you could say “She is now rotated 90 degrees away from the Statue.” This would be another description of orientation.

So how does all of this relate to a Tango device? In order to perform motion tracking, a device reports its pose (position and orientation) relative to its chosen frame of reference, which is fixed in 3D space. For example, the device might say “from the place that I first started motion tracking, I am now three feet forward and one foot up, and I have rotated 30 degrees to the right.” By doing this, the device has told you its position using meaningful directions: three feet forward and one foot up from its original starting position. It has also told you about a change in its orientation: rotated 30 degrees to the right relative to its starting position.

To set things up for motion tracking, you must do the following:

1. Choose a base frame. This is the thing you will be measuring from. As mentioned above, it is fixed in 3D space, like the Statue of Liberty in our example above. Example: the COORDINATE_FRAME_START_OF_SERVICE frame.
2. Choose a target frame. This is the thing you will be measuring to. For motion tracking this is usually COORDINATE_FRAME_DEVICE and represents your device’s pose at any given instant as it moves through 3D space. The pose of the target frame changes as your device moves, and is measured against the base frame (which never changes), up to 100 times per second. This constant stream of measurements creates your motion track.

The numerical measurements of the pose of the target frame relative to the base frame at any given instant answer the question: “What is the device’s position and orientation relative to its base frame of reference?”

In the next section, we discuss the use of start-of-service frame, area description frame, and device pose frame pairs for motion tracking. For certain applications, you may need to choose a frame pair that will enable you to make precise alignments of data sources from device components. We discuss these types of frame pairs later in this topic.

To learn more about the coordinate systems used for frames of reference, see Coordinate System Conventions.

Coordinate frames for motion tracking

The Tango APIs give you various frame pair options for motion tracking:

Target Frame Base Frame COORDINATE_FRAME_DEVICE COORDINATE_FRAME_START_OF_SERVICE COORDINATE_FRAME_DEVICE COORDINATE_FRAME_AREA_DESCRIPTION COORDINATE_FRAME_START_OF_SERVICE COORDINATE_FRAME_AREA_DESCRIPTION

Let’s consider a common use case:

Goal: Your app controls a camera in a fully virtual environment. You want the device to always calculate its pose relative to where it was when the Tango service started.

Solution: For the target frame, choose COORDINATE_FRAME_DEVICE. For the base frame, choose COORDINATE_FRAME_START_OF_SERVICE.

Here is the frame pair used in our example project titled cpp_hello_motion_tracking_example:

 TangoCoordinateFramePair pair;   pair.base = TANGO_COORDINATE_FRAME_START_OF_SERVICE;   pair.target = TANGO_COORDINATE_FRAME_DEVICE;   if (TangoService_connectOnPoseAvailable(1, &pair, onPoseAvailable) !=       TANGO_SUCCESS) {     LOGE("TangoHandler::OnResume, connectOnPoseAvailable error.");     std::exit(EXIT_SUCCESS);   }

Let’s look at the details of individual frame pairs.

Target Frame Base Frame COORDINATE_FRAME_DEVICE COORDINATE_FRAME_START_OF_SERVICE

This frame pair provides the pose of the device relative to when the Tango service first initialized successfully. This mode accumulates the movement of the device over time since the service started. The service can also detect if there is a motion tracking failure. During this period, the system reports an invalid pose. If TangoService_resetMotionTracking() is called or auto-reset is enabled in the service configuration, the system attempts to re-initialize tracking. After successful re-initialization, it makes a best effort attempt to recover the last known good pose of the device relative to the start of service frame and pick up where it left off. For more information, see Lifecycle of pose status. This frame pair does not include drift correction or localization. If your application does not use drift correction or localization, you can lower processing requirements by disabling area learning mode and not loading an ADF.

Target Frame Base Frame COORDINATE_FRAME_DEVICE COORDINATE_FRAME_AREA_DESCRIPTION

This frame pair provides the pose of the device, including corrections, relative to the loaded area description’s origin. It requires that area learning mode is turned on or a previously created ADF is loaded. If you turn on learning mode without loading an ADF, the origin of the area description base frame is initially the same as start of service. If you load an ADF with or without learning mode, the origin of the area description base frame is the origin stored in the ADF, and you will receive data only after the device has localized. Depending on your configuration settings, this mode is not always available. For more information, see Using Learning Mode and loaded Area Description Files. If you need to use motion tracking before the COORDINATE_FRAME_DEVICE to COORDINATE_FRAME_AREA_DESCRIPTION frame pair becomes valid, you can use the COORDINATE_FRAME_START_OF_SERVICE base frame in the interim.

Note: Drift corrections and localization events cause jumps in the pose. To avoid these jumps, use the COORDINATE_FRAME_START_OF_SERVICE base frame to drive the user-facing elements in your application and incorporate the ADF driven corrections using COORDINATE_FRAME_START_OF_SERVICE to COORDINATE_FRAME_AREA_DESCRIPTION update callbacks.
For pairs using the COORDINATE_FRAME_DEVICE target frame, updates are available at the pose estimation rate supported by the device.

Target Frame Base Frame COORDINATE_FRAME_START_OF_SERVICE COORDINATE_FRAME_AREA_DESCRIPTION

This frame pair provides updates only when a localization event or a drift correction occurs. This requires that area learning mode is turned on or a previously created ADF is loaded. If an ADF is loaded, the origin of the area description base frame is the origin stored in the ADF. This isolates the adjustments to the pose of the device from the incremental frame-to-frame motion, allowing you to decide when and how to incorporate the pose adjustments in your application to minimize disruption to the user experience.

Coordinate frames for component alignment

Target Frame Base Frame
COORDINATE_FRAME_DEVICE COORDINATE_FRAME_IMU
COORDINATE_FRAME_CAMERA_COLOR COORDINATE_FRAME_IMU
COORDINATE_FRAME_CAMERA_DEPTH COORDINATE_FRAME_IMU
COORDINATE_FRAME_CAMERA_FISHEYE COORDINATE_FRAME_IMU
Some applications need to align multiple data sources, such as the data from the color and depth cameras. You can pair the COORDINATE_FRAME_IMU base frame with one of the component target frames for these scenarios:

1. You want to query the relative offsets of the individual components to the IMU frame of reference without knowing the layout of the specific device.
2. You want the virtual image from the rendering camera to align with the center of the display.
   Combined with the motion tracking coordinate frames and timestamps on the data, these offsets give you a more complete understanding of the various sensor inputs in both space and time. This is necessary for aligning and compositing multiple data sources together.

Note: The relative offsets between two components are sometimes referred to as the extrinsic parameters.
Since devices are designed to be mechanically rigid, these offsets are not expected to change and updates will not occur in the API. However, devices vary in how their components are spaced. Updating extrinsic parameters over time is not currently supported by the Tango APIs. These values are generated either from a one-time factory calibration or from the manufacturer’s mechanical design files. Applications that require extremely tight requirements for the extrinsic parameters should consider implementing their own calibration procedure which can be performed by the end user.

The COORDINATE_FRAME_IMU base frame provides a common reference point for all of the internal components in the device. The origin of this base frame does not necessarily correspond to any one particular component and may differ between devices. Like other Android sensors, the axis of the device coordinate frame is aligned with the natural (default) orientation of the device as defined by the manufacturer. The manufacturer-defined natural orientation of the device may not match the desired orientation of your app. For maximum future compatibility, do not assume a Tango-compliant device has a natural orientation that is either landscape or portrait. Instead, use the Android getRotation() method to determine screen rotation, and then use the Android remapCoordinateSystem() method to map sensor coordinates to screen coordinates. For more general information about sensors, see the Android documentation on the sensor coordinate system. For a more detailed discussion of issues surrounding device orientation, see this Android Developers Blog post.

The component offsets are static and should only need to be queried once.

Note: The unit of measurement for coordinate frame pairs is meters.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.