XR_ANDROID_face_tracking OpenXR 扩展程序

名称字符串

XR_ANDROID_face_tracking

扩展程序类型

实例扩展程序

注册扩展程序编号

459

修订版本

1

扩展程序和版本依赖项

OpenXR 1.0

最后修改日期

2024-09-06

IP 状态

无已知 IP 声明。

贡献者

Spencer Quin, Google

Jared Finder, Google

Levana Chen, Google

概览

此扩展程序使应用能够获取混合形状的权重,并在 XR 体验中呈现面部表情。

此扩展程序旨在提供创建逼真头像和用户在虚拟空间中富有表现力的表示所需的信息。应用可以在获取混合形状权重之前检查面部校准激活状态。

面部追踪器

面部追踪器是一种通过面向用户的图像流和相机校准来追踪面部表情的传感设备。此扩展程序的主要目的是将用户面部表情映射到他们在虚拟场景中的头像。

面部追踪数据是敏感个人信息,与个人隐私和完整性密切相关。强烈建议存储或传输面部追踪数据的应用始终征得用户的积极且具体的同意才能进行此操作。

  • 在允许应用访问面部追踪器之前,创建活动面部追踪器时,应用将收到XR_ERROR_PERMISSION_INSUFFICIENT
  • 在使用xrGetFaceStateANDROID获取面部状态时,除非允许应用访问,否则XrFaceStateANDROID::isValid不会返回XR_TRUE

创建面部追踪器句柄

XR_DEFINE_HANDLE(XrFaceTrackerANDROID)

XrFaceTrackerANDROID 句柄表示用于面部追踪的面部追踪器。

此句柄可以用于使用此扩展程序中的其他函数访问面部追踪数据。

xrCreateFaceTrackerANDROID 函数定义如下

XrResult xrCreateFaceTrackerANDROID(
    XrSession                                   session,
    const XrFaceTrackerCreateInfoANDROID*       createInfo,
    XrFaceTrackerANDROID*                       faceTracker);

参数说明

应用可以使用 xrCreateFaceTrackerANDROID 函数创建 XrFaceTrackerANDROID 句柄。

如果系统不支持面部追踪,xrCreateFaceTrackerANDROID 将返回 XR_ERROR_FEATURE_UNSUPPORTED

有效用法(隐式)

返回代码

成功

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

失败

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_VALIDATION_FAILURE
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_LIMIT_REACHED

XrFaceTrackerCreateInfoANDROID 结构描述如下

typedef struct XrFaceTrackerCreateInfoANDROID {
    XrStructureType    type;
    void*              next;
} XrFaceTrackerCreateInfoANDROID;

成员说明

  • type 是此结构的 XrStructureType
  • nextNULL 或指向结构链中下一个结构的指针。核心 OpenXR 或此扩展程序中未定义此类结构。

XrFaceTrackerCreateInfoANDROID 结构描述了创建 XrFaceTrackerANDROID 句柄的信息。

有效用法(隐式)

xrDestroyFaceTrackerANDROID 函数定义如下

XrResult xrDestroyFaceTrackerANDROID(
    XrFaceTrackerANDROID                        faceTracker);

参数说明

xrDestroyFaceTrackerANDROID 函数在面部追踪体验结束后释放 faceTracker 和底层资源。

有效用法(隐式)

线程安全

  • faceTracker 和任何子句柄的访问必须进行外部同步

返回代码

成功

  • XR_SUCCESS

失败

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID

检查面部校准

xrGetFaceCalibrationStateANDROID 函数定义如下

XrResult xrGetFaceCalibrationStateANDROID(
    XrFaceTrackerANDROID                        faceTracker,
    XrBool32*                                   faceIsCalibratedOutput);

参数说明

应用可以使用 xrGetFaceCalibrationStateANDROID 函数检查面部校准状态。

如果系统不支持面部校准,xrGetFaceCalibrationStateANDROID 将返回 XR_ERROR_FEATURE_UNSUPPORTED。否则,faceIsCalibratedOutput 可能会设置为 XR_TRUE 以反映面部校准状态。

有效用法(隐式)

返回代码

成功

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

失败

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_VALIDATION_FAILURE
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_LIMIT_REACHED

获取面部表情

xrGetFaceStateANDROID 函数返回给定时间的面部表情混合形状。

XrResult xrGetFaceStateANDROID(
    XrFaceTrackerANDROID                        faceTracker,
    const XrFaceStateGetInfoANDROID*            getInfo,
    XrFaceStateANDROID*                         faceStateOutput);

参数说明

有效用法(隐式)

返回代码

成功

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

失败

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_VALIDATION_FAILURE
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_LIMIT_REACHED
  • XR_ERROR_TIME_INVALID

XrFaceStateGetInfoANDROID 结构描述了获取面部表情的信息。

typedef struct XrFaceStateGetInfoANDROID {
    XrStructureType    type;
    void*              next;
    XrTime             time;
} XrFaceStateGetInfoANDROID;

成员说明

  • type 是此结构的 XrStructureType
  • nextNULL 或指向结构链中下一个结构的指针。核心 OpenXR 或此扩展程序中未定义此类结构。
  • time 是请求面部表情时的 XrTime

应用请求一个等于渲染帧的预测显示时间的时间。

有效用法(隐式)

XrFaceStateANDROID 结构返回面部追踪状态和面部表情。

typedef struct XrFaceStateANDROID {
    XrStructureType               type;
    void*                         next;
    uint32_t                      parametersCapacityInput;
    uint32_t                      parametersCountOutput;
    float*                        parameters;
    XrFaceTrackingStateANDROID    faceTrackingState;
    XrTime                        sampleTime;
    XrBool32                      isValid;
} XrFaceStateANDROID;

成员说明

  • type 是此结构的 XrStructureType
  • nextNULL 或指向结构链中下一个结构的指针。核心 OpenXR 或此扩展程序中未定义此类结构。
  • parametersCapacityInput 是一个 uint32_t 值,描述 parameters 数组的容量,或为 0 以指示请求检索所需容量。
  • parametersCountOutput 是一个 uint32_t 值,描述 parameters 的数量,或者在 parametersCapacityInput 不足时所需的容量。
  • parameters 是指向应用分配的 float 数组的指针,该数组将填充面部表情混合形状的权重。
  • faceTrackingState 是面部追踪有效性状态的 XrFaceTrackingStateANDROID
  • sampleTime 是返回表达式被追踪或推断到的 XrTime 时间。如果当时推断成功,它等于请求表达式权重的时间。
  • isValid 指示数据是否有效,即使它不是来自当前帧。
  • 有关检索所需 parameters 大小的详细说明,请参阅缓冲区大小参数部分。

应用可以parametersCapacityInput 设置为 XR_FACE_PARAMETER_COUNT_ANDROID 以获取由 XrFaceParameterIndicesANDROID 索引的面部表情。

返回的 parameters 表示当前面部表情混合形状的权重。

parameters 数组的更新将按顺序排列,以便应用可以使用相应的面部表情枚举(例如 XrFaceParameterIndicesANDROID)索引元素。

有效用法(隐式)

XrFaceTrackingStateANDROID 枚举标识了面部追踪器的不同状态。

typedef enum XrFaceTrackingStateANDROID {
    XR_FACE_TRACKING_STATE_PAUSED_ANDROID = 0,
    XR_FACE_TRACKING_STATE_STOPPED_ANDROID = 1,
    XR_FACE_TRACKING_STATE_TRACKING_ANDROID = 2
} XrFaceTrackingStateANDROID;

枚举具有以下含义

枚举

说明

XR_FACE_TRACKING_STATE_PAUSED_ANDROID

表示面部追踪已暂停,但将来可能会恢复。

XR_FACE_TRACKING_STATE_STOPPED_ANDROID

追踪已停止,但客户端仍有一个活动的面部追踪器。

XR_FACE_TRACKING_STATE_TRACKING_ANDROID

面部正在被追踪,其姿态是当前的。

混合形状的约定

此扩展程序通过 XR_FACE_PARAMETER_COUNT_ANDROID 定义了 63 个混合形状,用于精简的 G-Nome 格式。此枚举中的每个参数都是混合形状数组的索引,其值为 float 类型,运行时归一化为 1 - 0。

typedef enum XrFaceParameterIndicesANDROID {
    XR_FACE_PARAMETER_INDICES_BROW_LOWERER_L_ANDROID = 0,
    XR_FACE_PARAMETER_INDICES_BROW_LOWERER_R_ANDROID = 1,
    XR_FACE_PARAMETER_INDICES_CHEEK_PUFF_L_ANDROID = 2,
    XR_FACE_PARAMETER_INDICES_CHEEK_PUFF_R_ANDROID = 3,
    XR_FACE_PARAMETER_INDICES_CHEEK_RAISER_L_ANDROID = 4,
    XR_FACE_PARAMETER_INDICES_CHEEK_RAISER_R_ANDROID = 5,
    XR_FACE_PARAMETER_INDICES_CHEEK_SUCK_L_ANDROID = 6,
    XR_FACE_PARAMETER_INDICES_CHEEK_SUCK_R_ANDROID = 7,
    XR_FACE_PARAMETER_INDICES_CHIN_RAISER_B_ANDROID = 8,
    XR_FACE_PARAMETER_INDICES_CHIN_RAISER_T_ANDROID = 9,
    XR_FACE_PARAMETER_INDICES_DIMPLER_L_ANDROID = 10,
    XR_FACE_PARAMETER_INDICES_DIMPLER_R_ANDROID = 11,
    XR_FACE_PARAMETER_INDICES_EYES_CLOSED_L_ANDROID = 12,
    XR_FACE_PARAMETER_INDICES_EYES_CLOSED_R_ANDROID = 13,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_DOWN_L_ANDROID = 14,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_DOWN_R_ANDROID = 15,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_LEFT_L_ANDROID = 16,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_LEFT_R_ANDROID = 17,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_RIGHT_L_ANDROID = 18,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_RIGHT_R_ANDROID = 19,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_UP_L_ANDROID = 20,
    XR_FACE_PARAMETER_INDICES_EYES_LOOK_UP_R_ANDROID = 21,
    XR_FACE_PARAMETER_INDICES_INNER_BROW_RAISER_L_ANDROID = 22,
    XR_FACE_PARAMETER_INDICES_INNER_BROW_RAISER_R_ANDROID = 23,
    XR_FACE_PARAMETER_INDICES_JAW_DROP_ANDROID = 24,
    XR_FACE_PARAMETER_INDICES_JAW_SIDEWAYS_LEFT_ANDROID = 25,
    XR_FACE_PARAMETER_INDICES_JAW_SIDEWAYS_RIGHT_ANDROID = 26,
    XR_FACE_PARAMETER_INDICES_JAW_THRUST_ANDROID = 27,
    XR_FACE_PARAMETER_INDICES_LID_TIGHTENER_L_ANDROID = 28,
    XR_FACE_PARAMETER_INDICES_LID_TIGHTENER_R_ANDROID = 29,
    XR_FACE_PARAMETER_INDICES_LIP_CORNER_DEPRESSOR_L_ANDROID = 30,
    XR_FACE_PARAMETER_INDICES_LIP_CORNER_DEPRESSOR_R_ANDROID = 31,
    XR_FACE_PARAMETER_INDICES_LIP_CORNER_PULLER_L_ANDROID = 32,
    XR_FACE_PARAMETER_INDICES_LIP_CORNER_PULLER_R_ANDROID = 33,
    XR_FACE_PARAMETER_INDICES_LIP_FUNNELER_LB_ANDROID = 34,
    XR_FACE_PARAMETER_INDICES_LIP_FUNNELER_LT_ANDROID = 35,
    XR_FACE_PARAMETER_INDICES_LIP_FUNNELER_RB_ANDROID = 36,
    XR_FACE_PARAMETER_INDICES_LIP_FUNNELER_RT_ANDROID = 37,
    XR_FACE_PARAMETER_INDICES_LIP_PRESSOR_L_ANDROID = 38,
    XR_FACE_PARAMETER_INDICES_LIP_PRESSOR_R_ANDROID = 39,
    XR_FACE_PARAMETER_INDICES_LIP_PUCKER_L_ANDROID = 40,
    XR_FACE_PARAMETER_INDICES_LIP_PUCKER_R_ANDROID = 41,
    XR_FACE_PARAMETER_INDICES_LIP_STRETCHER_L_ANDROID = 42,
    XR_FACE_PARAMETER_INDICES_LIP_STRETCHER_R_ANDROID = 43,
    XR_FACE_PARAMETER_INDICES_LIP_SUCK_LB_ANDROID = 44,
    XR_FACE_PARAMETER_INDICES_LIP_SUCK_LT_ANDROID = 45,
    XR_FACE_PARAMETER_INDICES_LIP_SUCK_RB_ANDROID = 46,
    XR_FACE_PARAMETER_INDICES_LIP_SUCK_RT_ANDROID = 47,
    XR_FACE_PARAMETER_INDICES_LIP_TIGHTENER_L_ANDROID = 48,
    XR_FACE_PARAMETER_INDICES_LIP_TIGHTENER_R_ANDROID = 49,
    XR_FACE_PARAMETER_INDICES_LIPS_TOWARD_ANDROID = 50,
    XR_FACE_PARAMETER_INDICES_LOWER_LIP_DEPRESSOR_L_ANDROID = 51,
    XR_FACE_PARAMETER_INDICES_LOWER_LIP_DEPRESSOR_R_ANDROID = 52,
    XR_FACE_PARAMETER_INDICES_MOUTH_LEFT_ANDROID = 53,
    XR_FACE_PARAMETER_INDICES_MOUTH_RIGHT_ANDROID = 54,
    XR_FACE_PARAMETER_INDICES_NOSE_WRINKLER_L_ANDROID = 55,
    XR_FACE_PARAMETER_INDICES_NOSE_WRINKLER_R_ANDROID = 56,
    XR_FACE_PARAMETER_INDICES_OUTER_BROW_RAISER_L_ANDROID = 57,
    XR_FACE_PARAMETER_INDICES_OUTER_BROW_RAISER_R_ANDROID = 58,
    XR_FACE_PARAMETER_INDICES_UPPER_LID_RAISER_L_ANDROID = 59,
    XR_FACE_PARAMETER_INDICES_UPPER_LID_RAISER_R_ANDROID = 60,
    XR_FACE_PARAMETER_INDICES_UPPER_LIP_RAISER_L_ANDROID = 61,
    XR_FACE_PARAMETER_INDICES_UPPER_LIP_RAISER_R_ANDROID = 62
} XrFaceParameterIndicesANDROID;
索引 名称 参考图片
0 BROW_LOWERER_L
Neutral face reference image
BROW_LOWERER_L = 0.0
face tracking reference image
BROW_LOWERER_L = 1.0
1 BROW_LOWERER_R
Neutral face reference image
BROW_LOWERER_R = 0.0
face tracking reference image
BROW_LOWERER_R = 1.0
2 CHEEK_PUFF_L
Neutral face reference image
CHEEK_PUFF_L = 0.0
face tracking reference image
CHEEK_PUFF_L = 1.0
3 CHEEK_PUFF_R
Neutral face reference image
CHEEK_PUFF_R = 0.0
face tracking reference image
CHEEK_PUFF_R = 1.0
4 CHEEK_RAISER_L
Neutral face reference image
CHEEK_RAISER_L = 0.0
face tracking reference image
CHEEK_RAISER_L = 1.0
5 CHEEK_RAISER_R
Neutral face reference image
CHEEK_RAISER_R = 0.0
face tracking reference image
CHEEK_RAISER_R = 1.0
6 CHEEK_SUCK_L
Neutral face reference image
CHEEK_SUCK_L = 0.0
face tracking reference image
CHEEK_SUCK_L = 1.0
7 CHEEK_SUCK_R
Neutral face reference image
CHEEK_SUCK_R = 0.0
face tracking reference image
CHEEK_SUCK_R = 1.0
8 CHIN_RAISER_B
Neutral face reference image
CHIN_RAISER_B = 0.0
face tracking reference image
CHIN_RAISER_B = 1.0
9 CHIN_RAISER_T
Neutral face reference image
CHIN_RAISER_T = 0.0
face tracking reference image
CHIN_RAISER_T = 1.0
10 DIMPLER_L
Neutral face reference image
DIMPLER_L = 0.0
face tracking reference image
DIMPLER_L = 1.0
11 DIMPLER_R
Neutral face reference image
DIMPLER_R = 0.0
face tracking reference image
DIMPLER_R = 1.0
12 EYES_CLOSED_L
Neutral face reference image
EYES_CLOSED_L = 0.0
face tracking reference image
EYES_CLOSED_L = 1.0
13 EYES_CLOSED_R
Neutral face reference image
EYES_CLOSED_R = 0.0
face tracking reference image
EYES_CLOSED_R = 1.0
14 EYES_LOOK_DOWN_L
Neutral face reference image
EYES_LOOK_DOWN_L = 0.0
face tracking reference image
EYES_LOOK_DOWN_L = 1.0
15 EYES_LOOK_DOWN_R
Neutral face reference image
EYES_LOOK_DOWN_R = 0.0
face tracking reference image
EYES_LOOK_DOWN_R = 1.0
16 EYES_LOOK_LEFT_L
Neutral face reference image
EYES_LOOK_LEFT_L = 0.0
face tracking reference image
EYES_LOOK_LEFT_L = 1.0
17 EYES_LOOK_LEFT_R
Neutral face reference image
EYES_LOOK_LEFT_R = 0.0
face tracking reference image
EYES_LOOK_LEFT_R = 1.0
18 EYES_LOOK_RIGHT_L
Neutral face reference image
EYES_LOOK_RIGHT_L = 0.0
face tracking reference image
EYES_LOOK_RIGHT_L = 1.0
19 EYES_LOOK_RIGHT_R
Neutral face reference image
EYES_LOOK_RIGHT_R = 0.0
face tracking reference image
EYES_LOOK_RIGHT_R = 1.0
20 EYES_LOOK_UP_L
Neutral face reference image
EYES_LOOK_UP_L = 0.0
face tracking reference image
EYES_LOOK_UP_L = 1.0
21 EYES_LOOK_UP_R
Neutral face reference image
EYES_LOOK_UP_R = 0.0
face tracking reference image
EYES_LOOK_UP_R = 1.0
22 INNER_BROW_RAISER_L
Neutral face reference image
INNER_BROW_RAISER_L = 0.0
face tracking reference image
INNER_BROW_RAISER_L = 1.0
23 INNER_BROW_RAISER_R
Neutral face reference image
INNER_BROW_RAISER_R = 0.0
face tracking reference image
INNER_BROW_RAISER_R = 1.0
24 JAW_DROP
Neutral face reference image
JAW_DROP = 0.0
face tracking reference image
JAW_DROP = 1.0
25 JAW_SIDEWAYS_LEFT
Neutral face reference image
JAW_SIDEWAYS_LEFT = 0.0
face tracking reference image
JAW_SIDEWAYS_LEFT = 1.0
26 JAW_SIDEWAYS_RIGHT
Neutral face reference image
JAW_SIDEWAYS_RIGHT = 0.0
face tracking reference image
JAW_SIDEWAYS_RIGHT = 1.0
27 JAW_THRUST
Neutral face reference image
JAW_THRUST = 0.0
face tracking reference image
JAW_THRUST = 1.0
28 LID_TIGHTENER_L
Neutral face reference image
LID_TIGHTENER_L = 0.0
face tracking reference image
LID_TIGHTENER_L = 1.0
29 LID_TIGHTENER_R
Neutral face reference image
LID_TIGHTENER_R = 0.0
face tracking reference image
LID_TIGHTENER_R = 1.0
30 LIP_CORNER_DEPRESSOR_L
Neutral face reference image
LIP_CORNER_DEPRESSOR_L = 0.0
face tracking reference image
LIP_CORNER_DEPRESSOR_L = 1.0
31 LIP_CORNER_DEPRESSOR_R
Neutral face reference image
LIP_CORNER_DEPRESSOR_R = 0.0
face tracking reference image
LIP_CORNER_DEPRESSOR_R = 1.0
32 LIP_CORNER_PULLER_L
Neutral face reference image
LIP_CORNER_PULLER_L = 0.0
face tracking reference image
LIP_CORNER_PULLER_L = 1.0
33 LIP_CORNER_PULLER_R
Neutral face reference image
LIP_CORNER_PULLER_R = 0.0
face tracking reference image
LIP_CORNER_PULLER_R = 1.0
34 LIP_FUNNELER_LB
Neutral face reference image
LIP_FUNNELER_LB = 0.0
face tracking reference image
LIP_FUNNELER_LB = 1.0
35 LIP_FUNNELER_LT
Neutral face reference image
LIP_FUNNELER_LT = 0.0
face tracking reference image
LIP_FUNNELER_LT = 1.0
36 LIP_FUNNELER_RB
Neutral face reference image
LIP_FUNNELER_RB = 0.0
face tracking reference image
LIP_FUNNELER_RB = 1.0
37 LIP_FUNNELER_RT
Neutral face reference image
LIP_FUNNELER_RT = 0.0
face tracking reference image
LIP_FUNNELER_RT = 1.0
38 LIP_PRESSOR_L
Neutral face reference image
LIP_PRESSOR_L = 0.0
face tracking reference image
LIP_PRESSOR_L = 1.0
39 LIP_PRESSOR_R
Neutral face reference image
LIP_PRESSOR_R = 0.0
face tracking reference image
LIP_PRESSOR_R = 1.0
40 LIP_PUCKER_L
Neutral face reference image
LIP_PUCKER_L = 0.0
face tracking reference image
LIP_PUCKER_L = 1.0
41 LIP_PUCKER_R
Neutral face reference image
LIP_PUCKER_R = 0.0
face tracking reference image
LIP_PUCKER_R = 1.0
42 LIP_STRETCHER_L
Neutral face reference image
LIP_STRETCHER_L = 0.0
face tracking reference image
LIP_STRETCHER_L = 1.0
43 LIP_STRETCHER_R
Neutral face reference image
LIP_STRETCHER_R = 0.0
face tracking reference image
LIP_STRETCHER_R = 1.0
44 LIP_SUCK_LB
Neutral face reference image
LIP_SUCK_LB = 0.0
face tracking reference image
LIP_SUCK_LB = 1.0
45 LIP_SUCK_LT
Neutral face reference image
LIP_SUCK_LT = 0.0
face tracking reference image
LIP_SUCK_LT = 1.0
46 LIP_SUCK_RB
Neutral face reference image
LIP_SUCK_RB = 0.0
face tracking reference image
LIP_SUCK_RB = 1.0
47 LIP_SUCK_RT
Neutral face reference image
LIP_SUCK_RT = 0.0
face tracking reference image
LIP_SUCK_RT = 1.0
48 LIP_TIGHTENER_L
Neutral face reference image
LIP_TIGHTENER_L = 0.0
face tracking reference image
LIP_TIGHTENER_L = 1.0
49 LIP_TIGHTENER_R
Neutral face reference image
LIP_TIGHTENER_R = 0.0
face tracking reference image
LIP_TIGHTENER_R = 1.0
50 LIPS_TOWARD
Neutral face reference image
LIPS_TOWARD = 0.0
face tracking reference image
JAW_DROP = 1.0 和 LIPS_TOWARD = 1.0
51 LOWER_LIP_DEPRESSOR_L
Neutral face reference image
LOWER_LIP_DEPRESSOR_L = 0.0
face tracking reference image
LOWER_LIP_DEPRESSOR_L = 1.0
52 LOWER_LIP_DEPRESSOR_R
Neutral face reference image
LOWER_LIP_DEPRESSOR_R = 0.0
face tracking reference image
LOWER_LIP_DEPRESSOR_R = 1.0
53 MOUTH_LEFT
Neutral face reference image
MOUTH_LEFT = 0.0
face tracking reference image
MOUTH_LEFT = 1.0
54 MOUTH_RIGHT
Neutral face reference image
MOUTH_RIGHT = 0.0
face tracking reference image
MOUTH_RIGHT = 1.0
55 NOSE_WRINKLER_L
Neutral face reference image
NOSE_WRINKLER_L = 0.0
face tracking reference image
NOSE_WRINKLER_L = 1.0
56 NOSE_WRINKLER_R
Neutral face reference image
NOSE_WRINKLER_R = 0.0
face tracking reference image
NOSE_WRINKLER_R = 1.0
57 OUTER_BROW_RAISER_L
Neutral face reference image
OUTER_BROW_RAISER_L = 0.0
face tracking reference image
OUTER_BROW_RAISER_L = 1.0
58 OUTER_BROW_RAISER_R
Neutral face reference image
OUTER_BROW_RAISER_R = 0.0
face tracking reference image
OUTER_BROW_RAISER_R = 1.0
59 UPPER_LID_RAISER_L
Neutral face reference image
UPPER_LID_RAISER_L = 0.0
face tracking reference image
UPPER_LID_RAISER_L = 1.0
60 UPPER_LID_RAISER_R
Neutral face reference image
UPPER_LID_RAISER_R = 0.0
face tracking reference image
UPPER_LID_RAISER_R = 1.0
61 UPPER_LIP_RAISER_L
Neutral face reference image
UPPER_LIP_RAISER_L = 0.0
face tracking reference image
UPPER_LIP_RAISER_L = 1.0
62 UPPER_LIP_RAISER_R
Neutral face reference image
UPPER_LIP_RAISER_R = 0.0
face tracking reference image
UPPER_LIP_RAISER_R = 1.0

面部追踪示例代码

以下示例代码演示了如何获取面部表情混合形状的所有权重。

XrSession session; // previously initialized, for example created at app startup.

// The function pointers are previously initialized using xrGetInstanceProcAddr.
PFN_xrCreateFaceTrackerANDROID xrCreateFaceTrackerANDROID; // previously initialized
PFN_xrDestroyFaceTrackerANDROID xrDestroyFaceTrackerANDROID; // previously initialized
PFN_xrGetFaceStateANDROID xrGetFaceStateANDROID; // previously initialized
PFN_xrGetFaceCalibrationStateANDROID xrGetFaceCalibrationStateANDROID; // previously initialized

XrFaceTrackerANDROID faceTracker;
XrFaceTrackerCreateInfoANDROID
        createInfo{.type = XR_TYPE_FACE_TRACKER_CREATE_INFO_ANDROID,
                    .next = nullptr};
CHK_XR(xrCreateFaceTrackerANDROID(session, &createInfo, &faceTracker));

// If the system supports face calibration:
bool isCalibrated;
CHK_XR(xrGetFaceCalibrationStateANDROID(faceTracker, &isCalibrated));
if (!isCalibrated) {
    // Redirect the user to system calibration setting.
}

XrFaceStateANDROID faceState;
float faceExpressionParameters[XR_FACE_PARAMETER_COUNT_ANDROID];
faceState.type = XR_TYPE_FACE_STATE_ANDROID;
faceState.next = nullptr;
faceState.parametersCapacityInput = XR_FACE_PARAMETER_COUNT_ANDROID;
faceState.parameters = faceExpressionParameters;

while (1) {
    // ...
    // For every frame in the frame loop
    // ...
    XrFrameState frameState; // previously returned from xrWaitFrame

    XrFaceStateGetInfoANDROID faceGetInfo{
            .type = XR_TYPE_FACE_STATE_GET_INFO_ANDROID,
            .next = nullptr,
            .time = frameState.predictedDisplayTime,
    };

    CHECK_XR(xrGetFaceStateANDROID(faceTracker, &faceGetInfo, &faceState));
    if (faceState.isValid) {
        for (uint32_t i = 0; i < XR_FACE_PARAMETER_COUNT_ANDROID; ++i) {
            // parameters[i] contains a weight of specific blend shape
        }
    }
}

// after usage
CHK_XR(xrDestroyFaceTrackerANDROID(faceTracker));

新对象类型

新枚举常量

  • XR_FACE_PARAMETER_COUNT_ANDROID

XrObjectType 枚举已扩展为

  • XR_OBJECT_TYPE_FACE_TRACKER_ANDROID

XrStructureType 枚举已扩展为

  • XR_TYPE_FACE_TRACKER_CREATE_INFO_ANDROID
  • XR_TYPE_FACE_STATE_GET_INFO_ANDROID
  • XR_TYPE_FACE_STATE_ANDROID

新枚举

新结构

新函数

问题

版本历史

  • 修订版 1,2024-09-05 (Levana Chen)
    • 初始扩展程序描述

OpenXR™ 和 OpenXR 徽标是 The Khronos Group Inc. 拥有的商标,并在中国、欧盟、日本和英国注册为商标。