[洪流学堂]Hololens开发高级篇2：手势（Gesture）

本教程基于Unity2017.2及Visual Studio 2017
本教程编写时间：2017年12月7日

本文内容提要

• 当跟踪到用户的手时提供反馈
• 使用导航手势旋转hologram
• 当用户的手要离开视线时提供反馈
• 使用操作事件（manipulation events）让用户使用手势移动hologram

预备知识

• 开发环境
• 已完成入门篇的学习
• 已完成高级篇1凝视的学习

资源下载

本文使用了官方教程的资源
原地址
如果下载有困难，百度云地址

0 创建工程

1. 将下载资源的解压出来
2. 打开Unitiy，选择Open，选择HolographicAcademy-Holograms-211-Gesture\Starting\ModelExplorer目录并打开
3. 在Project面板中Scenes目录下，双击打开ModelExplorer场景
4. 打开菜单File > Build Settings，选择Universal Windows Platform，点击Switch Platform按钮
5. Target device设置为Hololens，选中Unity C# Projects
   
6. 在Build Settings面板，点击Build，新建App文件夹并选择该文件夹
7. Build完成后，打开App文件夹下的ModelExplorer.sln，将Debug改为Release，ARM改为x86，并选中Hololens Emulator
8. 点击调试 > 开始执行（不调试）或者Ctrl+F5（注意：模拟器启动慢可能会引起部署超时，这时候不要关闭模拟器，直接再次Ctrl+F5即可）

1 手部检测反馈

1. 在Hierarchy面板中选择Managers物体，在Inspector面板中点击Add Component按钮，搜索Hands Manager并添加
2. 在Hierarchy面板中选择Cursor物体，在Inspector面板中点击Add Component按钮，搜索Cursor Feedback并添加
3. 将Project面板Assets\HoloToolkit-Gesture-211\Input\Prefabs目录下的HandDetectedFeedback资源拖到Inspector面板的Hand Detected Asset属性上
4. 在Hierarchy面板中，展开Cursor物体，将CursorBillboard物体拖到Cursor Feedback (脚本)的Feedback Parent属性上
5. build看一下效果！
   在模拟器中，右键/空格/回车/alt按下并保持（不抬起），模拟Ready手势
   

2 导航

1. 打开HandsManager.cs脚本，按照2.a注释提示补全代码，最终代码参考如下：

using UnityEngine;using UnityEngine.XR.WSA.Input;namespace Academy.HoloToolkit.Unity{    /// <summary>    /// HandsManager keeps track of when a hand is detected.    /// </summary>    public class HandsManager : Singleton<HandsManager>    {        [Tooltip("Audio clip to play when Finger Pressed.")]        public AudioClip FingerPressedSound;        private AudioSource audioSource;        /// <summary>        /// Tracks the hand detected state.        /// </summary>        public bool HandDetected        {            get;            private set;        }        // Keeps track of the GameObject that the hand is interacting with.        public GameObject FocusedGameObject { get; private set; }        void Awake()        {            EnableAudioHapticFeedback();            InteractionManager.InteractionSourceDetected += InteractionManager_InteractionSourceDetected;            InteractionManager.InteractionSourceLost += InteractionManager_InteractionSourceLost;            /* TODO: DEVELOPER CODE ALONG 2.a */            // 2.a: Register for SourceManager.SourcePressed event.            InteractionManager.InteractionSourcePressed += InteractionManager_InteractionSourcePressed;            // 2.a: Register for SourceManager.SourceReleased event.            InteractionManager.InteractionSourceReleased += InteractionManager_InteractionSourceReleased;            // 2.a: Initialize FocusedGameObject as null.            FocusedGameObject = null;        }        private void EnableAudioHapticFeedback()        {            // If this hologram has an audio clip, add an AudioSource with this clip.            if (FingerPressedSound != null)            {                audioSource = GetComponent<AudioSource>();                if (audioSource == null)                {                    audioSource = gameObject.AddComponent<AudioSource>();                }                audioSource.clip = FingerPressedSound;                audioSource.playOnAwake = false;                audioSource.spatialBlend = 1;                audioSource.dopplerLevel = 0;            }        }        private void InteractionManager_InteractionSourceDetected(InteractionSourceDetectedEventArgs obj)        {            HandDetected = true;        }        private void InteractionManager_InteractionSourceLost(InteractionSourceLostEventArgs obj)        {            HandDetected = false;            // 2.a: Reset FocusedGameObject.            ResetFocusedGameObject();        }        private void InteractionManager_InteractionSourcePressed(InteractionSourcePressedEventArgs hand)        {            if (InteractibleManager.Instance.FocusedGameObject != null)            {                // Play a select sound if we have an audio source and are not targeting an asset with a select sound.                if (audioSource != null && !audioSource.isPlaying &&                    (InteractibleManager.Instance.FocusedGameObject.GetComponent<Interactible>() != null &&                    InteractibleManager.Instance.FocusedGameObject.GetComponent<Interactible>().TargetFeedbackSound == null))                {                    audioSource.Play();                }                // 2.a: Cache InteractibleManager's FocusedGameObject in FocusedGameObject.                FocusedGameObject = InteractibleManager.Instance.FocusedGameObject;            }        }        private void InteractionManager_InteractionSourceReleased(InteractionSourceReleasedEventArgs hand)        {            // 2.a: Reset FocusedGameObject.            ResetFocusedGameObject();        }        private void ResetFocusedGameObject()        {            // 2.a: Set FocusedGameObject to be null.            FocusedGameObject = null;            // 2.a: On GestureManager call ResetGestureRecognizers            // to complete any currently active gestures.            GestureManager.Instance.ResetGestureRecognizers();        }        void OnDestroy()        {            InteractionManager.InteractionSourceDetected -= InteractionManager_InteractionSourceDetected;            InteractionManager.InteractionSourceLost -= InteractionManager_InteractionSourceLost;            // 2.a: Unregister the SourceManager.SourceReleased event.            InteractionManager.InteractionSourceReleased -= InteractionManager_InteractionSourceReleased;            // 2.a: Unregister for SourceManager.SourcePressed event.            InteractionManager.InteractionSourcePressed -= InteractionManager_InteractionSourcePressed;        }    }}

1. 在Hierarchy面板中点击Cursor物体
2. 将HoloToolkit\Input\Prefabs目录下的ScrollFeedback资源拖到Cursor Feedback (Script) 组件的Scroll Detected Asset属性上
3. 在Hierarchy面板中，点击AstroMan物体，在Inspector面板中点击Add Component按钮，搜索Gesture Action并添加
4. 打开GestureManager.cs脚本，按照代码中注释2.b的提示补全代码，最终代码参考如下：

using UnityEngine;using UnityEngine.XR.WSA.Input;namespace Academy.HoloToolkit.Unity{    public class GestureManager : Singleton<GestureManager>    {        // Tap and Navigation gesture recognizer.        public GestureRecognizer NavigationRecognizer { get; private set; }        // Manipulation gesture recognizer.        public GestureRecognizer ManipulationRecognizer { get; private set; }        // Currently active gesture recognizer.        public GestureRecognizer ActiveRecognizer { get; private set; }        public bool IsNavigating { get; private set; }        public Vector3 NavigationPosition { get; private set; }        public bool IsManipulating { get; private set; }        public Vector3 ManipulationPosition { get; private set; }        void Awake()        {            /* TODO: DEVELOPER CODING EXERCISE 2.b */            // 2.b: Instantiate the NavigationRecognizer.            NavigationRecognizer = new GestureRecognizer();            // 2.b: Add Tap and NavigationX GestureSettings to the NavigationRecognizer's RecognizableGestures.            NavigationRecognizer.SetRecognizableGestures(                GestureSettings.Tap |                GestureSettings.NavigationX);            // 2.b: Register for the Tapped with the NavigationRecognizer_Tapped function.            NavigationRecognizer.Tapped += NavigationRecognizer_Tapped;            // 2.b: Register for the NavigationStarted with the NavigationRecognizer_NavigationStarted function.            NavigationRecognizer.NavigationStarted += NavigationRecognizer_NavigationStarted;            // 2.b: Register for the NavigationUpdated with the NavigationRecognizer_NavigationUpdated function.            NavigationRecognizer.NavigationUpdated += NavigationRecognizer_NavigationUpdated;            // 2.b: Register for the NavigationCompleted with the NavigationRecognizer_NavigationCompleted function.             NavigationRecognizer.NavigationCompleted += NavigationRecognizer_NavigationCompleted;            // 2.b: Register for the NavigationCanceled with the NavigationRecognizer_NavigationCanceled function.             NavigationRecognizer.NavigationCanceled += NavigationRecognizer_NavigationCanceled;            // Instantiate the ManipulationRecognizer.            ManipulationRecognizer = new GestureRecognizer();            // Add the ManipulationTranslate GestureSetting to the ManipulationRecognizer's RecognizableGestures.            ManipulationRecognizer.SetRecognizableGestures(                GestureSettings.ManipulationTranslate);            // Register for the Manipulation events on the ManipulationRecognizer.            ManipulationRecognizer.ManipulationStarted += ManipulationRecognizer_ManipulationStarted;            ManipulationRecognizer.ManipulationUpdated += ManipulationRecognizer_ManipulationUpdated;            ManipulationRecognizer.ManipulationCompleted += ManipulationRecognizer_ManipulationCompleted;            ManipulationRecognizer.ManipulationCanceled += ManipulationRecognizer_ManipulationCanceled;            ResetGestureRecognizers();        }        void OnDestroy()        {            // 2.b: Unregister the Tapped and Navigation events on the NavigationRecognizer.            NavigationRecognizer.Tapped -= NavigationRecognizer_Tapped;            NavigationRecognizer.NavigationStarted -= NavigationRecognizer_NavigationStarted;            NavigationRecognizer.NavigationUpdated -= NavigationRecognizer_NavigationUpdated;            NavigationRecognizer.NavigationCompleted -= NavigationRecognizer_NavigationCompleted;            NavigationRecognizer.NavigationCanceled -= NavigationRecognizer_NavigationCanceled;            // Unregister the Manipulation events on the ManipulationRecognizer.            ManipulationRecognizer.ManipulationStarted -= ManipulationRecognizer_ManipulationStarted;            ManipulationRecognizer.ManipulationUpdated -= ManipulationRecognizer_ManipulationUpdated;            ManipulationRecognizer.ManipulationCompleted -= ManipulationRecognizer_ManipulationCompleted;            ManipulationRecognizer.ManipulationCanceled -= ManipulationRecognizer_ManipulationCanceled;        }        /// <summary>        /// Revert back to the default GestureRecognizer.        /// </summary>        public void ResetGestureRecognizers()        {            // Default to the navigation gestures.            Transition(NavigationRecognizer);        }        /// <summary>        /// Transition to a new GestureRecognizer.        /// </summary>        /// <param name="newRecognizer">The GestureRecognizer to transition to.</param>        public void Transition(GestureRecognizer newRecognizer)        {            if (newRecognizer == null)            {                return;            }            if (ActiveRecognizer != null)            {                if (ActiveRecognizer == newRecognizer)                {                    return;                }                ActiveRecognizer.CancelGestures();                ActiveRecognizer.StopCapturingGestures();            }            newRecognizer.StartCapturingGestures();            ActiveRecognizer = newRecognizer;        }        private void NavigationRecognizer_NavigationStarted(NavigationStartedEventArgs obj)        {            // 2.b: Set IsNavigating to be true.            IsNavigating = true;            // 2.b: Set NavigationPosition to be Vector3.zero.            NavigationPosition = Vector3.zero;        }        private void NavigationRecognizer_NavigationUpdated(NavigationUpdatedEventArgs obj)        {            // 2.b: Set IsNavigating to be true.            IsNavigating = true;            // 2.b: Set NavigationPosition to be obj.normalizedOffset.            NavigationPosition = obj.normalizedOffset;        }        private void NavigationRecognizer_NavigationCompleted(NavigationCompletedEventArgs obj)        {            // 2.b: Set IsNavigating to be false.            IsNavigating = false;        }        private void NavigationRecognizer_NavigationCanceled(NavigationCanceledEventArgs obj)        {            // 2.b: Set IsNavigating to be false.            IsNavigating = false;        }        private void ManipulationRecognizer_ManipulationStarted(ManipulationStartedEventArgs obj)        {            if (HandsManager.Instance.FocusedGameObject != null)            {                IsManipulating = true;                ManipulationPosition = Vector3.zero;                HandsManager.Instance.FocusedGameObject.SendMessageUpwards("PerformManipulationStart", ManipulationPosition);            }        }        private void ManipulationRecognizer_ManipulationUpdated(ManipulationUpdatedEventArgs obj)        {            if (HandsManager.Instance.FocusedGameObject != null)            {                IsManipulating = true;                ManipulationPosition = obj.cumulativeDelta;                HandsManager.Instance.FocusedGameObject.SendMessageUpwards("PerformManipulationUpdate", ManipulationPosition);            }        }        private void ManipulationRecognizer_ManipulationCompleted(ManipulationCompletedEventArgs obj)        {            IsManipulating = false;        }        private void ManipulationRecognizer_ManipulationCanceled(ManipulationCanceledEventArgs obj)        {            IsManipulating = false;        }        private void NavigationRecognizer_Tapped(TappedEventArgs obj)        {            GameObject focusedObject = InteractibleManager.Instance.FocusedGameObject;            if (focusedObject != null)            {                focusedObject.SendMessageUpwards("OnSelect");            }        }    }}

1. 打开 GestureAction.cs脚本，按照代码中注释2.c的提示补全代码，最终代码参考如下：

using Academy.HoloToolkit.Unity;using UnityEngine;/// <summary>/// GestureAction performs custom actions based on /// which gesture is being performed./// </summary>public class GestureAction : MonoBehaviour{    [Tooltip("Rotation max speed controls amount of rotation.")]    public float RotationSensitivity = 10.0f;    private Vector3 manipulationPreviousPosition;    private float rotationFactor;    void Update()    {        PerformRotation();    }    private void PerformRotation()    {        if (GestureManager.Instance.IsNavigating &&            (!ExpandModel.Instance.IsModelExpanded ||            (ExpandModel.Instance.IsModelExpanded && HandsManager.Instance.FocusedGameObject == gameObject)))        {            /* TODO: DEVELOPER CODING EXERCISE 2.c */            // 2.c: Calculate rotationFactor based on GestureManager's NavigationPosition.X and multiply by RotationSensitivity.            // This will help control the amount of rotation.            rotationFactor = GestureManager.Instance.NavigationPosition.x * RotationSensitivity;            // 2.c: transform.Rotate along the Y axis using rotationFactor.            transform.Rotate(new Vector3(0, -1 * rotationFactor, 0));        }    }    void PerformManipulationStart(Vector3 position)    {        manipulationPreviousPosition = position;    }    void PerformManipulationUpdate(Vector3 position)    {        if (GestureManager.Instance.IsManipulating)        {            /* TODO: DEVELOPER CODING EXERCISE 4.a */            Vector3 moveVector = Vector3.zero;            // 4.a: Calculate the moveVector as position - manipulationPreviousPosition.            // 4.a: Update the manipulationPreviousPosition with the current position.            // 4.a: Increment this transform's position by the moveVector.        }    }}

1. build测试一下吧！
   按下alt键，同时鼠标右键按下并移动，可以模拟此操作
   

3 手部引导

1. 在Hierarchy面板中选中Managers，在Inspector面板中点击Add Component按钮，搜索Hand Guidance并添加
2. 将Project面板下 HoloToolkit-Gesture-211\Input\Prefabs 文件夹下的HandGuidanceFeedback 资源拖到Inspector面板的Hand Guidance Indicator属性上
3. 在Hierarchy面板中展开Cursor物体
4. 在Hierarchy面板中选中Managers物体，将Cursor的子物体CursorBillboard 拖到Inspector面板中的Indicator Parent 属性上
5. build测试一下吧！
   按住alt键，再按下鼠标右键并移动可以模拟此操作，但是模拟器中并不能模拟此功能

4 移动

使用使用手势来移动宇航员
移动手势可以使用时光标提供反馈
1. 在Hierarchy面板中选中Managers，在Inspector面板中点击Add Component按钮，搜索Astronaut Manager并添加
2. 在Hierarchy面板中选中 Cursor
3. 将Project面板中的 HoloToolkit-Gesture-211\Input\Prefabs文件夹下的PathingFeedback 资源拖到Inspector面板的Pathing Detected Asset属性上
4. 编辑 GestureAction.cs 脚本，按照代码中注释4.a的提示补全代码，最终代码参考如下：

using Academy.HoloToolkit.Unity;using UnityEngine;/// <summary>/// GestureAction performs custom actions based on /// which gesture is being performed./// </summary>public class GestureAction : MonoBehaviour{    [Tooltip("Rotation max speed controls amount of rotation.")]    public float RotationSensitivity = 10.0f;    private Vector3 manipulationPreviousPosition;    private float rotationFactor;    void Update()    {        PerformRotation();    }    private void PerformRotation()    {        if (GestureManager.Instance.IsNavigating &&            (!ExpandModel.Instance.IsModelExpanded ||            (ExpandModel.Instance.IsModelExpanded && HandsManager.Instance.FocusedGameObject == gameObject)))        {            /* TODO: DEVELOPER CODING EXERCISE 2.c */            // 2.c: Calculate rotationFactor based on GestureManager's NavigationPosition.X and multiply by RotationSensitivity.            // This will help control the amount of rotation.            rotationFactor = GestureManager.Instance.NavigationPosition.x * RotationSensitivity;            // 2.c: transform.Rotate along the Y axis using rotationFactor.            transform.Rotate(new Vector3(0, -1 * rotationFactor, 0));        }    }    void PerformManipulationStart(Vector3 position)    {        manipulationPreviousPosition = position;    }    void PerformManipulationUpdate(Vector3 position)    {        if (GestureManager.Instance.IsManipulating)        {            /* TODO: DEVELOPER CODING EXERCISE 4.a */            Vector3 moveVector = Vector3.zero;            // 4.a: Calculate the moveVector as position - manipulationPreviousPosition.            moveVector = position - manipulationPreviousPosition;            // 4.a: Update the manipulationPreviousPosition with the current position.            manipulationPreviousPosition = position;            // 4.a: Increment this transform's position by the moveVector.            transform.position += moveVector;        }    }}

1. build测试一下吧！
   使用“Move Astronaut”语音控制从旋转切换为移动手势
   

5 模型展开/爆炸

将模型展开成碎片，每个小块都可以交互
每个碎片都可以单独旋转和移动
1. 编辑 AstronautManager.cs 脚本，按照注释5.a提示补全代码，最终代码如下：

using Academy.HoloToolkit.Unity;using System.Collections.Generic;using System.Linq;using UnityEngine;using UnityEngine.Windows.Speech;public class AstronautManager : Singleton<AstronautManager>{    float expandAnimationCompletionTime;    // Store a bool for whether our astronaut model is expanded or not.    bool isModelExpanding = false;    // KeywordRecognizer object.    KeywordRecognizer keywordRecognizer;    // Defines which function to call when a keyword is recognized.    delegate void KeywordAction(PhraseRecognizedEventArgs args);    Dictionary<string, KeywordAction> keywordCollection;    void Start()    {        keywordCollection = new Dictionary<string, KeywordAction>();        // Add keyword to start manipulation.        keywordCollection.Add("Move Astronaut", MoveAstronautCommand);        // Add keyword Expand Model to call the ExpandModelCommand function.        keywordCollection.Add("Expand Model", ExpandModelCommand);        // Add keyword Reset Model to call the ResetModelCommand function.        keywordCollection.Add("Reset Model", ResetModelCommand);        // Initialize KeywordRecognizer with the previously added keywords.        keywordRecognizer = new KeywordRecognizer(keywordCollection.Keys.ToArray());        keywordRecognizer.OnPhraseRecognized += KeywordRecognizer_OnPhraseRecognized;        keywordRecognizer.Start();    }    private void KeywordRecognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)    {        KeywordAction keywordAction;        if (keywordCollection.TryGetValue(args.text, out keywordAction))        {            keywordAction.Invoke(args);        }    }    private void MoveAstronautCommand(PhraseRecognizedEventArgs args)    {        GestureManager.Instance.Transition(GestureManager.Instance.ManipulationRecognizer);    }    private void ResetModelCommand(PhraseRecognizedEventArgs args)    {        // Reset local variables.        isModelExpanding = false;        // Disable the expanded model.        ExpandModel.Instance.ExpandedModel.SetActive(false);        // Enable the idle model.        ExpandModel.Instance.gameObject.SetActive(true);        // Enable the animators for the next time the model is expanded.        Animator[] expandedAnimators = ExpandModel.Instance.ExpandedModel.GetComponentsInChildren<Animator>();        foreach (Animator animator in expandedAnimators)        {            animator.enabled = true;        }        ExpandModel.Instance.Reset();    }    private void ExpandModelCommand(PhraseRecognizedEventArgs args)    {        // Swap out the current model for the expanded model.        GameObject currentModel = ExpandModel.Instance.gameObject;        ExpandModel.Instance.ExpandedModel.transform.position = currentModel.transform.position;        ExpandModel.Instance.ExpandedModel.transform.rotation = currentModel.transform.rotation;        ExpandModel.Instance.ExpandedModel.transform.localScale = currentModel.transform.localScale;        currentModel.SetActive(false);        ExpandModel.Instance.ExpandedModel.SetActive(true);        // Play animation.  Ensure the Loop Time check box is disabled in the inspector for this animation to play it once.        Animator[] expandedAnimators = ExpandModel.Instance.ExpandedModel.GetComponentsInChildren<Animator>();        // Set local variables for disabling the animation.        if (expandedAnimators.Length > 0)        {            expandAnimationCompletionTime = Time.realtimeSinceStartup + expandedAnimators[0].runtimeAnimatorController.animationClips[0].length * 0.9f;        }        // Set the expand model flag.        isModelExpanding = true;        ExpandModel.Instance.Expand();    }    public void Update()    {        if (isModelExpanding && Time.realtimeSinceStartup >= expandAnimationCompletionTime)        {            isModelExpanding = false;            Animator[] expandedAnimators = ExpandModel.Instance.ExpandedModel.GetComponentsInChildren<Animator>();            foreach (Animator animator in expandedAnimators)            {                animator.enabled = false;            }        }    }}

1. build来测试一下！
   

说“Expand Model”来展开模型
可以旋转单独的碎片
说“Move Astronaut”来移动碎片
说“Reset Model”来恢复模型整体

---

洪流学堂，最科学的Unity3d学习路线，让你快人一步掌握Unity3d开发核心技术！