• Tech News
    • Games
    • Pc & Laptop
    • Mobile Tech
    • Ar & Vr
    • Security
  • Startup
    • Fintech
  • Reviews
  • How To
What's Hot

Elementor #32036

January 24, 2025

The Redmi Note 13 is a bigger downgrade compared to the 5G model than you might think

April 18, 2024

Xiaomi Redmi Watch 4 is a budget smartwatch with a premium look and feel

April 16, 2024
Facebook Twitter Instagram
  • Contact
  • Privacy Policy
  • Terms & Conditions
Facebook Twitter Instagram Pinterest VKontakte
Behind The ScreenBehind The Screen
  • Tech News
    1. Games
    2. Pc & Laptop
    3. Mobile Tech
    4. Ar & Vr
    5. Security
    6. View All

    Bring Elden Ring to the table with the upcoming board game adaptation

    September 19, 2022

    ONI: Road to be the Mightiest Oni reveals its opening movie

    September 19, 2022

    GTA 6 images and footage allegedly leak

    September 19, 2022

    Wild west adventure Card Cowboy turns cards into weird and silly stories

    September 18, 2022

    7 Reasons Why You Should Study PHP Programming Language

    October 19, 2022

    Logitech MX Master 3S and MX Keys Combo for Business Gen 2 Review

    October 9, 2022

    Lenovo ThinkPad X1 Carbon Gen10 Review

    September 18, 2022

    Lenovo IdeaPad 5i Chromebook, 16-inch+120Hz

    September 3, 2022

    It’s 2023 and Spotify Still Can’t Say When AirPlay 2 Support Will Arrive

    April 4, 2023

    YouTube adds very convenient iPhone homescreen widgets

    October 15, 2022

    Google finishes iOS 16 Lock Screen widgets rollout w/ Maps

    October 14, 2022

    Is Apple actually turning iMessage into AIM or is this sketchy redesign rumor for laughs?

    October 14, 2022

    MeetKai launches AI-powered metaverse, starting with a billboard in Times Square

    August 10, 2022

    The DeanBeat: RP1 simulates putting 4,000 people together in a single metaverse plaza

    August 10, 2022

    Improving the customer experience with virtual and augmented reality

    August 10, 2022

    Why the metaverse won’t fall to Clubhouse’s fate

    August 10, 2022

    How Apple privacy changes have forced social media marketing to evolve

    October 16, 2022

    Microsoft Patch Tuesday October Fixed 85 Vulnerabilities – Latest Hacking News

    October 16, 2022

    Decentralization and KYC compliance: Critical concepts in sovereign policy

    October 15, 2022

    What Thoma Bravo’s latest acquisition reveals about identity management

    October 14, 2022

    What is a Service Robot? The vision of an intelligent service application is possible.

    November 7, 2022

    Tom Brady just chucked another Microsoft Surface tablet

    September 18, 2022

    The best AIO coolers for your PC in 2022

    September 18, 2022

    YC’s Michael Seibel clarifies some misconceptions about the accelerator • DailyTech

    September 18, 2022
  • Startup
    • Fintech
  • Reviews
  • How To
Behind The ScreenBehind The Screen
Home»Tech News»The Design & Implementation of Oculus Quest Hand-tracking in Myst
Tech News

The Design & Implementation of Oculus Quest Hand-tracking in Myst

August 6, 2022No Comments9 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
The Design & Implementation of Oculus Quest Hand-tracking in Myst
Share
Facebook Twitter LinkedIn Pinterest Email

Using Presence Platform’s upgraded Hand Tracking API, we introduced Hand Tracking with our most recent update to Myst on the Meta Quest Platform, titled ‘Hands & More’. We’re super excited to finally let folks play Myst on Quest without physical controllers! In this post, we’ll discuss the evolution and iteration of implementing hand tracking in Myst—and in particular, adding more support for it in Unreal Engine 4.27.2.

Guest Article by Hannah Gamiel

Hannah Gamiel is the Development Director at Cyan—the studio behind the original ‘Myst’ games—and helped develop the new ‘Myst (2020)‘ which includes VR support. Originally coming from a purely technical background, she now helps lead production on all titles and manages business & tech efforts at Cyan. She has worked on titles such as ‘Myst’ (2020), ‘The Witness’, ‘Braid, Anniversary Edition’, ‘Obduction’, ‘Firmament’ (coming soon!), and more.

Design Phase & Considerations

Designing Navigation for Hand Tracking

Picture indicating where you’d like to go. You likely thought of pointing, right? That’s why we opted to use a ‘pointing’ method for movement in Myst.

When you’re in teleport mode, you can point to where you’d like to go and the teleport ring appears at your destination. When you ‘un-point’ (by extending the rest of your fingers, or simply pulling your pointer finger back into your palm), the teleport is executed.

When you’re in smooth movement mode, pointing with your free-movement-dominant hand (which can be configured in our controls settings, but is the Left hand by default) will begin smoothly moving you around in the direction you’re pointing.

When playtesting movement with pointing, we found that hand tracking can sometimes be unreliable with your pointer finger and middle finger when it is occluded by the rest of your hand. The system isn’t sure whether those fingers are fully pointing or fully ‘enclosed’ in your hand. We added a bit of a ‘fudge’ factor to the code to account for more stable movement initiation/execution on this front—which we’ll go into a bit later when we discuss changes made to out-of-the-box Hand Tracking support in Unreal Engine.

Turning

The ‘point’ method doesn’t work for all navigation usages. When it comes to turning, we initially combined pointing with wrist rotation. Comparing the player’s wrist and the camera’s forward vector would indicate the direction of the turn (and how big the turn should be).We tried this initially since it seemed intuitive to keep the ‘pointing’ theme for navigation going between all modes.

Complications arose in comfort tests, however. In playtesting, most players would point forward with their palm facing the ground, as one likely would when attempting to point at something outside of a game as well. Rotating your wrist to the left and right (around the up axis of your wrist) while you have your palm facing the ground is challenging and has a very limited range of motion, especially if trying to turn away from your chest.

See also  Quest v43 Update May Include Long-awaited 16:9 Recording Mode – Road to VR

This issue is the same even if you asked a player to point at something in front of them with their palms facing inward. You can bend your wrist in toward your body quite a bit, but you won’t get the same range of motion bending your wrist away from your body.

So how did we solve this? We ended up assigning turning to a ‘thumbs-up’ gesture instead of a pointer-finger-pointing gesture.

Imagine giving a thumbs-up. Now turn your wrist right and left. Note that even though you don’t have a huge range of motion it’s still fairly consistent pointing either ‘left’ and ‘right’ with your thumb in this gesture.

This is what we settled on for turning in hand tracking mode. Although pointing with your thumb doesn’t seem like the most intuitive way to turn, it did end up being the most comfortable and consistent way of doing so.

With snap turning, rotating your wrist to the left or right from a thumbs-up position causes a single snap turn to initiate. You then have to return your hand to the ‘center’ (straight up) position in order to reset the snap, and additionally wait for a very short cooldown to occur to initiate a snap turn again.

With smooth turning, turning your wrist while in a thumbs-up position will begin rotating you left or right—once you leave a ‘dead zone’ that prevents a turn from occurring until you pass the threshold.

Handling Conflicts Between Movement & Object Interaction Poses

Of course, pointing a finger is too broad of a gesture to be assumed to only just be used for navigation. People will make the same pointing gesture to press buttons or interact with other things in the world just out of habit or their own expectation. It would be pretty jarring to walk up to (but not right up to) a button, point your finger to press it, and then suddenly (and unwantedly) move closer to it in-game (or initiate a teleport unintentionally)!

The way we prevent movement from occuring while the player may be interacting with something is by preventing any movement code from firing off when the hand making the ‘move’ gesture is within a certain range of an interactable object. This range has been tweaked multiple times to get to a good ‘sweet spot’ based on playtesting.

We’ve found that this sweet spot is around 25 cm from the world space location of the bone of the tip of the index finger. Myst is full of interactive objects of various sizes (everything from small buttons to very large levers) arrayed in both wide-open spaces and narrow hallways, so it took us quite a bit of testing to settle on this number. We initially tried 60 cm (about two feet), but that prevented movement from occurring when players still needed to get closer to an object. Likewise, anything below 25 cm caused undesired player movement to trigger when players were trying to grab or touch an object.

See also  'Date Me' Google Docs and the Hyper-Optimized Quest for Love

One of our best testing areas was the generator room on Myst Island, where you make your way through a narrow entryway and are then immediately greeted by a panel full of buttons. When the interaction testing area was too large, players were unable to move through the entry and toward the panel because it detected buttons within range of the index finger.

That said, 25 cm is what worked specifically for Myst. Other games may need to adjust this number if they want to implement something similar, with their own criteria in mind.

Designing Object Interactions for Hand Tracking

Right now, all grabbable interactions in Myst are built to work with hand tracking—turning valves, opening doors, pressing buttons, turning book pages, and so on.

The interactions piggy-back off what we had already set up for Myst with Touch controllers. There, pressing the grip button automatically blends the in-game mesh representation of your hand into a ‘grabbed’ pose, either putting your hand into a fist (if empty) or grabbing an object. With hand tracking, we’ve added code that will make a qualified guess as to when you have curled your fingers enough to ‘grab’ something and initiate the same logic as mentioned before.

For example, when you’re using hand tracking and your hand hovers over something that’s grabbable, your hand color turns orange (this is exactly what happens when you don’t use hand tracking in Myst VR as well). When you grab an interactable object by beginning to curl your fingers into a fist, an orange sphere replaces your hand mesh and represents where the hand is attached to the object.

The reason why we went with this method instead of making custom poseable meshes for your hands—or allowing your hands/fingers to appear to physically interact with portions of these objects—is because we wanted the interactions to be at parity with what we offer on the Touch controller side for now.

Pushing buttons works differently though. There’s no need for abstraction since buttons aren’t grabbable objects, and instead we allow you to simply push a button using generated capsule colliders between each of the finger joints on the poseable hand mesh. You can do all sorts of weird and fun things because of this—like using only your pinky or the knuckle of your ring finger to interact with every button in the game, if you really want to.

See also  Developing Rich VR Interactions Faster & Easier Using UltimateXR

This implementation differs slightly from the way Touch controllers interact with buttons in the game in that we usually expect players to use the grip button on their controller to set the hand to be a posed ‘finger pointing’ mesh to get an accurate in-game button press on their end. With hand tracking, there’s obviously significantly more flexibility in the pose you can create with your hand, and therefore significantly more ways to press buttons with the same level of accuracy.

Menu/UI Interactions

For interacting with menus, we ended up going with the same interaction paradigm that Meta uses for the Quest Platform: a two-finger pinch between thumb and index finger, on either hand. This can be used both to open our in-game menu and interact with elements in the menu. No sense in reinventing the wheel here when players are already taught to do this in the OS-level menus when they first enable hand tracking on Quest!

Communicating All of This to the Player

Because hand tracking isn’t as common an input on Quest as Touch controllers, and because there may be some people playing Myst for the very first time (or even playing their very first VR game!), we tried to be considerate with how we communicate all of this information about hand tracking to the player. We made sure to include another version of our “controller diagram” specifically tailored to describe hand tracking interactions (when enabled in Myst), and we show the player specialized notifications that tell them exactly how to move around with their hands.

Additionally, we thought it would be vital to remind the player how to have a smooth hand tracking experience, once enabled. The player is notified in Myst’s menu that hand tracking stability is much better if they ensure they’re in a well-lit room and keep their hands within their field of view.

Meta also informs players that these are key to a well-tracked hand tracking environment, but we recognize that some players might jump into a game not having parsed Meta’s notices about this first, so we’ve chosen to remind folks in case they forgot.

Continue on Page 2: Engine Modifications Made in Unreal »

Source link

Design Handtracking Implementation Myst Oculus Quest
Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Xiaomi’s Smart Band 8 Pro is a cheap and easy way to track health, control media and get notifications in an Apple Watch-style design

March 26, 2024

Meta Quest 3 review

October 9, 2023

Spoutible’s Low-Budget, Audacious Quest to Be the Next Twitter

June 13, 2023

One Man’s Quest to Revive the Great American Vacuum Tube

March 28, 2023
Add A Comment

Comments are closed.

Editors Picks

Chrome for iPhone gets first update since November

July 9, 2022

Soulframe is the fantasy spin-off of long-running on-line motion RPG Warframe

July 17, 2022

Congress Has a Lo-Fi Plan to Fix the Classified Documents Mess

February 5, 2023

The Division: Heartland closed test sign ups now open

September 11, 2022

Subscribe to Updates

Get the latest news and Updates from Behind The Scene about Tech, Startup and more.

Top Post

Elementor #32036

The Redmi Note 13 is a bigger downgrade compared to the 5G model than you might think

Xiaomi Redmi Watch 4 is a budget smartwatch with a premium look and feel

Behind The Screen
Facebook Twitter Instagram Pinterest Vimeo YouTube
  • Contact
  • Privacy Policy
  • Terms & Conditions
© 2025 behindthescreen.uk - All rights reserved.

Type above and press Enter to search. Press Esc to cancel.