Grab a Painting

Thought catching

Thinking of a painting, a picture, I realised how cool it would be if you can just make a frame pose with your hands and a painting appears in front of you instead of one already hanging in the air on app start!

1: Adding Hand Tracking

So I used Oculus Integration in this project. But now, when I am writing this article, I have used the latest Meta All-in-one SDK from the asset store in another project; and that makes it super easy to set your scene up for anything XR you want 😎. So, here I would explain how to setup hand tracking the easy way.

So go on and add Hand Tracking and Synthetic HandsBuilding Blocks.

This is how my hierarchy looks: -

Also, assign Synthetic Hand Left/Right in the right place (see below) to avoid any errors.

And that's all - your scene now has hand tracking.

2: Hand Pose Detection

Again it was Valem to my rescue. I tried setting hand pose detection using the same playlist I shared in the Hang a Painting post but something was not right. It was not working. I tried using the new Meta XR SDK's Building Blocks to do the same but couldn't get it working.

Then I went to the documentation which gave me a good idea how to set things up. Finally, I thought of checking out the sample scene provided with the SDK and of course that was working.

So, there was also a FramePoseTwoHanded hand pose prefab already provided with the SDK but that seems to be too specific and not what I wanted. Thus, I first tested the original ones in the scene and then tried creating my own Frame Hand Pose.

My Two-Handed Frame Hand Pose

So I wanted it to be something like this. This is basically composed of a few components: -

1. Index and thumb making an L Pose

2. All other fingers closed

3. Palms facing away from face

With my initial composition, the palm positions weren't getting detected so the L Pose was detected OK in every direction.

Then I closely examined the original FramePoseTwoHanded prefab and after making a few changes and correctly setting up ActiveSelectorGroups, the hand pose was getting detected as required.

ATTACH HAND POSE DEBUG IMG

3: Grab Transforms

The goal was to enable the user to move and scale the painting by using their hands (Hand Tracking Interaction). This would be doubly beneficial: -

1. It solves that positioning issue where the painting was not aligning with the wall

2. Adds a of feeling of really setting the painting in your room

To make the painting hand interactable, you just add these to it: -

1. OVR Grabbable

2. TwoHandTransformer

3. OneHandTransformer, as we want single hand interactions as well, and

4. Link 2 and 3 to 1 's Inspector references.

4: Painting instantiation on Pose detection

Now with all of the above set, the final thing for ver 2.0 was to instantiate the painting when the user makes a frame pose. This would not only be super cool but only apt - make a frame hand pose and tan-tan-an... a painting magically appears in front of you.

To set this up, I hooked the painting instantiation code with the Activated method of the hand pose's Active State Unity Event Wrapper.

Well that happened but the app crashes instantly afterwards.

Coming Up Next...

➡️Crash fix

➡️Spatial anchors