Group VR Project: Ikebana LearningIndividual VR Project: No Man Is An IslandIndividual AR Project: No Man Is An Island

AR/VR Mini Projects

AR/VR Design; Course Project
Project Overview
Methods& Tools
video editing; physical prototype with playdoh and cardboard; digital prototype in Unity& Vuforia Engine
People
Ikebana Learning in VR: Me and four team members

No Man Is An Island (AR&VR): Me
Time
Sep 2021- Dec 2021

Ikebana Learning in VR

Problem Space

What Is Ikebana

Ikebana (生け花, 活け花, "arranging flowers" or "making flowers alive") is the Japanese art of flower arrangement.

Due to the pandemic, fewer offline flower arrangement lessons are available than in the past. In a remote learning setting, videos and books are ineffective learning mediums without real-time hands-on instructions.

Why Learn Ikebana in VR

1. Fewer offline courses are offered since Covid
2. More 3D hands-on learning than online videos
3. Final arrangements can last forever
4. Get access to unlimited materials
5. More environmental-friendly and no more plant-killing

Why Choose VR Not AR

1. VR gives users more control over the setting (lighting and environmental surroundings), which is also an important aspect of Ikebana design.
2. A VR classroom is more immersive than AR. Users can feel they are in a virtual classroom without getting distracted by messy home environments.

Quick Lookup of Digital Prototype

Customize your vase and flowers

Arrange 3D flowers

Like peers' works and see your likes on the inspiration board

Storyboard and 360 Sketches

Storyboard Main Features

1. Select vases and flowers
2. Peers come over using controllers and give feedback
3. Highlight suggestions with a different color
4. Accept or reject suggestions

360 Sketch

Users are in a virtual classroom setting surrounded by peers and instructors.

Physical Prototype

This was made with papercuts, cardboards, and play-doh.

The whole environment was prototyped to visualize the full VR classroom setting.

Design Iteration

Feedback

The instructors suggested that we might want to:
1. Explore more interactions between instructors and students
2. Explore more possible interactions between peers, not just changing others' arrangements directly

Improvements

1. We made a practice mode. When instructors arrange their flowers, their vases will appear on the user's table. Users can follow instructors' processes to practice skills before starting their individual work. Users can also replay the instructors' arranging processes during individual work time.

2. For peer review, users will be able to switch to other tables, send likes to others, and collect the works they like on their inspiration board. After they return to their table, they can drag the vases they like out of the board to their table for inspiration.

Digital Prototype

Core Features

1. Real-Time Peer Interaction And Inspiration Board
2. Create And View 3d Flower Design Pieces
3. Real-Time Peer Interaction And Inspiration Board

 Limits and Future Work

LIMITATIONS

1. Users have fewer opportunities to learn the species of flowers and pick up the preferred one among a bunch of similar flowers.
2. Users can not touch nor smell the flowers, which is also a part of the experience

FUTURE WORK

1. Give users more customizability of surroundings (adjust lighting, environments, sounds)
2. Support more theory learning

APPLICATION

1. Our work can be applied to other art-craft education
2. Make a 3D flower exhibition with our models

No Man Is An Island (VR version)

Problem Space

Isolation causes lots of mental issues.

Isolation Since Covid

Since Covid started, people spend most of their time staying at home and much less time with other people. There have been many research reports arguing how isolation caused by Covid has negatively influenced our mental health. Although in-person meetings and physical contacts are highly restricted and unsafe, gatherings can be easy and accessible in XR. No matter where people are, as long as they have access to VR gear or AR app, they can meet friends or strangers in XR and enjoy the freedom to chat about anything without concerns about privacy since they wear avatars in XR and can even have some virtual supportive body contact with others.

Everyone is so isolated on their own islands, but we should keep in mind that we are never left behind. Many people will be there for you when you are in need. No man is an island.

Quick Lookup of Digital Prototype

Customize your vase and flowers

Users can pick up and open the bottles to check the messages inside.

Island and drifting bottles

To simulate our isolated experience because of social distance, players in this game will sit on a small island, where they can see nothing but the dark sea. Drifting bottles are the only light sources that come to your island from a distance to bring you hope.

Interact with others

Users can record a video message and send it to others. They can also hug, pat on others' shoulders, or pat on others' heads to give them some virtual physical support.

Storyboards

A user stays on his island and feels isolated. Then he notices some drifting bottles, so he wants to open the bottles to see what’s inside.

He finds video messages from others in the bottles and can reply to the messages. His motion, facial expression, and words will be recorded. He can give other people some support with virtual physical contact and words.

Physical Prototype and 360 Sketch

Digital Implementation of Key Interactions in Unity

Main Features

1. Simulate users walking to the shore using MSTK
2. Simulate bottles approaching the island using ProjectileMotion script
3. Interactions with objects and controllers using MSTK object manipulator

Credits:Realtime Rascals, Island Assets, Potions, Real Stars Skybox

Reflection

Unity is a powerful platform with plenty of plugins and assets. Although it is still not so user-friendly as design prototyping tools like Sketch or Figma as for now, I think they will become the next generation sketch if they keep iterating and make more drag-and-drop code modules to eliminate the burden of coding for designers. Honestly, my experience with unity reminded me of learning C++. Solve one bug then another comes. I guess it's inevitable that 3D design is much more complex than 2D prototyping since the designer acts as the director, scriptwriter, light man, and cameraman at the same time, but I still hope unity can be more intuitive than C++. At least let me know what I should do about those compiler errors. But I think importing assets to unity is pretty smooth from their asset store, which is a plus. In the second week, I got more familiar with this tool and started to like it more after I was able to fix some bugs. I think this is a really cool and powerful tool to use for VR design.

No Man Is An Island (AR version)

Quick Lookup of Digital Prototype

Pick up and open bottles

Listen to messages from others

Reply to messages with your avatar

Storyboards

1. Users see waves and an island under their feet, as well as bottles approaching them.
2. Users pick up the bottles and place them with guidance.
3. Users open the bottles. The markers are on the inner side of bottle caps.
4. After scanning the marker, users can listen to messages from other avatars and reply to them in their avatars.
5. Users can also send out their own messages.

Digital Implementation of Key Interactions with Vuforia Engine in Unity

Main Features

1. Pick up bottles with sound effect
2. Scan markers on bottles and watch messages
3. Reply to others’ messages in avatars

Reflection

I met many problems with AR digital prototyping platforms. Lens studio and snap camera never worked for my laptop since both of them said there were network issues. I couldn't make self-designed markers work in Unity even if I add them to databases. It was very frustrating when my video got stuck quite often in the game mode while recording my screen. Video players failed to pop up when I scan the marker image, but my instructor helped me replace it with a quad and place the video on the quad. All in all, I got my first taste of AR design and implementation during this process and got a decent outcome!