Experiential Design - Task 3

 

Task 3: Prototype

Based on the project proposal we submitted earlier, Angel and I began delegating tasks and experimenting with building the core functions. Our current concept combines both of our previous ideas: I proposed using animation to tell the story of each constellation, while Angel suggested incorporating gamification, allowing users to discover constellations in the sky and collect them. We've decided to proceed with the elements we each proposed. You may click here to refer back to our project proposal. Below is the task that I have done for this prototype:

Me: Drawing of icons and illustration for 6 constellations, Animation of star linking to each other, Button toggle between camera feed and skybox, Animation of UI fade in one by one, Animation of story panel for each constellation.

Drawing of Icon and Illustration

I am in charge of creating six constellations: Aries, Taurus, Gemini, Cancer, Leo, and Virgo. Meanwhile, Angel will be working on the other six constellations. We used Procreate to draw the line art for each constellation. Together, we discussed and explored different visual outcomes to determine what would look best when displayed in the sky. In the end, we decided on an art style that features clean line art, bright pastel colors, and gradient shading.

• Canvas size: Icon: Square (2048*2048px), World space constellation: A4 (210mm*297mm)
• Brush: Ponta Fina 
• Brush size: Icon: 6px, World space constellation: 2px

Drawing using Procreate

Animation of star linking to each other

For the star animation, instead of creating it in Blender, I decided to try building it directly in Unity, as I wanted to control the animation sequence through scripting. I used the Glowy material in Unity and applied it to 3D spheres to represent the stars. To create the connecting lines between the stars, I used thin cylinders. Each cylinder starts with a scale of 0 and then elongates to form a line. I created each animation individually and used scripting to control the sequence in which the stars and lines appear.

Group of elements for each constellation

Script to decide the sequence of the animation

Button to toggle between camera feed and skybox

I created two cameras in Unity, one for the virtual view and one for the real camera view. Each camera has a UI button that allows users to toggle between them. I have tried two methods, one is using the script to toggle the button and used the OnClick() function to switch between the cameras, and the toggle functionality itself is working fine. However, the issue I'm currently facing is that the skybox does not move with the camera, and I'm still working on fixing that.

Button OnClick()

Animation of UI fade in one by one

For the UI, I imported images from Figma into Unity and converted them into sprites. There are two types of UI used in the project: one in Screen Space and the other in World Space. The World Space UI is used for constellation illustrations. For example, when a user clicks on a star and the animation begins to connect the stars, an illustration will appear in the sky. The Screen Space UI is used for interface elements such as text (e.g., “You have unlocked XXX”) and buttons to play animations or close panels. 

Screen Space UI

World Space UI

Script for UI fading

Animation of story panel for each constellation

For the storytelling of each constellation, I created a dedicated UI panel for each one. I added illustrations to support the story and included a button to navigate to the next panel. There are still two types of UI: world space UI for the animated illustrations, and screen space UI for the narrative text and buttons. For the logic, I used the OnClick() event to manage the sequence of the story panels.

Screen Space UI

World Space UI

On Click () Event

However, before that, an issue arose where clicking the button triggered the wrong UI panel to appear. At first, I thought it was a bug, but I later realized that I should have set the unused UI elements to inactive by default. Only when the button is clicked should they become active, to prevent UI overlap that could cause the wrong panel to pop up. Nevertheless, this issue has been resolved thanks to Mr. Razif’s help!

Testing on Android Phone and Follow Up

After building the basic features needed for the project, I began testing the functionality in Unity using a webcam and an Android phone. Aside from the skybox issue, the slightly off scale since the dimension of the iPhone device and Android device is different, and the stars are a bit too small to interact with, everything else works well and functions as expected. Below is the showcase of how our constellation app looks like on a phone.

Testing on phone

This means I can now proceed with implementing the other constellations using the same logic. I then spent several hours working on the star animations, each star sphere and connecting line animation had to be created individually when clicked. It is still half-way done and this also means that I can also focus on how to make our app better since all the basic features are already set.

Placement of star for Pisces

Animation

Here is the list of tasks that are yet to be completed or improved. Our prototype has been considered quite successful, as most of the bugs we encountered have been resolved. We are looking forward to further enhancing our app!

Yet to be done:

• Fix skybox issue
• Combine with Angel's UI panel
• Arrangement of position of constellation
• Landing page
• Hide the constellation on the ground when switch to camera mode
• Sound effect and narrative voice for storytelling

Submission

Compilation Link: Google Link

Presentation Video

Reflection

Thankfully, we had learned the Game Development module earlier, so I had a basic understanding of the technical aspects. When it came to developing the AR app, aside from the initial setup such as configuring the AR camera, build settings, and 3D render pipeline, the overall logic felt familiar and manageable.

The challenging part was combining our work into one project file, since we were working in a group and each of us focused on specific features using Unity. Angel and I worked on completely different features, so we used the OnClick event to connect our parts when merging the files.

Nevertheless, on my side, the prototype performed very well smoother than I expected. Except for the skybox issue, all other bugs were resolved quite easily. I would say our project is already about 50% complete, and we now have ample time to make further improvements to enhance the user experience. We're really looking forward to refining it further!