Untitled Augmented Reality

Research

The first stage of the project was to understand and research current projects and technologies in this field. I spent a significant amount of time looking at projects that tackled similar problems.

I had prior experience with AR, but this project stretched my level of knowledge beyond my initial understanding of the technology. It took a lot of trial and error before I was able to efficiently deliver working prototypes in a short timespan.

IDEATION

After the initial research, I spend a few weeks coming up with sketches and ideas on what to build and research. During the stage of early ideation, I built several small prototypes to test out theories and better understand the overall problems with developing an AR experience.

Most of these ideas were quick prototypes that allowed me to get feedback by testing with other user.

Findings

A number of initial ideas were validated or completely disproved during the early stages of testing. All final designs, interactions, and details were a results of extensive building and constant testing.

Interactive and immersive mechanics designed and built for AR were proven to be very hard to accurately user test without properly working prototypes. This made the project very time-consuming and often focused around debugging.

AR experience without a single line of code.

Unreal Engine 4 was a powerful tool to use in the creation of an Augmented Reality experience. It allowed me to build fully functional prototypes using only blueprints and visual scripting. I was able quickly prototype ideas without writing a single line of code.

Designing interactions FOR augmented reality.

Building interactions for Augmented Reality was an exciting and often complicated task. Several different modes and from of interactions were explored during the research process. A combination of physicality and virtual interactions was the initial idea, but do to complications with interaction and user experience, a simpler mechanic and solution needed to be developed.

BUILDING, Prototyping, and debugging.

A significant portion of the project was focused around rapid prototyping and constant user research. The development of this project was significantly more difficult do to the deployment of AR experience on mobile devices. Almost every new change, mechanics, and ideas needed to be ported to the phone, tested, evaluated, and improved. To properly test and interact with the experience, the project almost always needed to be ported and packaged onto the target device.

building ar experience on different sizes and devices.

It was important to build the project with different screen sizes in mind to make sure it would look great on every screen sizes. All UI and menu elements were built with this flexibility in mind. Using UE4's widget system, UI elements and pages only needed to built once, which would dynamically adjust to different screen sizes. This made the project very easy to deploy on iPhone and iPad.

Exploring different levels of immersion.

Augmented Reality offers a level of immersion that is the ideal mix of real world physicality and added visual experience. During the development of the project, I had explored different levels of immersion to decide what would be the ideal balance for this AR experience. How much of the real versus virtual would is experienced during the interactions and gameplay.

DEVELOPING FOR AR

One of the biggest difficulties for developing an AR app was the pipeline to get the project onto the device and playtest it as quickly as possible. It was difficult to test AR functions and features inside the editor, so almost all iterations needed to be deployed on the target device.
‍

INTERACTIONS

Interacting with a Virtual World on a 2D screen was an interesting and challenging research task. I explored a combination of physical and visual interactions before a more polished and simplified experience was built.

STORYTELLING

Storytelling was a difficult part of the project since the user is given a view and camera into the world that they have full control over. To make sure that a story could be told while not obstructing the gamapley, several different strategies and directions were explored.

You can learn about the story by finding the "memory totem," which acts like a type of scavenger hunt in the experience. One you find these object and interact with them, you can learn bits of the story that ate the end, plate player needs to put together.

testing, testing, and testing.

Every single version of the experience was tested and evaluated. Each demo provided new insights and lessons that were applied to newer versions.

What made testing significantly more difficult was the constant need to deploy the app on the mobile device. It was proven to be difficult to test out ideas in AR without having a properly build early prototype for it. User often asked if the interaction is buggy or it meant to be like that.

Along the process, I developed quick solutions and tools that allowed me to debug and test out different iterations without re-packaging the entire project.

Understanding user interactions and identifying problems.

Almost all iterations of the project were user tested and played on a mobile device. It was important to understand and research how users interact with the AR environment without much of an explanation or previous experience.

A number of valuable feedbacks were incorporated from these testing into future versions of the project. Some of the ideas that seemed easy to use and understand, turned out to be the more diffiult mechanics, that made interacting with AR complicated.

project INSIGHTS AND SOLUTIONS.

The entire project was a huge learning experience. There was a number of questions and unknowns that needed to be explored.

Building for quick testing: I spent a significant amount of time just building "grey box" levels and actors that allowed me to quickly test ideas. As a result of this process, with every iteration, I got faster at building prototypes.

Questioning design decisions: Every design decision and solution was constantly questioned and evaluated. More often than not, ideas and solutions were completely scrapped and started fresh. Questioning the why, who, and how was key element of the project.
‍
Surfacing unexpected problems: While prototyping the experience, each iteration and build surfaced new interesting problems that were left completely unexplored before. This cerated a very rewarding learning curve and enhanced my understanding of AR development.

Levels of immersions: The project explored several different levels of immersions and to find a balance between blending the physical and virtual space. The final design makes the experience unique by pushing the augmented reality view to the limits.

what are the things that went wrong?

The initial idea of the project was completely changed after several iterations. Almost none of the original ideas made it into the final version of the experience, except one or two. While the end result of the project was significantly better than the original idea, it was an extremely insightful experience to learn and follow the evolution of a project.

There was a significant amount of time spent on figuring out problems with the AR technology itself in Unreal Engine 4. Packaging the application got more difficult with more complexity. Bugs within the engine with the AR technology often slowed the progress of the project down and quick solutions needed to be developed until fixes arrived.

With every project iterations, new problems came to the surface. The project was a constant marathon of problem solving and user testing. While creating a solution for a problem, it often occurred that a new problem was created as a result.