The Weekly
As we roll out new features for Umi we post updates about characters that we're designing, and new game mechanics that we are exploring, and messy process of developing a game on a shoe string budget.
Here's a quick preview of Map to AR gameplay. We’re using Foursquare’s Pilgrim API to automatically collect objects while the player is out in the world. When they return to the app they have objects waiting for them. Here you can see the player collecting objects on the app and the feeding them to their Umi. You can also see some of the Rifts on the map as well. They’ve been added in the last month.
When you’re designing interactions one of the most challenging aspects is communicating how they should work to your team. This becomes especially challenging if your team is distributed. In our case, we have team members collaborating across three continents. We end up communicating our designs through a variety of different software: stills in Figma, through user stories written in Asana, and through using Unity itself as a design tool.
For complex interactions, I’ve had Harry, a phenomenal technical artist we’ve worked with, communicate interactions in Unity through using the timeline tool.
For a recent interaction I found that just laying out how I wanted the objects to appear in Unity was the best solution.
You can see in this video that interactions are working, but it’s not really effortless. The player has pan the map around to reach the objects. I’ll offer an update on this one, when we have a more finished version.
A big part of buildling any product is iteratively testing, and finding the multiple bugs that you introduce for each new feature that you create.
Here some fun ones:
The Character is facing the backwards when you move forward on the map. You can see this really clearly when we’re driving.
Too many venues. Part of what’s challenging about creating a location based game is getting distribution of playable locations right. Here you can see our game sites are clustered too closely together.
Orientation bug. These types of bugs are legion when you’re developing a 3D game. This video is taken from our collect objects experience.
Multiple Scene Bug. This one is a tough nut, when you link Unity Scenes together, it’s easy to get really terrible bugs from scenes not being unloaded correctly. In this video you can see the whole app getting unstabel from multiple versions of the same scene loading on top of each other. We’re still dealing with this issue. Not fun.
With all the success we had with Unity we decided to build out a 3D version of our location based game. We’d been feeling for a while that having just the 2D map killed feeling that you were in a larger game world. Using a 3D map would also allow for more gameplay on the map itself. Now that we’d figured a good portion of the AR game play it was time to see if we could innovate on this side of things.
From having done this in the past we knew that the right way to move forward was evaluate all the existing platforms and weigh the pros and cons of using them. When we started our evaluation there were still two viable options.
Both Google and Mapbox had Unity versions of a 3D Map. I was originally more attracted to the Google Unity Map because of the rich data set it could provide. Google had also built out a playable locations API. From having worked with location based gaming for a while we knew that data set was a crucial piece.
But right as we started out technical evaluation Google announced that they were closing down their platform. Bummer, but fortunate for us that it shuttered before we got halfway into integrating it.
So Mapbox it was. The API itself was great and easy to use. The problem with Unity is always getting an outside API, working with your version of Unity. Bhanu our engineer got bogged down with the integration for a weeks before we finally solved it and got the map integrated in to our project.
Here are couple images of Umi on a Unity Map. Now the real fun begins.
During an in promptu design session my friend Ashley reinforced my hunch that we needed something dark in the world. She proceeded to sketch out a game loop on paper that I still have some where. That conversation led us to create a proof of concept of the game mechanic.
In the game the player would battle evil creatures that created rifts in time and space. After liking our POC we moved on to a more complex mechanic, one where you would have to close the portals themselves to stop the rifts from spawning. Here's a video of an early proof of concept prototype. As you can see its quite rough we used simple primitives to denote the rifts just to test out the idea.
When we first started working with Harry, I was excited to have him help bring the rifts to life. I started by sharing a mood board and he came back with something pretty interesting.
From there I asked Harry to help us visualize the complete battle through using his animation and effects skills. I knew from past experience that having an animation that depicts the interaction can help clarify the interaction immensely for development.
After having a complete sequence mapped out, we went to work. There were so many details to work out and it required a fair amount of coordination between Harry and Bhanu but they knocked it out of the park. Here is video of the full sequence incorporated into AR. We're pretty excited by it.
Something else we've revisited is how to make feeding Umi more interesting. In our earlier version of the game we used a point and tap method for feeding the character. You would select and object from the drawer and place it on the ground by tapping a plane. We wanted to see if we could re-create the Pokemon Go throw interaction but with food.
We started our feeding sprint with figuring out basic collision detection on the ground.
From there we moved on to adding a really fun, sloppy eating effect. Here you can see the collision detection and particle effect combined in the app.
Recreating the Pokemon Go throwing interaction for feeding turned out to be quite a lot of work. There are a lot subtleties in the interaction. You have to be able to place object in front of the camera, then there's the little animation of the object to entire the player to interact with it, then there's all the work in calculating the physics involved of throwing an object with a gesture...what happens if the user tries to select another object?In an early prototype of the interaction the food would come ricocheting back at you.
You also have keep mind that the players position in space can change. In this video you can see that the throwing behavior works appears to work correctly, but when I move around in my environment, I discover a bug.
This is what makes game development so hard but also satisfying – to make an interaction feel simple and intuitive takes a lot of iteration both from design and development. We spent a good amount of time on this and were able to create a pretty satisfying interaction with new shaders for objects, along with a feeding mechanic that includes the character getting hungry and then becoming fat and happy at the end.
One of the biggest benefits of Umi moving to Unity is that we can call on the amazing talent from people around the world. We discovered a talented effects artist living in Athens that goes by the name of Harry Alisavakis. Harry is bringing Umi to another level by his fantastic visual effects magic.
One of the traits of an Umi is that it's very vocal in showing you how it feels. Here is a reel of Umi's emotions that Harry developed for us.
We go through a design process I give Harry art direction through Asana in the form of mood boards. Harry then goes to work and develops effects which he shares through short videos that I review in asynchronously. Here's an effect for Umi getting energized that we developed through this process.
Once we have something that we're both happy with we ship them over to Bhanu who integrates them into the app. This takes some back amount of coordination and communications our little team spread out over the world but the end result is so worth it.
We're back in New York after a lovely stay on the Big Island of Hawaii. We got so much awesome work done there. Here's an extend clip showing how good Umi has gotten a maneuvering through the world.
2020 ended up being a tough year that ended on a positive note. We paused development during the first months of the pandemic to take stock of everything that was happening and simply survive. We picked up development later in the year once we'd settled into the new normal of quarantining, social distance, and working from home. Faced with a new reality we settled in and really thought hard about how to move forward.
In the fall we made the momentous decision to shift away from Scenekit and move to Unity. We'd started building Umi in Scenekit because of our familiarity with it from an earlier project. This turned out to be a bad decision. Scenekit though powerful is not built for robust game development – it neither has Apple's full support behind or a strong community using it. We kept running into situations where we wanted to improve something we thought was a small cosmetic change and instead ended up sinking an enormous amount of effort and in the end had very little to show for it. This would happen time and time again.
We also hard a time finding outside help because so few people had worked with Scenekit. Fortunately for us, Unity, had become a viable option for us with the release of Unity as Library. This new technology allows you to integrate Unity along side native components. For us this was a huge deal, we didn't have throw away all our hard work and start again from zero. Instead we could jettison the bad ideas while repurposing logic and UI components that we had already built. Armed with a powerful new game engine we focused in on the core player experience. We gave ourselves permission and the luxury to ask fundamental questions about the core player experience.
How would we move the character in Augmented Reality? In a regular 3D game, you can move a the player in through point and click or a joystick. This is fairly trivial in a confined space of a virtual in world.
In AR, the space is unknown to us. We only have the planes that camera has detected by the user moving it around in their immediate surroundings. This poses a lot of interesting problems for a designer to solve.
In our earlier version of the app we moved the character by detecting planes and within algorithm that would translate the character back towards the center of the screen as the player moved the device around. As the player walked around the camera would detect a new plane and we would move the character towards that plane. It worked, but created somewhat jerky, haphazard motion that we never figured out how to smoothen out.
When we migrated to Unity we allowed ourselves to experiment with controlling the character's movement with a joystick. This model of control is more similar to a traditional game or like controlling an RC car in the real world. With the help of Unity state machines and blend trees we were finally able to achieve beautiful animation in the app.
With new methods of control we discovered new problems. Our character would sometimes disappear by running off a plane. We created an algorithm that would detect the edge of the last plane and lock the character's movement in that direction. It's a bit like having the character on leash that extends to the last plane that the camera has detected. We tested the character's movement extensively on all kinds of surfaces: asphalt, sand, grass, and steep inclines.
We also experimented with jumping and flight. Jumping is one of the most basic movement strategies in games and but because of that we also keenly aware of how a good jump should feel. We've been having really fun focusing on these basic interactions and how to make them feel good in AR. Here's video of Umi learning how to jump.
And here's a video of Umi running down the beach. Note how the planes dictate how far Umi can run. You can also see the character taking flight.
As we're finalizing some of the core game loops I decided to spend some time working on UI elements in the game that have been neglected for way too long.
Below are some new designs for level up screens and some iterations on a daily mission system.
One of the core game mechanics that we've been working on is creating a world for your Umi while you go about your daily life.
Objects such as food items, seeds, and fun accessories can be found near real world venues.
The video shows how the player creates a garden by spitting seeds. You can also see some basic customization options that will grow richer as we continue to develop the game.
The video is taken at Transmitter Park, in Greenpoint, Brooklyn.
The Rifts followed Umi from their home planet to Earth, and threaten to destroy our planet as well.
In the game Rifts appear on the map near items that the player is searching for. The video below shows our prototype for a battle between a Rift and Umi.
Each side fires projectiles at the other, the Rift is destroyed after receiving several hits by the player.
