Milestone 1 is done. I have called it “Exist” because the goal is just to make the application exist. Took 8 work days, Jan 6th (Wed) to Jan 14th (Thu) and 60 hours recorded in Teamwork (59 hours, 55 minutes).
I have three goals for this post:
- Record history – have a place to see what happened
- Not waste time – to keep this going I need to be quick about these posts
- Help people – I hope info here can help others but I realize that in order to be really helpful I would need to put a lot more time and details into this. So! If people request expansion on specific subjects in this blog, I will then spend time to write about those specific subjects.
So what happened in Milestone 1?
From a high level these are the things that happened.
- Fake press release
- Created Base Project
- Created Global Illuminated room environment
- Created 3D character (Ella Bella)
- Added basic activity to Ella Bella (Interactive Agent)
- Hooked up 3D sound
Fake Press Release
Before starting I wrote a fake press release to imagine what people’s reactions should be to the product. This is to get a sense of what is probably important from the outside. There’s another page but this should be enough for you to see what I did.
I created a base project where Oculus SDK libraries are better contained. In this I separated out the VrCubeWorld code from the main app. All Oculus libraries are in the “Vendor” branch. Here’s a link to the base project I extracted from my project on GitHub:
GI Room Environment
My original plan was to bake out stereo cubemaps for the environment. The idea being that I could have super highly detailed and super nicely lit environment for the game at a low cost. This ended up being a failure. For a couple of reasons:
- Plugin I wanted to use to collect stereo cubemaps costs 300 Euros
- Oculus 360 Photos app does not have stereo cubemap code currently. The released app does but the SDK on Oculus website doesn’t appear to. I could be wrong but pretty sure I didn’t see it there in the cube map drawing area.
- Time – I actually came up with a few possible solution ideas but decided not worth pursuing right now.
So what did I do?
Decided to do a nicely lit Global Illuminated environment in Unity, bake out a cubemap, and just look at it mono-scopic. Surprisingly, this actually looks pretty darn good. Apparently if you get lots of good lighting info baked into your environment and at a high resolution it looks quite good. There are a few spots that look bad though, like the “chandelier” above.
In Unity I used, “Instant High-Res Screenshot” plugin to collect my cubemap. Although I had to hack a change in to enable Anti-aliasing. This made a HUGE difference in how nice the cubemap looked.
Cube Map – with 4x AA
Cube Map – No AA (Jaggies most noticeable on Yellow Cube)
After collecting the 6 sides, I put them into PVRTexTool to make a .pvr file. This natively supports cubemaps AND the Oculus SDK automatically detects the cubemaps in pvr or ktx files and will properly put into cubemap texture in OpenGL.
Then I drew a cube and used that texture. I really hope to get back to stereo cubemaps in the semi-near future.
3D Character (Ella Bella)
I considered loading up an FBX model but decided against it. For that to happen I would need: make a model (probably in Blender – which I would have to get refreshed with), convert the FBX to OVRScene file, load that model. It doesn’t look like much but I didn’t want to spend that time right now so I generated a model from code.
Artificial Intelligence – a dirty word(s)
The latest lessons I studied (http://courses.nucl.ai/) do not like those words so I will call this Interactive Agent instead.
My Interactive Agent (IA) is super simple right now but consists of 3 “brains” connected together using a messaging system. The 3 brains are (behavior, navigation, movement). This actually worked out really well. Although I spent A LOT of time building up a generic messaging system for them to use. C++11 had a lot of new magical things that made this easier than it used to be. Generally I was able to create a few hash tables with [ID, Function] pairs and some [String, ID] pairs to make things a little more user friendly. I’m not 100% sure of the performance implications but I am using std::function with std:bind to setup my callbacks which works nicely when dealing with classes and callbacks. I’ve also liberally used ASSERTs. I would prefer to had more compile time checks on the communication between the modules but couldn’t find smart ways to do this without getting too creative.
Right now Ella chooses semi-random locations to move around to. She then faces and “walks” around. She will then turn to you and sing every so often… in 3D!
3D Sound – FMOD
I already had FMOD hooked up to the project, so this was mostly about hooking Oculus headset to the listener and allowing Ella to sing in 3D.
Ran into only a few issues: FMOD is left-handed, OpenGL is right-handed, needed to slightly restructure to allow the audio manager get messages from the “brains” when they wanted to play sounds.
I am using the standard FMOD for Android, no special Oculus stuff for 3D sound.
That’s a Wrap
Capturing video for games/apps on GearVR is hard because no direct way to do this at a decent framerate. So I’m going to put a couple screenshots and maybe take a video with my other phone. That feels so ghetto. I suppose this makes sense. We are trying to capture this full 3D, 360, Stereo experience and trap it into a little rectangle on your computer screen. Maybe that is supposed to be difficult.
Ella Bella – Milestone 1