In the Stream of Learning

I’m entering a phase I’m calling the “Stream of Learning” phase which I expect to last 2-3 weeks. During this time I am taking 3 online courses to improve my Android and VR development skill.

  1. Android Fundamentals Part 1
  2. Ray Tracing and Global Illumination
  3. Digital Signal Processing

The first class is obviously helpful. The main challenge I’ve found with doing VR development on GearVR is dealing with Android and its build/development environment. Learning what a “normal” Android App looks like is helping very much.

The next two seem speculative but thus far I have found to be very helpful. The advanced graphics class will help with much of the offline processing I plan on doing. Also, I’m looking for ways to cheat as much as possible to still get great effects and quality on mobile.

The DSP class is a last minute addition. I was already doing DSP but didn’t realize it. In my last endeavor I was sampling and extracting information from digital sound waves to synchronize with my visuals. After doing this I realized that I may have made it work but I’m extremely limited with last approach (using averages and approximations). I stumbled upon the DSP course on coursera and thus far I’m happy with what it is teaching me but can’t be sure of its value just yet.

Here are some screenshots of my sound responsive app playing on the actual Gear VR. It only responds to bass hits, explanation of that process is in the last post. Also I don’t know of a good way to collect video from the phone yet. This shows much better with sound and motion, since that was the whole point (synchronizing sound with visuals). But here are screenshots anyway. The points pulse to the beat while doing a general sine wave. Kind of looks like a point sprite blanket:

In addition I am still pursuing general programming tasks. Specifically I’ve been looking for a platform to quickly and easily experiment. I checked out Processing.org and think it is ok. I think it is a little too verbose and not quite as powerful or as stable as I was hoping. So I found that VertexShaderArt.com is much better. I have started experimenting and was able to create a super simple shader just to show sound waves from SoundCloud… for fun I will list the code right here:

void main() 
{
  float vertex_per = vertexId / (vertexCount-1.0); 
  float x = vertex_per*2.0 - 1.0; 
  float y = texture2D(sound,vec2(vertex_per,0.0)).a; 
  gl_PointSize = 10.0; 
  gl_Position = vec4(x,y,0,1); 
  v_color = vec4(1.0); 
}

And pretty pictures… well not that pretty but yeah:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s