So I got the ambient occlusion effect implemented in the actual game shader code.  I followed the technique outlined in this article.  I must say, it’s very subtle, but it looks nice.  The real benefit here is that when I turn off the “check depth range” mechanism, I get a very awesome surreal dark outline around all of the models in the scene.  This will be an excellent effect to use when I work on the second half of the game (which will use an artistic black and white noire post processing style signifying the protagonists slowly weakening grip on reality).

I didn’t understand how ambient occlusion worked at all before implementing the effect, but now I must say, I have a pretty reasonable grasp on it.  Basically in one brief description, it takes random samples at every point on the surface to determine how much ambient light from the rest of the scene can reach that point.  In other words, how much of the ambient light is occluded by other nearby surfaces.  This is done by randomly sampling points within a small hemisphere emanating from the surface.  What makes SSAO so nice, is that this technique is approximated entirely in screen space, making it reasonable for real-time rendering in a game.  The downside is it seems to be a little noisy and I see artifacts from time to time on the screen, but hopefully I’ll be able to clean some of that up in time.

Below are some more progress shots showing the kinds of subtle shadowing I can achieve using SSAO in the VGLPP/DrivebyGangster engine.

Today was huge!  I am exhausted.  I busted my ass today to get an affect known as screen-space ambient occlusion (SSAO) working in a prototype scene in verto studio.  Right when I was about to give up, I got it working!  I’m too groggy to explain anything about it, so I’ll just post pictures!

This is going to add a huge boost to the visual look-and-feel of the game without a severe performance penalty.  Needless to say, I’m pretty stoked!

 

 

So asset creation in the form of modeling the final environment that’ll be used in the game took the past few days.  It’s amazed how much faster it went this time compared the first scene (the city street).  I chalk this up to experience that I’ve picked up now that I’ve been doing this awhile, and my understanding of things that are very time consuming to model vs things I can quickly grab, import, texture, and place from turbosquid.  The perfect balance of both techniques has led to me being done with the dark night alley street scene after just a few days.

Where I spent the most time this time was in the shaders.  I really wanted to have quality rendering and go out with a bang since this’ll likely be the last environment that I do for the game.  I want the dark alleyway to appear foggy and dark, like its just about to rain.  That includes moist shiny cobblestone rocks, and a damp fog effect that feels very palpable to the player.

This idea of mine posed two major problems that I haven’t had to tackle until now.

Point Light Shadows

The first major problem, is that I could no longer “cheat” using a single directional light source representing the “sun” in the scene.  Instead, I had to place a street lamp every building or so and have these be the only sources of light in the scene.  This means, in order to keep my shadows working, I now had to implement shadows from a point light source.  In some ways this was easier than point lighting (no longer necessary to compute an optimal shadow bounding box to be used for the light’s ortho matrix projection).  However, now I needed render to and utilize cube maps to store the shadow depth information.  Surprisingly, there was very little comprehensive information on the web about how to properly do PCF shadows using cubemaps.

What I found that works is the following.

• Create a cubemap renderer that is positioned at the exact same position as one of the point lights - this special vgl engine object renders the scene into 6 faces of a cubemap with 90-degree fov angles to properly capture the entire scene from “all angles”.
• Format the cubemap shader to have each face hold floating point depth-component values and keep the size small (256×256) since there will be alot of these.
• Define an “override” shader to be used by the above cubemap renderer to ensure a simple specialized “light render” shader was used when rendering the scene into the cubemap faces.  
• Set the cubemap compare mode (GL_TEXTURE_COMPARE_MODE) for the rendered cubemap texture to GL_COMPARE_REF_TO_TEXTURE which helps with percentage-closest-filtering (PCF) mode to allow smoother linear-interpolated shadow edges.  
• Lastly, pass a rendered shadow cubemap for each light to the scene shaders in the form of samplerCubeShadow

The “light render” shader mentioned above which is used when rendering the depth information to the cubemaps looks like so:

Basically this encodes the distance from the surface point to the light in the form normalized depth information.  This is done by manually overriding the depth value stored in gl_FragDepth.  Each light has its own cubemap renderer with a custom-tailored shader like this (using the correct light index to compute distance from).

The per-fragment lighting shader code that utilizes the cube shadow maps looks like:

Essentially the highlighted code above compares the distance between the surface point to the light to the stored depth value in the cube texture using the fragment-light vector as a lookup value into the cubemap.  Because we’re sampling using a special “shadow” version of the cubemap sampler, the result will be properly interpolated between shadow texels to avoid ugly edges between shadowed and non-shadowed areas.

Luckily, I was able to build this into the Verto Studio editor and associated graphics system and test this out with relatively little trouble.  Even though I have 4 or 5 statically rendered cubemap shadow textures in the entire scene.  I was able to keep performance high by building versions of the shaders for each side of the street so that each individual shader only has to shade with at most 3 lights at a time.  This worked out better than I had expected.

Light-Fog Interaction (Volume FX)

This part was tricky..  I had an idea in my head of how I wanted this to look.  So I did something that usually leads to trouble;  I tried to come up with a volume rendering technique on my own from scratch and implement it and just kinda see how it goes.

The basic idea stems from what I’ve observed in real life from foggy, dark, damp nights and lighting.  Essentially, fog at night is black… IF there is no light around to interact with the fog.  Naturally, if the water droplets don’t have any light to interact with them, you won’t see them and the fog will appear black.  However, if any light interacts with the water vapor in the air, it’ll create the illusion of a whiter colored and denser fog.  So this is what I set out to emulate with my shader.

Now atmospheric shader effects can often lead to the necessity of raymarching and heavy iteration in the fragment shader to simulate the accumulation of light-atomosphere interaction.  To this I said “hell no” since ray marching of any kind in a shader terribly robs performance.  I quickly realized that I could avoid raymarching entirely if I used a simple model to represent the light-fog interaction that I was going for.

In my case, it turned out I could do the whole effect using something as simple as a sphere intersection test.  Basically, when I’m shading a pixel (a point on the surface), I’m interested in what happens to the light on its way back from the surface to the viewer, the surface-to-viewer vector.  If the atmosphere affects the light at any point along this vector, I’ll need to compute that.  In other words, if the ray from the surface to the viewer intersects a sphere centered at the light, then the fog affects the light on the way back to the viewer.  How much?  Well if I calculate the length of the segment between the entry and exit points of the ray intersection (how much of the sphere the ray pierces), I find that length is proportional to both the perceived increase in density of the fog and the brightening of the fog color.

This algorithm is given below in fragment shader code:

 

I’m sure I’m not the first one to come up with this idea before, but it still did feel pretty cool to reason my way through a shading problem like this.  The visual result looks amazing.

The progress of this all is below in a gallery.  

Intro

So this morning is the first morning that I am working without a day job.  I must say its liberating…. but enough about that crap!  It’s time to get to work!

This morning was a programming morning.  The considerable performance drop of my scene during my modeling efforts led me to investigate methods for improving my scene’s rendering performance, both in the editor and in the game itself (since they both use the exact same rendering engine).  Back in the day, I used to laboriously accomplish this using frustum culling.  Frustum culling is a technique that uses a spatial data structure, an octree, to categorize the mesh objects in a scene into cubic regions, and then mathematically detect whether or not those regions are in the viewing volume (frustum) currently visible in the scene.  This technique works okay, but it’s a pain in the ass to implement and I’d rather not if I don’t absolutely need to.  Furthermore, it doesn’t handle the situation of occlusion, when a very large 3D object is in front of a smaller one, eclipsing it, making it entirely invisible and useless to render since it’ll fail the z-buffer test.

Thus, enters “occlusion queries”.  A very cool OpenGL technique which allows you query exactly how much of a 3D object was truly rendered, and decide whether or not to keep rendering it in the future.  This is exactly what I needed.  It all sounds great in theory, now let me tell you about some of the issues I had implementing it.  I’ll try to avoid some of the ugly Objective-C syntax that surrounds this code in my actual system in my snippets.

Technique Overview

So in practice, occlusion culling is quite simple.  There are basically 3 steps.

First, you must render the scene using very simple solid bounding-box geometry.  ie, for each discrete mesh object within your scene, you render a giant solid box that entirely bounds that object. You only render this box with a very simple flat-color shader which will keep your query render very fast.  You don’t actually want these boxes to appear in your scene, so you do this step with color buffer writes and depth buffer writes turned off (masks set to FALSE).

Next, you query the results of the above for each box rendered and determine which models were visible (not occluded).  You make a note of the ones that were.

Finally, you render the scene normally, with the extra check to ensure that you don’t render the model objects that were not visible.

Setup

So the first thing I needed was a single occlusion query per mesh object in my scene.  In OpenGL, these (like many things in GL) are GLuint ids.  I dropped these into my entity mesh class

Then, in the model init and dealloc code, I generate the query objects as needed.

I then set up a special method that renders the solid bounding box geometry used during the occlusion query.  Now here’s where things get tricky.  There’s a way to do occlusion queries wrong (which I found out the hard way).  So much so that the performance benefit that they offer is entirely negated by the pipeline stalling that you can inadvertently cause.  Note the check against the EntityMeshOcclusionWaiting state.  This will be explained in the next section.

 

Scene Rendering

To kick this off, I added a new special method to my Scene class called renderOcclusionQueries.  I then inserted a call to this method in my scene’s main render method like so.  Note the usage of glColorMask and glDepthMask to ensure the query bounding boxes don’t actually render to the screen.

With the queries properly set up, I can now use them during my main rendering pass of all the entities to ensure I only draw whats necessary.  Again, this was tricky.  I had to absolutely make sure I never stall the pipeline.  No matter what.  This means I don’t retrieve a query result unless GL_QUERY_RESULT_AVAILABLE is true.  If it isn’t, I leave the query in the “waiting” state.  I also don’t start a new query when it’s in the waiting state (note the check against this in the above entity renderOcclusionQuery method).  This essentially means that the occlusion queries are entirely asynchronous with respect to the main rendering.  

That’s pretty much all there is to it.

Considerations

Now aint nothin in this world fo free.  So there’s some things I should mention.  First, if I wasn’t also targeting iOS mobile, I would have probably used the OpenGL conditional rendering method which essentially does alot of the above checking for me automatically.  I noodled around with this and couldn’t get equivalently good performance so I just moved onto the manual way.  I also don’t like how I still have to submit all the expensive drawing “and non” drawing calls with conditional rendering and essentially trust the driver to do whats best.  My method ensures NOTHING is ran if the object isn’t visible.  With the downside being initiating readbacks from the OpenGL device back to the CPU.  However, I’m getting very decent performance with this so I’m happy.

Also, because the queries are truly async, I can get myself into trouble when running this code on very slow or buggy graphics cards (ahem.. intel..ahem).  The problem being, if the query takes too long, you may look at a space where an object should be, and not see it for a few frames while waiting for the query to catch up.  This finally explains to me why when playing some games on my wii u (such as call of duty), I sometimes turn real fast, and see an object suddenly appear a few frames late.