Category Archives: Programming

Quick update

Yesterday was a lot of work.  I worked really hard to obtain a higher frame rate for the game, because I know that my postprocessing effects will be costly.  Sometime certain obvious things take a few days to sink in. It’s funny how I was looking everywhere to try to improve my fill-rate and bandwidth performance when I knew the problem was a polyon issue.  Drawing the entire scene twice (once for lightmap/shadow pass and again for actual rendering) was really stupid.  I was doing this because certain elements in the shadow maps such as the walking men were dynamic, thus the scene needed to be redrawn into the shadowmap every time.  However, the mass majority of the polygons in the scene where static.  The solution of this was to pre-render this static geometry into one shadow map, and render the dynamic geometry into a separate one, and then adapt all of my shaders to take an optional second shadow texture.  This wasn’t easy since much of my architecture wasn’t designed for multiple shadowmaps per light.  But I got it working.

The above boosted my framerates back into a very acceptable range so I’m happy.  It’s a little annoying, that for some reason on OSX Yosemite, I can no longer turn off Vsync in a windowed mode.  So I’ve been having to go fullscreen during debugging to see my max framerates which are hovering now well-over 100.

As far as swift goes, I’m now on the fully released Xcode 6.1 and I’m having far less problems with the source kit crashing which is great.  The IDE is generally still a lot slower than its predecessor which I hope improves with time. as I’m writing more and more code in Swift, certain features are starting to really prove useful to me as I get used to the language.  Among these are the tuples, willSet/didSet, and operator overloading.  On the ObjC side, I’ve had to rely on this bulky MathVector class to do my vector maths.  With swift I’ve been able to add all of the math operators I need directly to my low-level C structs via an extension.  This has already saved me alot of work and made the math code that I’ve written for the game alot cleaner.

Here’s an example of the vector math extension code in swift

 

Game Development in Swift

First Impressions

So let me start off by saying, Swift is FAR from finished.  I know Apple is calling it “released”, but that’s a load of garbage.  My two main annoyances with swift from the previous post reign true, alongside a new victor:  the infamous can-go-to-hell sourcekit crash.  This problem occurs so frequently on the latest Xcode to GM candidate, that I came very very near abandoning Swift as the language for this game entirely yesterday.  If it wasn’t for a post that I found on the Apple developer forums related to deleting the “ModuleCache” dir within XCode’s DerivedData, I would have probably had to abandon swift which is really sad considering how much effort apple has put into it already.  Let me just say that a craftsman is only as good as his tools, and when it comes to XCode 6 alongside Swift, at this point I’d be very very cautious using it for anything that you absolutely must count on to be finished in a reasonable timeline.  This goes without saying that apple seriously needs to re-evaluate what they consider to be a GM release.

Either way… Because my project is an experiment and has no real pressure of any release deadline (apart from the fictional one I’ve set for myself), I will continue using Swift as the language.  After using it to put together the stuff that I’m going to show today, I must say that I definitely like coding in Swift compared to Objective-C for class design.  It’s a much less uglier language for the most part.  Some things still tick me off such as continuos need for explicit casting.  But overall, I can see its appeal as a language choice over Objective-C.  I say over ObjC because if you are stuck using Objective-C, you’re already stuck with an OSX/iOS-only application.  So you might as well make things easier on yourself and use Swift instead where possible (once it matures).

Lastly, I’m not setting out to use every single feature swift has to offer.  I likely won’t touch a lot of them.  The whole point here is to get the game done and to use the language as exercise to see what it offers to me to put together A 3D game in the OOP-style that I’m used to using.  My background is in C++, so I’ll likely find certain features more useful such as nested type capabilities.

The Game Architecture

So writing about programming is always more complicated than writing about 3D modeling or artwork.  At the beginning of this project, I stated that I would care more about the game than the programming itself.  That has already proven difficult for me, since my years of experience programming always leads me to architect a good solution for the task at hand.  One of the things I can’t stand in a game (or program in general) is over-architecture.  So what I present below hopefully won’t change that much moving forward.  Which leads me to say that I really want to focus on the game now.  Now that I’m in the main stretch of developing this thing, I need to put all efforts on the game development itself.  This means I might be ramping down temporarily the number of posts on this blog until the game begins nearing completion.  I hope to be at that phase within the next two weeks.

Graphics Engine

As I’ve mentioned in the past, the game will take the form of a series of swift classes that are going to be built on top of my existing Objective-C codebase.  This is the same codebase that powers my Verto Studio 3D modeling tool that I’ve used to put assemble the 3D assets.  Because Swift is designed to interoperate with ObjC pretty easily, this shouldn’t be a problem.  A closely-related C++11 mirror of the lowest layer of this system is available open-sourced at https://github.com/mlfarrell/VGLPP.

I loosely refer to all of this code as a “graphics engine”.  It contains routines for working with OpenGL 3.2 to draw 3D graphics primitives (VGL layer), and load/render 3D scenes and models (VOM / Object-Model) layer.  In the Objective-C code, these layers blur a bit but you get the idea.  The important classes that I interact with in the game (in the swift code) are as follows.  Because this system is quite large, this list is not exhaustive.

Objective-C Classes (Graphics Engine)

VGL Layer

  • VertoGLUserShaderEffect (subclassable to provide shader effects)
  • VertoGLStateMachine (GL 1.1-like state and state delegation)
  • VertoGLSystem (delegation and control for the behavior of the entire VGL system)
  • VecTypes (C structs for low-level OpenGL-needed things like float3, float4, float16, etc).
  • MatTypes (C structs for low-level OpenGL-needed math things like mat3, mat4, etc).

VOM Layer

  • SceneManager (3D Scene)
  • Entity (3D Model type)
  • EntityMaterial (Material properties)
  • Texture2D (Texture map)
  • RenderPassManager (Multipass rendering manager)

The Game Classes

After one day, here’s the basic architecture I have for the game.  The idea here follows the same concept I’ve used for many years now:  a game loop class with three methods that manage input (processEvents), processing(processLogic/update) and output (rendering).  At any given time, the game loop will be in one of many game states.  These states dictate how the game loop behaves.  Currently the planned game states are Splash (loading up), Menu (the title menu), Driveby (the actual gameplay), and Game over.  The game states are managed and executed from the game loop using a new GameStateRunner protocol type.  The game state holds on to a single object conforming to the GameStateRunner protocol and forwards the processEvents, processLogic and render methods to it accordingly.  This allows me to nicely separate and organize my code into special GameStateRunner conforming classes.  The interesting thing here is that swift actually allows me to embed some logic into the game state enum itself which I’ve already found useful to allow it to load and construct the GameStateRunner objects for me.  Confused yet?  Awesome!

Objective-C Classes (Driveby Gangster Game)

  • GameLoop (base class that interacts with SDL C-API and sets up OpenGL, this stuff is just simpler to do in C (which Objective-C is fully backwards compatible with compared to swift)
  • SkeletalAnimationShaderEffect : VertoGLUserShaderEffect

Swift Classes (Driveby Gangster Game)

Subclassing or Protocol conformance denoted with colon notation

  • GangsterGameLoop : GameLoop 
  • DrivebyGameStateRunner : GameStateRunner
  • TitleMenuStateRunner : GameStateRunner
  • SplashStateRunner : GameStateRunner
  • GameOverStateRunner : GameStateRunner
  • GameObject (base class game object for all visual objects in the game)
  • GameObjectSkeletalAnimated : GameObject (any game object that has a skeletal animation)
  • GameObjectPlayer : GameObjectSkeletalAnimated (the game player / protagonist)
  • GameObjectEnemy : GameObjectSkeletalAnimated (enemies in the game)
  • GameObjectBystander : GameObjectSkeletalAnimated (bystanders in the game)
  • GangsterSceneManager : SceneManager (game-oriented scene manager subclass)

Now the good news is, above should be just about it.  All of the shadow stuff, shaders, texturing, rendering of the game scene is all saved in the Verto Studio file and should load and render just like I had it in the editor when I load it into the game via SceneManager’s load methods.  The skeletal animation support (apart from the special shader effect subclass) is also all entirely provided by my graphics engine so I shouldn’t have to tinker too much with that stuff besides optimization.

Now this post has already gotten crazy long and it would be too much to include the source code of all of the above classes in this post. So instead I’m going to provide snippets of some of the base classes so you can understand how the system will work.

The Game Loop subclass in Swift

Using the ability to embed a “load” method directly in the game state enum is probably the most “swift-esque” thing I’ve done sofar.  Moving onto the GameStateRunner protocol and sample implementation.

 

Lastly, a sample lineage of the early GameObject class implementation.

 

 

That was alot.  The end.

Why most indie games are 2D games.

Why Most Indie Games are 2D Games

Okay so do you want to know why most indie games are in 2D?  Read the posts from the last month or so in my blog.  In summary, you’ll notice that 90% of all that work was asset creation.  Keeping in mind, this is supposed to be a small-scale single-level 3D mini game.  Artwork, modeling, texturing, more artwork, more texturing, more modeling.. etc.  Simply stated, asset creation for 3D games is LONG, HARD work.. and in my case, I didn’t even do it all myself (I had a character modeler/animator help me with my single base character model, and I’ve relied quite a few times on turbosquid).  Even still, compared to 2D game development, this process can be extremely time consuming.  If this was a 2D platformer, most devs would have had at least basic assets for a “level 1″ finished probably in the first day, and would have been programming the actual game long before now.  I’ve always tried to ignore this fact but its true.  Look at the number of delays, setbacks, and ultimate release date shifts that have plagued recent AAA game projects project and you’ll see that rapid 3D game development is not an easy task even for seasoned developers.

Most independent game developers have certain skillsets that usually fall into the domains of programmer or artist.  Now I know there’s stortywriting, sound effects, music, etc.  But just about everything involved in a game’s development really splits into the two big umbrellas of either programming or creative (artwork, asset creation, etc).  The problem here is that as one advances in their particular domain, and starts to get really good at one aspect of the field, they often have to leave the other side of things to somewhat wither.  To become a really good programmer, I mean a really good programmer, you need to get obsessed with your craft.  That means even if you were a decent artist, you probably won’t keep up your artistry skillset as much.  The same goes for becoming a very good artist, and even a 3D artist.

As the industry has made the shift into large-scale theatric AAA games, more and more specialized craftsmen/craftswomen are needed to spend 8-hour days on something as simple as the texture of a claw of a single enemy in a single dungeon of a game.  The sheer manpower that AAA studios can throw at a game allows for the large attention to detail that 3D games require to feel right.  Even a “simple” game like mario 64 likely required dozens of artists to pan and pan over every keyframe of mario’s animations to make sure it was perfect.  Now don’t get me wrong, this was also true of the SNES/Genesis 2D era.  However, attention-to-detail, or lack thereof in these kind of games comes with less of a penalty, and certainly less of a time commitment.

It’s a hard pill to swallow since my dream of making games that started at a young age, has always predominantly been a dream of 3D game development.  But you know what?  I still love 3D game development.  It’s supposed to be hard.  The hard… is what makes it great.  Tom Hanks is right.  

So where does that leave the indie developer with respect to 3D game development?  Well when you have a team of only 2-3 people (or in my case, 1 people), you CAN succeed at independent 3D game development.  The absolutely crucial difference here is to understand and accept your limitations as an independent.  For starters, no matter who you are, limit your freaking scope.  Understand that you simply can’t make an AAA-length 3D game in any reasonable amount of time.  If you’re 1 person, and a hobbyist with almost zero budget (like me), set a goal for a game that requires at most 1-2 3D environments with very few (if any) unique character models.  If you are weak with art but you have the funds, consider using services like turbosquid to procure assets.  If you are weak with 3D programming, consider (after buying a few books) using a 3D engine such as unity or unreal instead of building your own.  

Above all, plan out the scope of your game to a t.  Then, give yourself a pre-determined amount of time to finish the game project… and..   quadruple it.

Driveby Gangster Update – a Dive Into Swift

Over one month in to my planned “2 week” 3D game project, 90% of my assets are done!  I still need a 3D tommy gun and an old-timey driveby gangster car.  But I’m going to start actually programming (imagine that) the very simple AI that will drive the walking of most of the bystanders in the game.  I’ve got my work cut out for me since most of the game logic code I’ve written to this point is all throwaway “get-it-to-work” POC code.  I now need to organize and get a basic heirarchical game-architecture together.  More fun, as I mentioned in my first post, I’ll be doing most of the game classes and logic in swift.  Swift, despite being officially released, still has its problems.  Namely for me, the speed of the IDE (for things like auto-completion) while typing in swift is the biggest gripe I have with the language so far.  That problem is followed by a close-second of annoying cryptic compiler errors that have nothing to do with the underlying problem of “I’m expecting an Int32, but you gave me an Int”.  Either way, I’m keeping my promise to write the game in swift as an experiment to see if the language is truly something I’ll want to leverage for game development in the future.

The ways of shadows

Ahh shadows.  I’ve been putting this off because lets face it, 3D shadow mapping is not frickin easy.  There are countless advanced algorithms for 3D shadow mapping to make shadows look as pretty as possible on our discrete-centric graphics hardware.  Some of them are crazy complicated and quite difficult to implement.  I’ve been messing around with 3D graphics programming for over 10 years now and let me say that shadows have always been just out of reach for me.  This week, I decided to put an end to that.

As a plan, I’ve decided to keep things as simple as possible.  I’m making a game here, not a game engine, so I wanted to get shadows working reasonably well, and get back to the game programming aspect of this project.

The basic outline of the simplest shadow mapping technique:

  • Render the scene from the perspective of the infinitely far direction light (shadow pass) into a shadow-depth texture.
  • Render the scene normally using the depth-information from the shadow-depth texture to determine whether or not a particular pixel is visible to the light or not (in shadow or not).

The Shadow Pass

Above, it sounds simple.  In practice, there are many caveats.  For starters, its absolutely critical to get the most of out of the “shadow-depth” texture in terms of resolution as possible.  Thus, when rendering the shadow pass, we want to contain the entire scene into the light’s view with the constraint that we show as much (are as zoomed in) as possible.  If we zoom in too little, we hurt the resolution of the shadow map.  If we zoom too much, we risk clipping the scene resulting in some shadows being lost.  Furthermore, we want to render this step with as simple of a shader as possible, to avoid unnecessary wasted computation on the GPU.

Going back to the optimal viewport containment (zooming) issue, this boils down to computing the optimal Ortho-box that the scene will be contained in.  We’ll use this box as the parameters to the ortho projection matrix given during the light/shadow rendering pass.  Optimally bounding the scene with this box presents a problem due to the fact that the box is in light-view-space coordinates, and all of our scene bounding boxes are in world-space.  Trying to work through this last night, I resorted to pencil and paper.

The algorithm essentially involves grabbing the light’s “viewing” transformation which consists of a simple lookAt transform and applying it to the 8 corner vertices of the world-space bounding box of the entire scene.  Once I have these coordinates in light-view-space, hopefully a computation of a new axis-aligned bounding box of these 8 points will be the ortho-box I’m looking for.  It turns out, that this worked quite well.

The actual code of this algorithm ended up looking more like this..

 

Below is a sample result of a shadow pass done using my cheap and simple bounding algorithm ran on our street scene (vantage of the light).  Note that this is stored into a depth-component texture attached to the depth-attachment of an offscreen FBO.

Goooood.

The Shadow-Application (main) Pass

During the main rendering pass, I needed to modify my shaders to include the application of the shadows from the light-map.  Alongside the light-map texture, I needed the a variant of the same “MVP” model-view-projection used to transform a world-space position into projected light-view-space coordinates.  This matrix is commonly referred to as a “bias shadow matrix” because its optimized to express the result in a normalized texture-coordinate form that GLSL texture routines are expecting.  In short, it simply applies the lighting-transform, divides the coordinates by 2 and then shifts them by 0.5.

Armed with the shadow matrix and the shadow map texture, I generate the needed shadow coordinate information in vertex shader.  I also compute a shadow bias to combat a well-known phenomenon known as “shadow acne” essentially caused by z-fighting from the shadowmap texture.   

Lastly, in the fragment shader, I sample the shadow texture to determine whether or not the shadow coordinate of the given fragment is visible or not to the light.  I can vary the visibility factor to be as dark or light as I want to achieve the desired effect.  Note that I’m using a shadow sampler here.  This special hardware sampler takes multiple samples of the shadow map for me and interpolates the results automatically to produce a smoother shadow edge.

The results of all of this craziness is something quite nice.  Shadows casted in my scene that can lie across curved surfaces.  This was quite a bit of work but I think it’ll be quite worth it since now my graphics engine is shadow-capable.  Down the road I’d like to add point-light shadow capability via rendering into shadow cubemaps and general shadow capability for Verto Studio, but for now, directional light shadows satisfies the needs of my game project.

More of texturing day

Yesterday I worked quite a bit with my character animator Tyler Hurdle to get the animations properly exported from his modeling software into my graphics engine.  After much wrestling, I got the simple walk cycle loaded in and it looks awesome.  

Things happened today.  Those things included me finishing up the texturing for the Hotel.  I must say, it’s really starting to look good. I can’t imagine how it’s going to look once I add in the final shadowing and post-processing effects.  I didn’t take as many intermediate screenshots as I should have this time. So I only really have the final results of where I am at the end of today.  Nothing too-far out of the ordinary happened during the last steps of the texturing of the hotel besides me modifying the basic window shader effects to include partial transparency. I did this so that I could “cheat” with the interior shops of the hotel, modeling the interior as a simple gaussian-blurred backdrop which is partially of obscured by the semitransparent window.

I also decided to get rid of the ugly default “sand” texture that I’ve been using for my background terrain.  I spiffied this up a bit with multitexturing effects using a detail texture which came out pretty awesome.

After all this, the frame rate performance within the editor and the game started to both get really bad.  So I had to stop modeling and dive into some optimization once again.  Using instruments within xcode, I uncovered some horrors related to the terrible performance of objective-c’s NSString methods (namely stringWithFormat) which forced me to eliminate their usage and some of the more critical sections of the rendering engine’s code.  That alone, gained me another ten frames per second back and started getting me to question the viability of Objective-C for hardcore game engine development.  I sure hope Swift’s string methods are faster than Objective-C’s.

Continuing with optimization, I put off the long-needed step of sorting scene entities first by transparency, and then by material.  This helped me avoid unnecessary state changes which propagate to the shader and harm performance.  I also hard-coded a backface culling test which showed that I really need a per-model “cull backfaces” option within the editor.  All of this optimization added up quite a bit to bring my performance back up to a reasonable level.

All of this work today uncovered quite a few new bugs in the editor itself, so tomorrow will likely be spent fixing those…

The insanity that is OpenGL Occlusion Queries

Intro

So this morning is the first morning that I am working without a day job.  I must say its liberating…. but enough about that crap!  It’s time to get to work!

This morning was a programming morning.  The considerable performance drop of my scene during my modeling efforts led me to investigate methods for improving my scene’s rendering performance, both in the editor and in the game itself (since they both use the exact same rendering engine).  Back in the day, I used to laboriously accomplish this using frustum culling.  Frustum culling is a technique that uses a spatial data structure, an octree, to categorize the mesh objects in a scene into cubic regions, and then mathematically detect whether or not those regions are in the viewing volume (frustum) currently visible in the scene.  This technique works okay, but it’s a pain in the ass to implement and I’d rather not if I don’t absolutely need to.  Furthermore, it doesn’t handle the situation of occlusion, when a very large 3D object is in front of a smaller one, eclipsing it, making it entirely invisible and useless to render since it’ll fail the z-buffer test.

Thus, enters “occlusion queries”.  A very cool OpenGL technique which allows you query exactly how much of a 3D object was truly rendered, and decide whether or not to keep rendering it in the future.  This is exactly what I needed.  It all sounds great in theory, now let me tell you about some of the issues I had implementing it.  I’ll try to avoid some of the ugly Objective-C syntax that surrounds this code in my actual system in my snippets.

Technique Overview

So in practice, occlusion culling is quite simple.  There are basically 3 steps.

First, you must render the scene using very simple solid bounding-box geometry.  ie, for each discrete mesh object within your scene, you render a giant solid box that entirely bounds that object. You only render this box with a very simple flat-color shader which will keep your query render very fast.  You don’t actually want these boxes to appear in your scene, so you do this step with color buffer writes and depth buffer writes turned off (masks set to FALSE).

Next, you query the results of the above for each box rendered and determine which models were visible (not occluded).  You make a note of the ones that were.

Finally, you render the scene normally, with the extra check to ensure that you don’t render the model objects that were not visible.

Setup

So the first thing I needed was a single occlusion query per mesh object in my scene.  In OpenGL, these (like many things in GL) are GLuint ids.  I dropped these into my entity mesh class

Then, in the model init and dealloc code, I generate the query objects as needed.

I then set up a special method that renders the solid bounding box geometry used during the occlusion query.  Now here’s where things get tricky.  There’s a way to do occlusion queries wrong (which I found out the hard way).  So much so that the performance benefit that they offer is entirely negated by the pipeline stalling that you can inadvertently cause.  Note the check against the EntityMeshOcclusionWaiting state.  This will be explained in the next section.

 

Scene Rendering

To kick this off, I added a new special method to my Scene class called renderOcclusionQueries.  I then inserted a call to this method in my scene’s main render method like so.  Note the usage of glColorMask and glDepthMask to ensure the query bounding boxes don’t actually render to the screen.

With the queries properly set up, I can now use them during my main rendering pass of all the entities to ensure I only draw whats necessary.  Again, this was tricky.  I had to absolutely make sure I never stall the pipeline.  No matter what.  This means I don’t retrieve a query result unless GL_QUERY_RESULT_AVAILABLE is true.  If it isn’t, I leave the query in the “waiting” state.  I also don’t start a new query when it’s in the waiting state (note the check against this in the above entity renderOcclusionQuery method).  This essentially means that the occlusion queries are entirely asynchronous with respect to the main rendering.  

That’s pretty much all there is to it.

Considerations

Now aint nothin in this world fo free.  So there’s some things I should mention.  First, if I wasn’t also targeting iOS mobile, I would have probably used the OpenGL conditional rendering method which essentially does alot of the above checking for me automatically.  I noodled around with this and couldn’t get equivalently good performance so I just moved onto the manual way.  I also don’t like how I still have to submit all the expensive drawing “and non” drawing calls with conditional rendering and essentially trust the driver to do whats best.  My method ensures NOTHING is ran if the object isn’t visible.  With the downside being initiating readbacks from the OpenGL device back to the CPU.  However, I’m getting very decent performance with this so I’m happy.

Also, because the queries are truly async, I can get myself into trouble when running this code on very slow or buggy graphics cards (ahem.. intel..ahem).  The problem being, if the query takes too long, you may look at a space where an object should be, and not see it for a few frames while waiting for the query to catch up.  This finally explains to me why when playing some games on my wii u (such as call of duty), I sometimes turn real fast, and see an object suddenly appear a few frames late.

Linux stuff

In keeping with the multi-platform spirit, finally did a test build on Linux tonight.  Apart from some major annoyances related to case sensitivity, building wasn’t too bad.  Ubuntu has made it really easy to install all necessary dependencies (clang, xorg-dev, SDL, freetype, etc).

                                                              Test run on linux

 

 

Words…

So, I bit the bullet the other day and implemented something that I knew I’d be needing soon. Annoyed that I didn’t have a text-rendering system in place to display things like the current FPS on the screen (and because I hate reading console output), I sought out a decent portable text rendering solution for OpenGL that I could easily port to Direct3D later down the road if needed.  Normally, on the mac platform, I’d just drop into Core Graphics and render the text string to a blank bitmap and roll that bitmap into a texture.  But since I’m trying to be portable, I can’t rely on anything specific to mac os, or windows, or anything!

One thing became readily apparent was that any solution I devise will use FreeType2, a very useful portable font library for rendering TrueType fonts.  One thing that I was not predicting however, was that there exists a multitude of libraries that run on top of FreeType2 (such FreeTypeGL) as whose sole purpose is to use freetype to render text on OpenGL.  This annoyed me.  I don’t want to drop in an entire heavy-weight font library complete with 30+ source files, with its own license, just to render a single static string to a texture.  Furthermore, these libraries become another barrier for portability in terms of OpenGL/Direct3D abstraction.  I’m ok with linking graphics-independent libs such as freetype, but entire subsystems relying on OpenGL, that’s where I draw the line.

So, I decided to do it myself. Freetype can’t be that hard anyway right?  So it really only takes about 100 lines of code to use freetype to render to a image buffer suitable for rolling into an OpenGL texture.  So here you have it, my experimental, non-optimized Text Texture class..

 

Not very efficient yet, but it works!

To save memory, I decided to store the text image in a single channel texture (GL_RED format).  So, I have to use a special swizzle in the GLSL code so the texture doesn’t render red.

And.. it works!

Finally.  I can show my FPS on the screen like a proper game.