Category Archives: Driveby Gangster

C++11′s take on retain cycles

I’d like to take a brief moment to discuss something rather programmy today.  Namely, the topic of retain cycles.  If you’ve ever programmed in Objective-C before (since the addition of ARC) or Swift, this concept will be very familiar to you.  But if you’re coming from other programming languages such as the older (pre-11) C++, This concept will be new and you might want to read on.

Now before I get started, let me say that this example may not be the most correct way to handle delegation via lambda (block) functions in C++11.  Many will tell you that shared ownership of objects in C++ can get you into trouble, and this is certainly a case of how that can happen.  However it’s the most conceptually compatible with other languages, which I value over “c++ correctness” due to the fact that I’m porting code from another language to start with, and shared ownership allows me to get work done without reasoning around the lifespan of my objects.

Now as many newcomers to C++11 know, concepts such as smart pointers and lambda callback delegation can really save you alot of time when developing new systems.  However, there’s ahidden evil in using these two concepts together, that must be uncovered.

Example 1 :: All is well

Consider this example:

and its output:

What we’ve essentially done here is created an object (via shared pointer), set a callback lambda, and issued a method call which at some point calls our callback.  We see that everything is working as desired.  Most importantly, we see that our object is properly deleted when we are finished our program (when the shared pointer goes out of scope).

Example 2 :: Not so much

Now lets look what happens when I add a slightly more complicated, second case.

Obj2 essentially is the same as obj1 right?  All we do is add a little printout of obj2′s someProperty value from within the lambda… and of course, C++11 dictates that we must explicitly declare all external objects that we access within our lambda (a capture list).

So what is the output then?

Note the alarming lack of “object freed” being displayed for obj2.  That’s right, the above code memory-leaks obj2.  You see, obj2 owns someCallback, and someCallback owns obj2 via a capture. This is a retain cycle and it permanently causes obj2 to be “forever retained” and therefore leaks.  Once a retain cycle is made, you can consider things to be more-or-less, too late.

This type of bug is a dangerous silent killer of the night.  Why?  Because it can be so easy to miss when developing.  Objective-C and swift have evolved over time to add compiler and IDE warnings about this kind of “retain cycle” problem.  C++11 assumes that you just wouldn’t write this type of code.  But when porting code over from Objective-C, this kind of code can pop up ALL the time.

So what do we do?

Solution 1 :: The ObjC way

Well Objective-C handles this problem by introducing the concept of “weak pointers”.  These are special pointers that ensure that they do not own their reference, they only keep track of whether or not it is allocated (if any shared pointers to it exist).  Luckily, C++11 brings the weak pointer to us as well.  A solution that uses them looks like so:

You’ll note now that running this code will solve the leak.  It’s also important to note that, while this is an ugly solution, this is/was the only way to solve a retain cycle of this nature in Objective-C.

However, for this explicitly unique case, in C++, there’s another way.

Solution 2 :: The tempting C++ way

What if I told you there was a way to solve the original problem by adding one character to the original code?  Here it is:

See that little ‘&’ added before obj2 in the capture?  This allows us to capture a reference to the smart pointer and not a value.  Why does this matter?  Well when a smart_pointer value is copied, it essentially increments its retain count of the object in question, and when it finally is destroyed by going out of scope, it’ll decrement the retain count, restoring the natural balance to the universe.  However, as I mentioned before, a retain cycle prevents the retain count from ever reaching zero in this case.  A reference to obj2′s smart pointer value allows us to forego the copy-increment mechanism and essentially stop obj2 from being over-retained.  Its essentially like taking a pointer to a pointer….

However, this solution can be dangerous, as it assumes the coder understands the rammifcations of doing things like capturing a reference to a stack variable, an argument, or anything that may be “dead” by the time the callback is ran.  Almost always, this “solution” may give unintended consequences.

The “correct” way

Here’s where things get somewhat opinionated. The “correct” way to handle this really depends on the situation at hand. I suggest that if you a working with a team, you might want to use the safer and more verbose weak pointer method since it is more explicit in its intentions and helps explain why strong ownership could be a problem.  Furthermore, if you are working in a smaller system where shared pointership is not necessary, you might want to avoid using shared pointers altogether (assuming you fully understand the ownership semantics of the objects being used in the captures).

Either way, the golden rule is to understand the side effects of the code you are writing and when using shared pointers, keep that little retain cycle demon in the back of your mind… always

Free the code

One of the reasons why love game programming so much is that it forces you to do things that would otherwise seem quite ridiculous. I have always liked to shoot first and ask questions later when it comes to implementing an idea. Sometimes it’ll just pop in my head and then I will start working on it and before I know it…it’s done. The last week and a half has been more work than I ever wanted to do but the result is awesome. A very large portion of my entire Verto Studio graphics engine and code base has now been rewritten in C++ and is now portable to any operating system.

This means I can now load and render my Verto Studio files on Windows, Linux, anything I want without any strict dependencies on Mac anymore.  As a game developer, this is an awesome feeling and I am beyond stoked.  I worked very hard on the Verto Studio tech over the last three years and to finally free it from OSX/iOS is great.

I’ve moved on the actual swift code of the in-progress Driveby Gangster Game project and hopefully I’ll have that done soon next week.  All-in-all I’d say the extra week of work was definitely worth it.

Screens!!

Too much talk about code lately on here.

Here’s some latest screenshots.  Before my decision to switch languages, I completely finished off the “practice mode”

I also recorded a gif during my testing with the bullet collision detection against arbitrary polygons in the scene.

Swift is fired

So it had to happen…

The more I work on this game the more I realize that I kind of like it… and that means there’s certain things need to change.  For starters, it’s definitely running longer than I expected it to (the original two-week estimate now sounds pretty “out there”).  For this reason, I need to shave down on the programming time. It has become pretty obvious to me that for every 1/2 hour that I spend writing code in swift for this game, 10 minutes of it is spent screwing around with the language and that is friction then I can no longer accept.  So I’m dropping swift for this game.

Since I’m switching languages and I’m going to be rewriting some code, I decided to open things up quite a bit and go back to C++.  It’s going to cost me roughly a week or two to rewrite things, but the benefit is going to be huge.  For starters, since this game depended on my very large and very complicated Objective-C graphics engine (Verto Studio), I now have the “opportunity” to continue rewriting all of this code in C++ (this was originally started on my github as VGLPP).  This means that this game and any other game that I want to use with the ported engine will be able to run on ANY platform including windows.  This is a major win for me since making a game for OSX/iOS only isn’t really that smart.  I’ve been wanting to do this for awhile and my frustrations with swift finally pushed me to do it.

So that’s what I’m going to be up to for awhile.  As I’ve mentioned on twitter, I’ll be streaming roughly twice a week on my twitch channel for anyone who is curiuos about the process and wants to follow along.  http://twitch.tv/vertostudio3d

I’ve already gotten quite a bit done and have surpassed my biggest barrier for doing this which was finding a reasonable way to parse verto studio files (spat out by NSKeyedArchiver/NSCoder objective-c mechanisms) in a portable way in C++.  TinyXML and a little of reverse engineering did the trick.

With respect to all of the “benefits” swift and Objective C were providing over a lower level language such as C++, I’ve essentially found a reasonable replacement for each one thanks to C++11′s new features.

Basic Outline of Porting to C++ 

Swift/Objc C++11
Strong Pointer (Reference Counting) std::shared_ptr
Weak Pointer (Reference Counting) std::weak_ptr
Blocks & Closures Lambda Functions
Protocols Multiple Inheritence
Dynamic Typechecking via “isKindOfClass” C++ typeid / RTTI

So you get the idea, nothings really out of reach with C++. It’s still more annoying and I’m not 100% thrilled with dealing with things like templates, header files and verbose syntax again, but I’ll take it over an unstable language any day.

Lots of code… lots

I’ve been less active on here because I’ve been writing quite a bit of code.  Like I said previously, I’m in the main swing of developing this game.  This is where things start to get crazy.  Namely, I’ve added quite a bit of classes to the project to handle everything from basic collisions and 3D math extensions to generating and displaying 3D text on the screen.  I feel like (despite my previous post regarding Swift) things are keeping organized quite well and I haven’t strayed too far from my original architecture plan.  I’ve spent most of my time working on the PracticeGameStateRunner class which runs the “Target Practice” initial level in the game.  This mini level serves as a point for the player to learn the very simple controls and game mechanics of the game… and it’s serving excellently as a sandbox for me to test these all during development as well.  

I’ll cover just a few pieces of code today to show the changes that I’ve made regarding the game State Runner protocol, and some cool stuff that I’ve been able to do with “smart” enums in swift.  There’s a lot more I can talk about that I don’t want this post to go on forever.

Some Code

State runner protocol.  Now I have some stuff in there to quickly respond to “game controller” and mouse events, all stemming from the game loop class.

Mesh Line Collider – a badly needed construct to determine whether or not a bullet-trajectory would intersect a polygonal mesh in the scene or not (and if so where).  I tried porting this over to ObjC before swift and it was a nitemare.  Swift’s version is definitely simpler thanks to operator overloading and multiple return values.  Note the unsafe pointer craziness in swift which is considerably easier to deal with in C.  (ported from http://geomalgorithms.com/a06-_intersect-2.html)

Cool little WIP controller button enum nested in the game loop class

 

Some Screens

Its midnight..

Recently worked alot of basic walking pattern AI for the bystander game objects, a practice range which will precede the first “day” (level) in the game, and some cool shader effects using particles for the gangster car kicking up dust while driving through the desert.

Too tired to describe more.

Pictures!  Please forgive the quality of the video(s).

AI Walking Video

Quick update

Yesterday was a lot of work.  I worked really hard to obtain a higher frame rate for the game, because I know that my postprocessing effects will be costly.  Sometime certain obvious things take a few days to sink in. It’s funny how I was looking everywhere to try to improve my fill-rate and bandwidth performance when I knew the problem was a polyon issue.  Drawing the entire scene twice (once for lightmap/shadow pass and again for actual rendering) was really stupid.  I was doing this because certain elements in the shadow maps such as the walking men were dynamic, thus the scene needed to be redrawn into the shadowmap every time.  However, the mass majority of the polygons in the scene where static.  The solution of this was to pre-render this static geometry into one shadow map, and render the dynamic geometry into a separate one, and then adapt all of my shaders to take an optional second shadow texture.  This wasn’t easy since much of my architecture wasn’t designed for multiple shadowmaps per light.  But I got it working.

The above boosted my framerates back into a very acceptable range so I’m happy.  It’s a little annoying, that for some reason on OSX Yosemite, I can no longer turn off Vsync in a windowed mode.  So I’ve been having to go fullscreen during debugging to see my max framerates which are hovering now well-over 100.

As far as swift goes, I’m now on the fully released Xcode 6.1 and I’m having far less problems with the source kit crashing which is great.  The IDE is generally still a lot slower than its predecessor which I hope improves with time. as I’m writing more and more code in Swift, certain features are starting to really prove useful to me as I get used to the language.  Among these are the tuples, willSet/didSet, and operator overloading.  On the ObjC side, I’ve had to rely on this bulky MathVector class to do my vector maths.  With swift I’ve been able to add all of the math operators I need directly to my low-level C structs via an extension.  This has already saved me alot of work and made the math code that I’ve written for the game alot cleaner.

Here’s an example of the vector math extension code in swift

 

Game Development in Swift

First Impressions

So let me start off by saying, Swift is FAR from finished.  I know Apple is calling it “released”, but that’s a load of garbage.  My two main annoyances with swift from the previous post reign true, alongside a new victor:  the infamous can-go-to-hell sourcekit crash.  This problem occurs so frequently on the latest Xcode to GM candidate, that I came very very near abandoning Swift as the language for this game entirely yesterday.  If it wasn’t for a post that I found on the Apple developer forums related to deleting the “ModuleCache” dir within XCode’s DerivedData, I would have probably had to abandon swift which is really sad considering how much effort apple has put into it already.  Let me just say that a craftsman is only as good as his tools, and when it comes to XCode 6 alongside Swift, at this point I’d be very very cautious using it for anything that you absolutely must count on to be finished in a reasonable timeline.  This goes without saying that apple seriously needs to re-evaluate what they consider to be a GM release.

Either way… Because my project is an experiment and has no real pressure of any release deadline (apart from the fictional one I’ve set for myself), I will continue using Swift as the language.  After using it to put together the stuff that I’m going to show today, I must say that I definitely like coding in Swift compared to Objective-C for class design.  It’s a much less uglier language for the most part.  Some things still tick me off such as continuos need for explicit casting.  But overall, I can see its appeal as a language choice over Objective-C.  I say over ObjC because if you are stuck using Objective-C, you’re already stuck with an OSX/iOS-only application.  So you might as well make things easier on yourself and use Swift instead where possible (once it matures).

Lastly, I’m not setting out to use every single feature swift has to offer.  I likely won’t touch a lot of them.  The whole point here is to get the game done and to use the language as exercise to see what it offers to me to put together A 3D game in the OOP-style that I’m used to using.  My background is in C++, so I’ll likely find certain features more useful such as nested type capabilities.

The Game Architecture

So writing about programming is always more complicated than writing about 3D modeling or artwork.  At the beginning of this project, I stated that I would care more about the game than the programming itself.  That has already proven difficult for me, since my years of experience programming always leads me to architect a good solution for the task at hand.  One of the things I can’t stand in a game (or program in general) is over-architecture.  So what I present below hopefully won’t change that much moving forward.  Which leads me to say that I really want to focus on the game now.  Now that I’m in the main stretch of developing this thing, I need to put all efforts on the game development itself.  This means I might be ramping down temporarily the number of posts on this blog until the game begins nearing completion.  I hope to be at that phase within the next two weeks.

Graphics Engine

As I’ve mentioned in the past, the game will take the form of a series of swift classes that are going to be built on top of my existing Objective-C codebase.  This is the same codebase that powers my Verto Studio 3D modeling tool that I’ve used to put assemble the 3D assets.  Because Swift is designed to interoperate with ObjC pretty easily, this shouldn’t be a problem.  A closely-related C++11 mirror of the lowest layer of this system is available open-sourced at https://github.com/mlfarrell/VGLPP.

I loosely refer to all of this code as a “graphics engine”.  It contains routines for working with OpenGL 3.2 to draw 3D graphics primitives (VGL layer), and load/render 3D scenes and models (VOM / Object-Model) layer.  In the Objective-C code, these layers blur a bit but you get the idea.  The important classes that I interact with in the game (in the swift code) are as follows.  Because this system is quite large, this list is not exhaustive.

Objective-C Classes (Graphics Engine)

VGL Layer

  • VertoGLUserShaderEffect (subclassable to provide shader effects)
  • VertoGLStateMachine (GL 1.1-like state and state delegation)
  • VertoGLSystem (delegation and control for the behavior of the entire VGL system)
  • VecTypes (C structs for low-level OpenGL-needed things like float3, float4, float16, etc).
  • MatTypes (C structs for low-level OpenGL-needed math things like mat3, mat4, etc).

VOM Layer

  • SceneManager (3D Scene)
  • Entity (3D Model type)
  • EntityMaterial (Material properties)
  • Texture2D (Texture map)
  • RenderPassManager (Multipass rendering manager)

The Game Classes

After one day, here’s the basic architecture I have for the game.  The idea here follows the same concept I’ve used for many years now:  a game loop class with three methods that manage input (processEvents), processing(processLogic/update) and output (rendering).  At any given time, the game loop will be in one of many game states.  These states dictate how the game loop behaves.  Currently the planned game states are Splash (loading up), Menu (the title menu), Driveby (the actual gameplay), and Game over.  The game states are managed and executed from the game loop using a new GameStateRunner protocol type.  The game state holds on to a single object conforming to the GameStateRunner protocol and forwards the processEvents, processLogic and render methods to it accordingly.  This allows me to nicely separate and organize my code into special GameStateRunner conforming classes.  The interesting thing here is that swift actually allows me to embed some logic into the game state enum itself which I’ve already found useful to allow it to load and construct the GameStateRunner objects for me.  Confused yet?  Awesome!

Objective-C Classes (Driveby Gangster Game)

  • GameLoop (base class that interacts with SDL C-API and sets up OpenGL, this stuff is just simpler to do in C (which Objective-C is fully backwards compatible with compared to swift)
  • SkeletalAnimationShaderEffect : VertoGLUserShaderEffect

Swift Classes (Driveby Gangster Game)

Subclassing or Protocol conformance denoted with colon notation

  • GangsterGameLoop : GameLoop 
  • DrivebyGameStateRunner : GameStateRunner
  • TitleMenuStateRunner : GameStateRunner
  • SplashStateRunner : GameStateRunner
  • GameOverStateRunner : GameStateRunner
  • GameObject (base class game object for all visual objects in the game)
  • GameObjectSkeletalAnimated : GameObject (any game object that has a skeletal animation)
  • GameObjectPlayer : GameObjectSkeletalAnimated (the game player / protagonist)
  • GameObjectEnemy : GameObjectSkeletalAnimated (enemies in the game)
  • GameObjectBystander : GameObjectSkeletalAnimated (bystanders in the game)
  • GangsterSceneManager : SceneManager (game-oriented scene manager subclass)

Now the good news is, above should be just about it.  All of the shadow stuff, shaders, texturing, rendering of the game scene is all saved in the Verto Studio file and should load and render just like I had it in the editor when I load it into the game via SceneManager’s load methods.  The skeletal animation support (apart from the special shader effect subclass) is also all entirely provided by my graphics engine so I shouldn’t have to tinker too much with that stuff besides optimization.

Now this post has already gotten crazy long and it would be too much to include the source code of all of the above classes in this post. So instead I’m going to provide snippets of some of the base classes so you can understand how the system will work.

The Game Loop subclass in Swift

Using the ability to embed a “load” method directly in the game state enum is probably the most “swift-esque” thing I’ve done sofar.  Moving onto the GameStateRunner protocol and sample implementation.

 

Lastly, a sample lineage of the early GameObject class implementation.

 

 

That was alot.  The end.