Monthly Archives: December 2014

OpenGL point light shadows & Atomspheric FX hacks

So asset creation in the form of modeling the final environment that’ll be used in the game took the past few days. It’s amazed how much faster it went this time compared the first scene (the city street). I chalk this up to experience that I’ve picked up now that I’ve been doing this awhile, and my understanding of things that are very time consuming to model vs things I can quickly grab, import, texture, and place from turbosquid. The perfect balance of both techniques has led to me being done with the dark night alley street scene after just a few days.

Where I spent the most time this time was in the shaders. I really wanted to have quality rendering and go out with a bang since this’ll likely be the last environment that I do for the game. I want the dark alleyway to appear foggy and dark, like its just about to rain. That includes moist shiny cobblestone rocks, and a damp fog effect that feels very palpable to the player.

This idea of mine posed two major problems that I haven’t had to tackle until now.

Point Light Shadows

The first major problem, is that I could no longer “cheat” using a single directional light source representing the “sun” in the scene. Instead, I had to place a street lamp every building or so and have these be the only sources of light in the scene. This means, in order to keep my shadows working, I now had to implement shadows from a point light source. In some ways this was easier than point lighting (no longer necessary to compute an optimal shadow bounding box to be used for the light’s ortho matrix projection). However, now I needed render to and utilize cube maps to store the shadow depth information. Surprisingly, there was very little comprehensive information on the web about how to properly do PCF shadows using cubemaps.

What I found that works is the following.

Create a cubemap renderer that is positioned at the exact same position as one of the point lights - this special vgl engine object renders the scene into 6 faces of a cubemap with 90-degree fov angles to properly capture the entire scene from “all angles”.
Format the cubemap shader to have each face hold floating point depth-component values and keep the size small (256×256) since there will be alot of these.
Define an “override” shader to be used by the above cubemap renderer to ensure a simple specialized “light render” shader was used when rendering the scene into the cubemap faces.
Set the cubemap compare mode (GL_TEXTURE_COMPARE_MODE) for the rendered cubemap texture to GL_COMPARE_REF_TO_TEXTURE which helps with percentage-closest-filtering (PCF) mode to allow smoother linear-interpolated shadow edges.
Lastly, pass a rendered shadow cubemap for each light to the scene shaders in the form of samplerCubeShadow

The “light render” shader mentioned above which is used when rendering the depth information to the cubemaps looks like so:

precision mediump float;
in mediump vec4 w_pos;
out vec4 fragColor;

struct Light
{
  vec4 worldPosition;
};
uniform Light lights[8];

#define LIGHT      0
uniform vec2 nearFarPlane;

void main()
{
  // distance to light
  float distanceToLight = distance(lights[LIGHT].worldPosition.xyz, w_pos.xyz);

  float resultingColor = (distanceToLight - nearFarPlane.x) /
           (nearFarPlane.y - nearFarPlane.x);

  gl_FragDepth = resultingColor;
  fragColor = vec4(1.0);
}

precision mediump float;

in mediump vec4 w_pos;

out vec4 fragColor;

struct Light

{

vec4 worldPosition;

};

uniform Light lights[8];

#define LIGHT 0

uniform vec2 nearFarPlane;

void main()

{

// distance to light

float distanceToLight = distance(lights[LIGHT].worldPosition.xyz, w_pos.xyz);

float resultingColor = (distanceToLight - nearFarPlane.x) /

(nearFarPlane.y - nearFarPlane.x);

gl_FragDepth = resultingColor;

fragColor = vec4(1.0);

}

Basically this encodes the distance from the surface point to the light in the form normalized depth information. This is done by manually overriding the depth value stored in gl_FragDepth. Each light has its own cubemap renderer with a custom-tailored shader like this (using the correct light index to compute distance from).

The per-fragment lighting shader code that utilizes the cube shadow maps looks like:

/*....*/

#define LIGHT 0
#define ATTENUATION

uniform samplerCubeShadow shadowMap;

/*.......*/

void pointLight0(in mediump vec3 normal, in mediump vec3 eye, in mediump vec3 ecPosition3)
{
  mediump float nDotVP;       // normal . light direction
  mediump float nDotHV;       // normal . light half vector
  //mediump float pf = 0.0;           // power factor
  mediump float attenuation = 1.0;  // computed attenuation factor
  mediump float d;            // distance from surface to light source
  mediump vec3  VP;           // direction from surface to light position
  mediump vec3  halfVector;   // direction of maximum highlights

  // Compute vector from surface to light position
  VP = vec3(lights[LIGHT].position) - ecPosition3;

#ifdef ATTENUATION
  // Compute distance between surface and light position
  d = length(VP);
#endif

  // Normalize the vector from surface to light position
  VP = normalize(VP);

  // Compute attenuation
#ifdef ATTENUATION
  {
    attenuation = 1.0 / (lights[LIGHT].constantAttenuation +
                         lights[LIGHT].linearAttenuation * d +
                         lights[LIGHT].quadraticAttenuation * d * d);
  }
#endif

  nDotVP = dot(normal, VP);
  mediump vec2 frontAndBack = vec2(nDotVP, -nDotVP);
  frontAndBack = max(vec2(0.0), frontAndBack);

  float visibility = 1.0;

  // difference between position of the light source and position of the fragment
  vec3 fromLightToFragment = lights[LIGHT].worldPosition.xyz - va_position.xyz;

  // normalized distance to the point light source
  float distanceToLight = length(fromLightToFragment);
  float currentDistanceToLight = (distanceToLight - nearFarPlane.x) / (nearFarPlane.y - nearFarPlane.x);

  // normalized direction from light source for sampling
  fromLightToFragment = normalize(fromLightToFragment);      
  visibility *= max(texture(shadowMap, vec4(-fromLightToFragment, currentDistanceToLight-shadowBias), 0.0), 0.0);

  //  if(nDotVP > 0.0)
  {
    diffuse  += visibility*material.diffuse*lights[LIGHT].diffuse * frontAndBack.x * attenuation;
    diffuseBack += visibility*material.diffuse*lights[LIGHT].diffuse * frontAndBack.y * attenuation;
  }

  //if(lights[LIGHT].doSpec)
  {
    mediump vec2 cutOff = step(frontAndBack, vec2(0.0));

    halfVector = normalize(VP + eye);
    nDotHV = dot(normal, halfVector);
    frontAndBack = vec2(nDotHV, -nDotHV);
    frontAndBack = max(vec2(0.0), frontAndBack);

    lowp vec2 pf = pow(frontAndBack, vec2(material.shininess, material.shininess));
    specular += visibility*material.specular*lights[LIGHT].specular * pf.x * attenuation * cutOff.y;
    specularBack += lights[LIGHT].specular * pf.y * attenuation * cutOff.x;
  }

  ambient += lights[LIGHT].ambient * attenuation;
}

/*....*/

#define LIGHT 0

#define ATTENUATION

uniform samplerCubeShadow shadowMap;

/*.......*/

void pointLight0(in mediump vec3 normal, in mediump vec3 eye, in mediump vec3 ecPosition3)

{

mediump float nDotVP; // normal . light direction

mediump float nDotHV; // normal . light half vector

//mediump float pf = 0.0; // power factor

mediump float attenuation = 1.0; // computed attenuation factor

mediump float d; // distance from surface to light source

mediump vec3 VP; // direction from surface to light position

mediump vec3 halfVector; // direction of maximum highlights

// Compute vector from surface to light position

VP = vec3(lights[LIGHT].position) - ecPosition3;

#ifdef ATTENUATION

// Compute distance between surface and light position

d = length(VP);

#endif

// Normalize the vector from surface to light position

VP = normalize(VP);

// Compute attenuation

#ifdef ATTENUATION

{

attenuation = 1.0 / (lights[LIGHT].constantAttenuation +

lights[LIGHT].linearAttenuation * d +

lights[LIGHT].quadraticAttenuation * d * d);

}

#endif

nDotVP = dot(normal, VP);

mediump vec2 frontAndBack = vec2(nDotVP, -nDotVP);

frontAndBack = max(vec2(0.0), frontAndBack);

float visibility = 1.0;

// difference between position of the light source and position of the fragment

vec3 fromLightToFragment = lights[LIGHT].worldPosition.xyz - va_position.xyz;

// normalized distance to the point light source

float distanceToLight = length(fromLightToFragment);

float currentDistanceToLight = (distanceToLight - nearFarPlane.x) / (nearFarPlane.y - nearFarPlane.x);

// normalized direction from light source for sampling

fromLightToFragment = normalize(fromLightToFragment);

visibility *= max(texture(shadowMap, vec4(-fromLightToFragment, currentDistanceToLight-shadowBias), 0.0), 0.0);

// if(nDotVP > 0.0)

{

diffuse += visibility*material.diffuse*lights[LIGHT].diffuse * frontAndBack.x * attenuation;

diffuseBack += visibility*material.diffuse*lights[LIGHT].diffuse * frontAndBack.y * attenuation;

}

//if(lights[LIGHT].doSpec)

{

mediump vec2 cutOff = step(frontAndBack, vec2(0.0));

halfVector = normalize(VP + eye);

nDotHV = dot(normal, halfVector);

frontAndBack = vec2(nDotHV, -nDotHV);

frontAndBack = max(vec2(0.0), frontAndBack);

lowp vec2 pf = pow(frontAndBack, vec2(material.shininess, material.shininess));

specular += visibility*material.specular*lights[LIGHT].specular * pf.x * attenuation * cutOff.y;

specularBack += lights[LIGHT].specular * pf.y * attenuation * cutOff.x;

}

ambient += lights[LIGHT].ambient * attenuation;

}

Essentially the highlighted code above compares the distance between the surface point to the light to the stored depth value in the cube texture using the fragment-light vector as a lookup value into the cubemap. Because we’re sampling using a special “shadow” version of the cubemap sampler, the result will be properly interpolated between shadow texels to avoid ugly edges between shadowed and non-shadowed areas.

Luckily, I was able to build this into the Verto Studio editor and associated graphics system and test this out with relatively little trouble. Even though I have 4 or 5 statically rendered cubemap shadow textures in the entire scene. I was able to keep performance high by building versions of the shaders for each side of the street so that each individual shader only has to shade with at most 3 lights at a time. This worked out better than I had expected.

Light-Fog Interaction (Volume FX)

This part was tricky.. I had an idea in my head of how I wanted this to look. So I did something that usually leads to trouble; I tried to come up with a volume rendering technique on my own from scratch and implement it and just kinda see how it goes.

The basic idea stems from what I’ve observed in real life from foggy, dark, damp nights and lighting. Essentially, fog at night is black… IF there is no light around to interact with the fog. Naturally, if the water droplets don’t have any light to interact with them, you won’t see them and the fog will appear black. However, if any light interacts with the water vapor in the air, it’ll create the illusion of a whiter colored and denser fog. So this is what I set out to emulate with my shader.

Now atmospheric shader effects can often lead to the necessity of raymarching and heavy iteration in the fragment shader to simulate the accumulation of light-atomosphere interaction. To this I said “hell no” since ray marching of any kind in a shader terribly robs performance. I quickly realized that I could avoid raymarching entirely if I used a simple model to represent the light-fog interaction that I was going for.

In my case, it turned out I could do the whole effect using something as simple as a sphere intersection test. Basically, when I’m shading a pixel (a point on the surface), I’m interested in what happens to the light on its way back from the surface to the viewer, the surface-to-viewer vector. If the atmosphere affects the light at any point along this vector, I’ll need to compute that. In other words, if the ray from the surface to the viewer intersects a sphere centered at the light, then the fog affects the light on the way back to the viewer. How much? Well if I calculate the length of the segment between the entry and exit points of the ray intersection (how much of the sphere the ray pierces), I find that length is proportional to both the perceived increase in density of the fog and the brightening of the fog color.

This algorithm is given below in fragment shader code:

/*.....*/

uniform float fogDensity;
uniform vec3 fogColor;

//sphere intersection
bool intersect(vec3 raydir, vec3 rayorig, vec3 pos, float radiusSquared, 
               out float innerSegmentLength)
{
   float t0, t1; // solutions for t if the ray intersects

   // geometric solution
   vec3 L = pos - rayorig;
   float tca = dot(L, raydir);

   //seems to be true if ray o is inside the sphere
   //we want this to be a positive..
   //if(tca < 0) 
   //  return false;

   float d2 = dot(L, L) - tca * tca;
   if(d2 > radiusSquared)
     return false;
   float thc = sqrt(radiusSquared - d2);
   t0 = tca - thc;
   t1 = tca + thc;

   innerSegmentLength = abs(t0-t1);

   return true;
}

vec4 computeFog(vec4 color)
{
  vec3 viewDirection = normalize(cameraPosition - vec3(va_position));

  vec3 surfacePos = ec_pos.xyz/va_position.w;
  const float LOG2 = 1.442695;
  float fogFactor = exp2(-fogDensity * length(surfacePos) * LOG2);  

  vec3 fogCol = fogColor;
  vec3 rayO = vec3(va_position);
  vec3 rayD = viewDirection;
  float len = 0.0;
  const float r = 80.0;

  //for each light we interact with...
  if(intersect(rayD, rayO, lights[LIGHT].worldPosition.xyz, r*r, len))
  {
    float d = len/r;
    float p = clamp(log(d)*d, 0.0, 1.0);
    fogCol = mix(fogColor, vec3(0.4), p);

    len = 0.0;
    const float innerR = 25.0f;
    if(intersect(rayD, rayO, lights[LIGHT].worldPosition.xyz, innerR*innerR, len))
    {
      float len10 = len/innerR;
      float nd = min(len10*0.25, 1.0);
      fogFactor *= mix(1.0, 0.0, nd);

      fogCol += mix(vec3(0.0), vec3(0.2), len10);
    }
  }

  if(intersect(rayD, rayO, lights[LIGHT1].worldPosition.xyz, r*r, len))
  {
    float d = len/r;
    float p = clamp(log(d)*d, 0.0, 1.0);
    fogCol = mix(fogColor, vec3(0.4), p);

    len = 0.0;
    const float innerR = 25.0f;
    if(intersect(rayD, rayO, lights[LIGHT1].worldPosition.xyz, innerR*innerR, len))
    {
      float len10 = len/innerR;
      float nd = min(len10*0.25, 1.0);
      fogFactor *= mix(1.0, 0.0, nd);

      fogCol += mix(vec3(0.0), vec3(0.2), len10);
    }
  }

  return mix(vec4(fogCol, 1.0), color, fogFactor);
}

/*.....*/

uniform float fogDensity;

uniform vec3 fogColor;

//sphere intersection

bool intersect(vec3 raydir, vec3 rayorig, vec3 pos, float radiusSquared,

out float innerSegmentLength)

{

float t0, t1; // solutions for t if the ray intersects

// geometric solution

vec3 L = pos - rayorig;

float tca = dot(L, raydir);

//seems to be true if ray o is inside the sphere

//we want this to be a positive..

//if(tca < 0)

// return false;

float d2 = dot(L, L) - tca * tca;

if(d2 > radiusSquared)

return false;

float thc = sqrt(radiusSquared - d2);

t0 = tca - thc;

t1 = tca + thc;

innerSegmentLength = abs(t0-t1);

return true;

}

vec4 computeFog(vec4 color)

{

vec3 viewDirection = normalize(cameraPosition - vec3(va_position));

vec3 surfacePos = ec_pos.xyz/va_position.w;

const float LOG2 = 1.442695;

float fogFactor = exp2(-fogDensity * length(surfacePos) * LOG2);

vec3 fogCol = fogColor;

vec3 rayO = vec3(va_position);

vec3 rayD = viewDirection;

float len = 0.0;

const float r = 80.0;

//for each light we interact with...

if(intersect(rayD, rayO, lights[LIGHT].worldPosition.xyz, r*r, len))

{

float d = len/r;

float p = clamp(log(d)*d, 0.0, 1.0);

fogCol = mix(fogColor, vec3(0.4), p);

len = 0.0;

const float innerR = 25.0f;

if(intersect(rayD, rayO, lights[LIGHT].worldPosition.xyz, innerR*innerR, len))

{

float len10 = len/innerR;

float nd = min(len10*0.25, 1.0);

fogFactor *= mix(1.0, 0.0, nd);

fogCol += mix(vec3(0.0), vec3(0.2), len10);

}

if(intersect(rayD, rayO, lights[LIGHT1].worldPosition.xyz, r*r, len))

{

float d = len/r;

float p = clamp(log(d)*d, 0.0, 1.0);

fogCol = mix(fogColor, vec3(0.4), p);

len = 0.0;

const float innerR = 25.0f;

if(intersect(rayD, rayO, lights[LIGHT1].worldPosition.xyz, innerR*innerR, len))

{

float len10 = len/innerR;

float nd = min(len10*0.25, 1.0);

fogFactor *= mix(1.0, 0.0, nd);

fogCol += mix(vec3(0.0), vec3(0.2), len10);

}

return mix(vec4(fogCol, 1.0), color, fogFactor);

}

I’m sure I’m not the first one to come up with this idea before, but it still did feel pretty cool to reason my way through a shading problem like this. The visual result looks amazing.

The progress of this all is below in a gallery.

How to 3D sound in windows (hint: it’s not OpenAL)

Leave a reply

So yesterday and today, I bit the bullet and started working on compiling the game for windows. I wanted to do this before the project got too much further along so I didn’t write any code that would have serious implications on windows. I must say, I’m amazed at how much trouble I didn’t have with the rendering system. The levels/screens load up and look great! I did have some issues with visual studio that I chalk up to lack of experience in windows dev on my part. Namely getting the builds to be 64-bit instead of the default 32, SSE3 operator overload stuff, and getting debug builds to forego some of the extremely slow settings that make the engine run at a crawl. Once I took care of all that, my little game was running on windows like a champ, complete with same stellar OpenGL 3.3 performance I get on mac. All was well… that is except for one little tiny issue. The sound.

One thing that became apparent almost right away was that OpenAL support on windows is in a terrible state. Counting on it to just be there like OpenGL is a no no.. and installing your own via something like OpenAL soft gives pretty poor support. I realized that I had to do something I’ve never done before, and that’s directly use a DirectX subsystem. After some quick googling I discovered what I need can be done using a combination of xAudio2 and X3DAudio. Both of these systems come with DirectX and the DX SDK so I was already set up to use em.

So basically, the plan on windows was pretty simple: since I already have all my 3D sound calls abstracted out in my SoundManager class, I all I had to do is re-implement the stuff that OpenAL handled using this new xAudio2 stuff.. and pray that it played nice along side SDL_Mixer which I use for mp3 music. More or less, things went relatively painless and I was able to find an xAudio2/X3DAudio analog for just about everything I was doing in OpenAL EXCEPT for a little issue I had with source playback.

On mac, I’m able to play a source overtop of itself as many times as I want to. On windows, this seems to be a no-go. So for win, I had to manually pre-allocate pools of sources for the sounds that I would be playing rapidly such as bullet ricochet sounds and typewriter keystrokes and ensure that I return an available source whenever I ask to play that sound. That process more or less worked out just fine.

After all that, we have my reworked soundmanager class for windows. The code is below for anyone who wants to learn from it.

//
//  SoundManager.h
//  Gangster Driveby
//
//

#ifndef __Gangster_Driveby__SoundManager__
#define __Gangster_Driveby__SoundManager__

#include <windows.h>
#include <xaudio2.h>
#include <x3daudio.h>
#include <vector>
#include <iostream>
#include "VecTypes.h"
#include "Wave.h"
#include "SDL_mixer.h"

class SoundManager
{
public:
  enum Effect
  {
    Engine = 0,
    Typewriter1,
    Typewriter2,
    Typewriter3,
    Typewriter4,
    Typewriter5,
    TypewriterSpace,
    TypewriterDing,
    TypewriterFeed,
    TommyGun,
    TommyGunLoop,
    GlassBreak,
    BulbBreak,
    BulletHoleGlass,
    DramaticBoom,
    PaperIn,
    PaperOut,
    OceanClose,
    WavesAmbience,

    //MUST be last
    LongMusic
  };

  enum Music
  {
    DarkFlashes = 0,
    NightDocksSax, 
    DontGoWayNobody
  };

  class MultiSource
  {
    friend class SoundManager;
  public:
    MultiSource(IXAudio2 *engine, Wave *wave, int maxSources = 1)
    {
      for(int i = 0; i < maxSources; i++)
      {
        IXAudio2SourceVoice *source = nullptr;

        if(FAILED(engine->CreateSourceVoice(&source, wave->wf())))
        {
          std::cerr << "source creation error!" << std::endl;
        }
        sources.push_back(source);
      }

      ZeroMemory(&emitter, sizeof(emitter));
      emitter.ChannelCount = 1; //not 100% on this
      emitter.CurveDistanceScaler = FLT_MIN;

      ZeroMemory(&dspSettings, sizeof(dspSettings));
      FLOAT32 *dspMatrix = nullptr;

      XAUDIO2_DEVICE_DETAILS details;
      engine->GetDeviceDetails(0, &details);

      dspMatrix = new FLOAT32[details.OutputFormat.Format.nChannels];
      dspSettings.SrcChannelCount = 1;
      dspSettings.DstChannelCount = details.OutputFormat.Format.nChannels;
      dspSettings.pMatrixCoefficients = dspMatrix;
    }

    ///Returns an available source
    IXAudio2SourceVoice *getSource();

    ///Updates and calculates 3D sound stuff
    void update3D(SoundManager *sm);

    //One or many sources per sound
    std::vector<IXAudio2SourceVoice *> sources;

    //3D sound emitter (if needed for this source)
    X3DAUDIO_EMITTER emitter;
    X3DAUDIO_DSP_SETTINGS dspSettings;
    bool is3D = false, is3DUpdated = false;

    //Useful to keep track of this since 3D audio calculates its own
    float basePitch = 1.0f;
  };

  SoundManager();
  ~SoundManager();

  static SoundManager *manager(); 

  inline bool isActive() { return (engine != nullptr); }

  void setEffectBasicProperties(Effect effect, float gain, float pitch=1.0f, bool looping=false);
  void setEffectGain(Effect effect, float gain);
  void setEffectPitch(Effect effect, float pitch);

  void setEffectSpatialProperties(Effect effect, float referenceDistance, float maxDistance, float rollOffFactor=0.0f,
                                  float3 velocity = float3::zero, float3 direction = float3::zero);

  void setEffectPosition(Effect effect, float3 pos);

  void setListenerPosition(float3 pos, float3 orientation[]);

  void playEffect(Effect effect);
  void stopEffect(Effect effect);

  void playSong(Music song);
  void stopSong();
  void setSongVolume(float volume);
  void setSongPosition(double pos);
  float getSongVolume();

  void loadLongMusicEffect(const char *fn);
  void unloadLongMusicEffect();

  inline void setSoundAvailable(bool b) { soundAvailable = b; }

  void update3D();

private:
  void loadSoundEffects();

  IXAudio2 *engine = nullptr;
  IXAudio2MasteringVoice *master = nullptr;
  XAUDIO2_DEVICE_DETAILS deviceDetails;

  X3DAUDIO_HANDLE engine3d;
  X3DAUDIO_LISTENER listener;

  std::vector<MultiSource *> sources;
  std::vector<Wave *> buffers;
  Wave *longMusicBuffer = nullptr;
  MultiSource *longMusicSource = nullptr;

  Mix_Music *currentSong = nullptr;

  bool soundAvailable = true;  
  float currentSongVolume = 1.0f;
};

#endif /* defined(__Gangster_Driveby__SoundManager__) */

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

// SoundManager.h

// Gangster Driveby

#ifndef __Gangster_Driveby__SoundManager__

#define __Gangster_Driveby__SoundManager__

#include <windows.h>

#include <xaudio2.h>

#include <x3daudio.h>

#include <vector>

#include <iostream>

#include "VecTypes.h"

#include "Wave.h"

#include "SDL_mixer.h"

class SoundManager

{

public:

enum Effect

{

Engine = 0,

Typewriter1,

Typewriter2,

Typewriter3,

Typewriter4,

Typewriter5,

TypewriterSpace,

TypewriterDing,

TypewriterFeed,

TommyGun,

TommyGunLoop,

GlassBreak,

BulbBreak,

BulletHoleGlass,

DramaticBoom,

PaperIn,

PaperOut,

OceanClose,

WavesAmbience,

//MUST be last

LongMusic

};

enum Music

{

DarkFlashes = 0,

NightDocksSax,

DontGoWayNobody

};

class MultiSource

{

friend class SoundManager;

public:

MultiSource(IXAudio2 *engine, Wave *wave, int maxSources = 1)

{

for(int i = 0; i < maxSources; i++)

{

IXAudio2SourceVoice *source = nullptr;

if(FAILED(engine->CreateSourceVoice(&source, wave->wf())))

{

std::cerr << "source creation error!" << std::endl;

}

sources.push_back(source);

}

ZeroMemory(&emitter, sizeof(emitter));

emitter.ChannelCount = 1; //not 100% on this

emitter.CurveDistanceScaler = FLT_MIN;

ZeroMemory(&dspSettings, sizeof(dspSettings));

FLOAT32 *dspMatrix = nullptr;

XAUDIO2_DEVICE_DETAILS details;

engine->GetDeviceDetails(0, &details);

dspMatrix = new FLOAT32[details.OutputFormat.Format.nChannels];

dspSettings.SrcChannelCount = 1;

dspSettings.DstChannelCount = details.OutputFormat.Format.nChannels;

dspSettings.pMatrixCoefficients = dspMatrix;

}

///Returns an available source

IXAudio2SourceVoice *getSource();

///Updates and calculates 3D sound stuff

void update3D(SoundManager *sm);

//One or many sources per sound

std::vector<IXAudio2SourceVoice *> sources;

//3D sound emitter (if needed for this source)

X3DAUDIO_EMITTER emitter;

X3DAUDIO_DSP_SETTINGS dspSettings;

bool is3D = false, is3DUpdated = false;

//Useful to keep track of this since 3D audio calculates its own

float basePitch = 1.0f;

};

SoundManager();

~SoundManager();

static SoundManager *manager();

inline bool isActive() { return (engine != nullptr); }

void setEffectBasicProperties(Effect effect, float gain, float pitch=1.0f, bool looping=false);

void setEffectGain(Effect effect, float gain);

void setEffectPitch(Effect effect, float pitch);

void setEffectSpatialProperties(Effect effect, float referenceDistance, float maxDistance, float rollOffFactor=0.0f,

float3 velocity = float3::zero, float3 direction = float3::zero);

void setEffectPosition(Effect effect, float3 pos);

void setListenerPosition(float3 pos, float3 orientation[]);

void playEffect(Effect effect);

void stopEffect(Effect effect);

void playSong(Music song);

void stopSong();

void setSongVolume(float volume);

void setSongPosition(double pos);

float getSongVolume();

void loadLongMusicEffect(const char *fn);

void unloadLongMusicEffect();

inline void setSoundAvailable(bool b) { soundAvailable = b; }

void update3D();

private:

void loadSoundEffects();

IXAudio2 *engine = nullptr;

IXAudio2MasteringVoice *master = nullptr;

XAUDIO2_DEVICE_DETAILS deviceDetails;

X3DAUDIO_HANDLE engine3d;

X3DAUDIO_LISTENER listener;

std::vector<MultiSource *> sources;

std::vector<Wave *> buffers;

Wave *longMusicBuffer = nullptr;

MultiSource *longMusicSource = nullptr;

Mix_Music *currentSong = nullptr;

bool soundAvailable = true;

float currentSongVolume = 1.0f;

};

#endif /* defined(__Gangster_Driveby__SoundManager__) */

//
//  SoundManager.cpp
//  Gangster Driveby
//
//  Created by Mike Farrell on 11/14/14.
//  Copyright (c) 2014 Mike Farrell. All rights reserved.
//

#include <cstdlib>
#include <iostream>
#include "PortableSoundInfo.h"
#include "SoundManager.h"
#include "SDL_mixer.h"

static SoundManager *currentSoundManager = nullptr;

static const char *sounds[] = {
  "eng_1_mid_loop.wav",
  "type1.wav",
  "type2.wav",
  "type3.wav",
  "type4.wav",
  "type5.wav",
  "type_space.wav",
  "type_ding.wav",
  "type_feed.wav",
  "tommy.wav",
  "tommy_loop.wav",
  "glassbreak.wav",
  "bulb_break.wav",
  "bullethole_glass.wav",
  "dramatic_boom.wav",
  "paper_in.wav",
  "paper_out.wav",
  "ocean_close.wav",
  "waves_ambience.wav"
};

static const int soundSourceCounts[] = {
  1, 
  8, 8, 8, 8, 8, 8, 
  1, 1, 
  8, 1, 
  8, 8, 8, 
  1, 1, 1, 1, 1 
};

static const char *songs[] = {
/*....*/
};

static const int numSounds = sizeof(sounds)/sizeof(char *);
//static const int numSongs = sizeof(songs)/sizeof(char *);

using namespace std;

SoundManager::SoundManager()
{
  //create the engine
  if(!FAILED(XAudio2Create(&engine)))
  {
    //create the mastering voice
    if(!FAILED(engine->CreateMasteringVoice(&master)))
    {
      //create 3D sound engine
      DWORD dwChannelMask;
      if(!FAILED(engine->GetDeviceDetails(0, &deviceDetails)))
      {
        dwChannelMask = deviceDetails.OutputFormat.dwChannelMask;
        X3DAudioInitialize(dwChannelMask, X3DAUDIO_SPEED_OF_SOUND, engine3d);
        ZeroMemory(&listener, sizeof(listener));    
      }
      else
      {
        cerr << "3D sound init failed!" << endl;
      }

      //and load sound fx
      loadSoundEffects();
    }
  }
  else
  {
    cerr << "Could not initialize DirectSound for sound!" << endl;
  }

  currentSoundManager = this;
}

SoundManager::~SoundManager()
{
  if(engine)
    engine->Release();
  for(Wave *wave : buffers) if(wave)
    delete wave;
  for(MultiSource *ms : sources) if(ms)
    delete ms;
  if(currentSong)
    Mix_FreeMusic(currentSong);
}

SoundManager *SoundManager::manager()
{
  return currentSoundManager;
}

void SoundManager::loadSoundEffects()
{
  auto sz = numSounds;

  for(int i = 0; i < numSounds; i++)
  {
    Wave *buffer = new Wave();
    if(!buffer->load(sounds[i]))
    {
      cerr << "shilt" << endl;
    }

    int maxSources = soundSourceCounts[i];
    MultiSource *source = new MultiSource(engine, buffer, maxSources);

    buffers.push_back(buffer);
    sources.push_back(source);
  }

  //long music placeholder
  sources.push_back(nullptr);
  buffers.push_back(nullptr);
}

void SoundManager::update3D()
{
  for(auto s : sources) if(s)
  {
    s->update3D(this);
  }
}

void SoundManager::loadLongMusicEffect(const char *fn)
{
  if(longMusicBuffer)
    unloadLongMusicEffect();

  PortableSoundInfo sound(fn);
  longMusicBuffer = new Wave();
  if(!longMusicBuffer->load(sound))
  {
    cerr << "long music load failed!" << endl;
  }
  longMusicSource = new MultiSource(engine, longMusicBuffer, 1);

  sources[(int)LongMusic] = longMusicSource;
  buffers[(int)LongMusic] = longMusicBuffer;
}

void SoundManager::unloadLongMusicEffect()
{
  delete longMusicBuffer;
  longMusicBuffer = nullptr;
  delete longMusicSource;
  longMusicSource = nullptr;
}

void SoundManager::setEffectGain(SoundManager::Effect effect, float gain)
{
  const float vol = gain;

  auto source = sources[(int)effect];
  for(auto s : source->sources)
  {
    s->SetVolume(vol);
  }
}

void SoundManager::setEffectBasicProperties(SoundManager::Effect effect, float gain, float pitch, bool looping)
{
  setEffectGain(effect, gain);
  if(pitch != 0.0f)
  {
    setEffectPitch(effect, pitch);
  }

  if(!looping)
  {
    buffers[(int)effect]->xaBuffer()->LoopCount = 0;
  }
  else
  {
    buffers[(int)effect]->xaBuffer()->LoopCount = XAUDIO2_MAX_LOOP_COUNT;
  }
}

void SoundManager::setEffectPitch(SoundManager::Effect effect, float pitch)
{
  auto source = sources[(int)effect];
  for(auto s : source->sources)
  {
    const float ratio = pitch;
    s->SetFrequencyRatio(ratio);
  }

  //mm.. mmmm, pitch!
  source->basePitch = pitch;
}

void SoundManager::setEffectSpatialProperties(SoundManager::Effect effect, float referenceDistance, float maxDistance, float rollOffFactor,
  float3 velocity, float3 direction)
{
  //stereo tracks get a horrible sound blasting result when you try to do this
  //lets just do what openAL does, which is silently fail
  if(buffers[(int)effect]->wf()->nChannels > 1)
  {
    return;
  }

  //not reversible for now
  auto source = sources[(int)effect];
  source->is3D = true;

  source->emitter.Velocity = { velocity.x, velocity.y, velocity.z };
  if(direction != float3::zero)
  {
    source->emitter.OrientFront = { direction.x, direction.y, direction.z };
    source->emitter.OrientTop = listener.OrientTop;
  }

  //a "max distance" isn't available in x3d audio it seems..

  float distanceScalar = referenceDistance - rollOffFactor*2.5;
  source->emitter.CurveDistanceScaler = max(0.0f, distanceScalar);
}

void SoundManager::setEffectPosition(Effect effect, float3 pos)
{
  auto source = sources[(int)effect];
  source->emitter.Position = { pos.x, pos.y, pos.z };
}

void SoundManager::setListenerPosition(float3 pos, float3 orientation[])
{
  listener.Position = { pos.x, pos.y, pos.z };
  listener.OrientFront = { orientation[0].x, orientation[0].y, orientation[0].z };
  listener.OrientTop = { orientation[1].x, orientation[1].y, orientation[1].z };
}

void SoundManager::playEffect(SoundManager::Effect effect)
{
  auto ms = sources[(int)effect];
  auto source = ms->getSource();

  //ensure initial 3D calculations done if requested
  if(ms->is3D && !ms->is3DUpdated)
  {
    ms->update3D(this);
  }

  source->Start();
  source->SubmitSourceBuffer(buffers[(int)effect]->xaBuffer());
}

void SoundManager::stopEffect(SoundManager::Effect effect)
{
  auto source = sources[(int)effect];
  for(auto s : source->sources)
  {
    s->Stop();
  }
}

void SoundManager::playSong(SoundManager::Music song)
{
  if(soundAvailable)
  {
    if(currentSong)
      Mix_FreeMusic(currentSong);
    currentSong = Mix_LoadMUS(songs[(int)song]);
    Mix_PlayMusic(currentSong, -1);
  }
}

void SoundManager::stopSong()
{
  if(soundAvailable)
  {
    Mix_HaltMusic();
  }
}

void SoundManager::setSongVolume(float volume)
{
  currentSongVolume = volume;
  if(soundAvailable)
  {
    Mix_VolumeMusic((int)roundf(volume*MIX_MAX_VOLUME));
  }
}

void SoundManager::setSongPosition(double pos)
{
  if(soundAvailable)
  {
    Mix_SetMusicPosition(pos);
  }
}

float SoundManager::getSongVolume()
{
  return currentSongVolume;
}

IXAudio2SourceVoice *SoundManager::MultiSource::getSource()
{
  auto isPlaying = [](IXAudio2SourceVoice *source)->bool {
    XAUDIO2_VOICE_STATE state;
    ZeroMemory(&state, sizeof(XAUDIO2_VOICE_STATE));

    source->GetState(&state);
    return (state.BuffersQueued != 0);
  };

  int i = 0;
  while(i < sources.size() && isPlaying(sources[i]))
  {
    i++;
  }

  if(i == sources.size())
  {
    if(sources.size() > 1)
    {
#ifdef DEBUG
      cout << "Warning:  sound queue will occur (this will suck)" << endl;
#endif
    }
    return sources[0];
  }

  return sources[i];
}

void SoundManager::MultiSource::update3D(SoundManager *sm)
{
  if(is3D)
  {
    //X3DAUDIO_CALCULATE_LPF_DIRECT ?
    X3DAudioCalculate(sm->engine3d, &sm->listener, &emitter, X3DAUDIO_CALCULATE_MATRIX | X3DAUDIO_CALCULATE_DOPPLER, &dspSettings);
    for(auto s : sources)
    {
      s->SetOutputMatrix(sm->master, 1, sm->deviceDetails.OutputFormat.Format.nChannels, dspSettings.pMatrixCoefficients);
      s->SetFrequencyRatio(basePitch * dspSettings.DopplerFactor);
    }

    is3DUpdated = true;
  }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

// SoundManager.cpp

// Gangster Driveby

// Created by Mike Farrell on 11/14/14.

#include <cstdlib>

#include <iostream>

#include "PortableSoundInfo.h"

#include "SoundManager.h"

#include "SDL_mixer.h"

static SoundManager *currentSoundManager = nullptr;

static const char *sounds[] = {

"eng_1_mid_loop.wav",

"type1.wav",

"type2.wav",

"type3.wav",

"type4.wav",

"type5.wav",

"type_space.wav",

"type_ding.wav",

"type_feed.wav",

"tommy.wav",

"tommy_loop.wav",

"glassbreak.wav",

"bulb_break.wav",

"bullethole_glass.wav",

"dramatic_boom.wav",

"paper_in.wav",

"paper_out.wav",

"ocean_close.wav",

"waves_ambience.wav"

};

static const int soundSourceCounts[] = {

8, 8, 8, 8, 8, 8,

1, 1,

8, 1,

8, 8, 8,

1, 1, 1, 1, 1

};

static const char *songs[] = {

/*....*/

};

static const int numSounds = sizeof(sounds)/sizeof(char *);

//static const int numSongs = sizeof(songs)/sizeof(char *);

using namespace std;

SoundManager::SoundManager()

{

//create the engine

if(!FAILED(XAudio2Create(&engine)))

{

//create the mastering voice

if(!FAILED(engine->CreateMasteringVoice(&master)))

{

//create 3D sound engine

DWORD dwChannelMask;

if(!FAILED(engine->GetDeviceDetails(0, &deviceDetails)))

{

dwChannelMask = deviceDetails.OutputFormat.dwChannelMask;

X3DAudioInitialize(dwChannelMask, X3DAUDIO_SPEED_OF_SOUND, engine3d);

ZeroMemory(&listener, sizeof(listener));

}

else

{

cerr << "3D sound init failed!" << endl;

}

//and load sound fx

loadSoundEffects();

}

else

{

cerr << "Could not initialize DirectSound for sound!" << endl;

}

currentSoundManager = this;

}

SoundManager::~SoundManager()

{

if(engine)

engine->Release();

for(Wave *wave : buffers) if(wave)

delete wave;

for(MultiSource *ms : sources) if(ms)

delete ms;

if(currentSong)

Mix_FreeMusic(currentSong);

}

SoundManager *SoundManager::manager()

{

return currentSoundManager;

}

void SoundManager::loadSoundEffects()

{

auto sz = numSounds;

for(int i = 0; i < numSounds; i++)

{

Wave *buffer = new Wave();

if(!buffer->load(sounds[i]))

{

cerr << "shilt" << endl;

}

int maxSources = soundSourceCounts[i];

MultiSource *source = new MultiSource(engine, buffer, maxSources);

buffers.push_back(buffer);

sources.push_back(source);

}

//long music placeholder

sources.push_back(nullptr);

buffers.push_back(nullptr);

}

void SoundManager::update3D()

{

for(auto s : sources) if(s)

{

s->update3D(this);

}

void SoundManager::loadLongMusicEffect(const char *fn)

{

if(longMusicBuffer)

unloadLongMusicEffect();

PortableSoundInfo sound(fn);

longMusicBuffer = new Wave();

if(!longMusicBuffer->load(sound))

{

cerr << "long music load failed!" << endl;

}

longMusicSource = new MultiSource(engine, longMusicBuffer, 1);

sources[(int)LongMusic] = longMusicSource;

buffers[(int)LongMusic] = longMusicBuffer;

}

void SoundManager::unloadLongMusicEffect()

{

delete longMusicBuffer;

longMusicBuffer = nullptr;

delete longMusicSource;

longMusicSource = nullptr;

}

void SoundManager::setEffectGain(SoundManager::Effect effect, float gain)

{

const float vol = gain;

auto source = sources[(int)effect];

for(auto s : source->sources)

{

s->SetVolume(vol);

}

void SoundManager::setEffectBasicProperties(SoundManager::Effect effect, float gain, float pitch, bool looping)

{

setEffectGain(effect, gain);

if(pitch != 0.0f)

{

setEffectPitch(effect, pitch);

}

if(!looping)

{

buffers[(int)effect]->xaBuffer()->LoopCount = 0;

}

else

{

buffers[(int)effect]->xaBuffer()->LoopCount = XAUDIO2_MAX_LOOP_COUNT;

}

void SoundManager::setEffectPitch(SoundManager::Effect effect, float pitch)

{

auto source = sources[(int)effect];

for(auto s : source->sources)

{

const float ratio = pitch;

s->SetFrequencyRatio(ratio);

}

//mm.. mmmm, pitch!

source->basePitch = pitch;

}

void SoundManager::setEffectSpatialProperties(SoundManager::Effect effect, float referenceDistance, float maxDistance, float rollOffFactor,

float3 velocity, float3 direction)

{

//stereo tracks get a horrible sound blasting result when you try to do this

//lets just do what openAL does, which is silently fail

if(buffers[(int)effect]->wf()->nChannels > 1)

{

return;

}

//not reversible for now

auto source = sources[(int)effect];

source->is3D = true;

source->emitter.Velocity = { velocity.x, velocity.y, velocity.z };

if(direction != float3::zero)

{

source->emitter.OrientFront = { direction.x, direction.y, direction.z };

source->emitter.OrientTop = listener.OrientTop;

}

//a "max distance" isn't available in x3d audio it seems..

float distanceScalar = referenceDistance - rollOffFactor*2.5;

source->emitter.CurveDistanceScaler = max(0.0f, distanceScalar);

}

void SoundManager::setEffectPosition(Effect effect, float3 pos)

{

auto source = sources[(int)effect];

source->emitter.Position = { pos.x, pos.y, pos.z };

}

void SoundManager::setListenerPosition(float3 pos, float3 orientation[])

{

listener.Position = { pos.x, pos.y, pos.z };

listener.OrientFront = { orientation[0].x, orientation[0].y, orientation[0].z };

listener.OrientTop = { orientation[1].x, orientation[1].y, orientation[1].z };

}

void SoundManager::playEffect(SoundManager::Effect effect)

{

auto ms = sources[(int)effect];

auto source = ms->getSource();

//ensure initial 3D calculations done if requested

if(ms->is3D && !ms->is3DUpdated)

{

ms->update3D(this);

}

source->Start();

source->SubmitSourceBuffer(buffers[(int)effect]->xaBuffer());

}

void SoundManager::stopEffect(SoundManager::Effect effect)

{

auto source = sources[(int)effect];

for(auto s : source->sources)

{

s->Stop();

}

void SoundManager::playSong(SoundManager::Music song)

{

if(soundAvailable)

{

if(currentSong)

Mix_FreeMusic(currentSong);

currentSong = Mix_LoadMUS(songs[(int)song]);

Mix_PlayMusic(currentSong, -1);

}

void SoundManager::stopSong()

{

if(soundAvailable)

{

Mix_HaltMusic();

}

void SoundManager::setSongVolume(float volume)

{

currentSongVolume = volume;

if(soundAvailable)

{

Mix_VolumeMusic((int)roundf(volume*MIX_MAX_VOLUME));

}

void SoundManager::setSongPosition(double pos)

{

if(soundAvailable)

{

Mix_SetMusicPosition(pos);

}

float SoundManager::getSongVolume()

{

return currentSongVolume;

}

IXAudio2SourceVoice *SoundManager::MultiSource::getSource()

{

auto isPlaying = [](IXAudio2SourceVoice *source)->bool {

XAUDIO2_VOICE_STATE state;

ZeroMemory(&state, sizeof(XAUDIO2_VOICE_STATE));

source->GetState(&state);

return (state.BuffersQueued != 0);

};

int i = 0;

while(i < sources.size() && isPlaying(sources[i]))

{

i++;

}

if(i == sources.size())

{

if(sources.size() > 1)

{

#ifdef DEBUG

cout << "Warning: sound queue will occur (this will suck)" << endl;

#endif

}

return sources[0];

}

return sources[i];

}

void SoundManager::MultiSource::update3D(SoundManager *sm)

{

if(is3D)

{

//X3DAUDIO_CALCULATE_LPF_DIRECT ?

X3DAudioCalculate(sm->engine3d, &sm->listener, &emitter, X3DAUDIO_CALCULATE_MATRIX | X3DAUDIO_CALCULATE_DOPPLER, &dspSettings);

for(auto s : sources)

{

s->SetOutputMatrix(sm->master, 1, sm->deviceDetails.OutputFormat.Format.nChannels, dspSettings.pMatrixCoefficients);

s->SetFrequencyRatio(basePitch * dspSettings.DopplerFactor);

}

is3DUpdated = true;

}

All of the multiple source stuff is essentially handled by my inner MultiSource class which handles the souce pools for sounds that need em. I found that I never needed more than 8 sources per sound. I also had to add a new update3D method to update the 3D sound calculations and call it once every other frame or so (which wasn’t needed in OpenAL since it was done automatically).

Not shown here is a Wave class which essentially is a slightly modified version of the one from this tutorial. That class basically handles wav file loading for me and manages the sound buffer data internally.

That’s just about it. Everything else was pretty straightforward.

Screenshot of the arbitrary time period

Leave a reply

Nobody ever said developing in windows was easy…

So much progress

Leave a reply

So much has been done. I’ve gotten the menu screen (title) finished and it’s beutifal. I got the early music soundtrack for the title and first two levels in place, and a way to transition between all of these things.

My latest work has been the pre and post status screen in the game showing you how many enemies and bystanders to expect before each level, and how many you correctly or incorrectly killed at the end.

Some of the funnest work has been coding up the cutscenes for introducing the environments in the game. I’m still planning on only doing 3 environments (otherwise I’ll never finish this game), and making the 36 total levels out of different takes on these same environments.

Here’s a tease of the intro into the second environment of the game.

Just screenshots today

Leave a reply

Two screenshots..

C++11′s take on retain cycles

Leave a reply

I’d like to take a brief moment to discuss something rather programmy today. Namely, the topic of retain cycles. If you’ve ever programmed in Objective-C before (since the addition of ARC) or Swift, this concept will be very familiar to you. But if you’re coming from other programming languages such as the older (pre-11) C++, This concept will be new and you might want to read on.

Now before I get started, let me say that this example may not be the most correct way to handle delegation via lambda (block) functions in C++11. Many will tell you that shared ownership of objects in C++ can get you into trouble, and this is certainly a case of how that can happen. However it’s the most conceptually compatible with other languages, which I value over “c++ correctness” due to the fact that I’m porting code from another language to start with, and shared ownership allows me to get work done without reasoning around the lifespan of my objects.

Now as many newcomers to C++11 know, concepts such as smart pointers and lambda callback delegation can really save you alot of time when developing new systems. However, there’s ahidden evil in using these two concepts together, that must be uncovered.

Example 1 :: All is well

Consider this example:

#include <iostream>
#include <functional>
#include <memory>

using namespace std;

class A
{
public:
  ~A()
  {
    cout << "object freed!" << endl;
  }

  std::function<void()> someCallback;

  void f()
  {
    if(someCallback)
    {
      someCallback();
    }
  }  

  int someProperty = 100;
};

int main()
{
  shared_ptr<A> obj1;  
  obj1 = make_shared<A>();

  obj1->someCallback = [] {
    cout << "obj1 callback called!" << endl;
  };

  obj1->f();
  return 0;
}

#include <iostream>

#include <functional>

#include <memory>

using namespace std;

class A

{

public:

~A()

{

cout << "object freed!" << endl;

}

std::function<void()> someCallback;

void f()

{

if(someCallback)

{

someCallback();

}

int someProperty = 100;

};

int main()

{

shared_ptr<A> obj1;

obj1 = make_shared<A>();

obj1->someCallback = [] {

cout << "obj1 callback called!" << endl;

};

obj1->f();

return 0;

}

and its output:

obj1 callback called!
object freed!

Exited with status code: 0

obj1 callback called!

object freed!

Exited with status code: 0

What we’ve essentially done here is created an object (via shared pointer), set a callback lambda, and issued a method call which at some point calls our callback. We see that everything is working as desired. Most importantly, we see that our object is properly deleted when we are finished our program (when the shared pointer goes out of scope).

Example 2 :: Not so much

Now lets look what happens when I add a slightly more complicated, second case.

#include <iostream>
#include <functional>
#include <memory>

using namespace std;

class A
{
public:
  ~A()
  {
    cout << "object freed!" << endl;
  }

  std::function<void()> someCallback;

  void f()
  {
    if(someCallback)
    {
      someCallback();
    }
  }  

  int someProperty = 100;
};

int main()
{
  shared_ptr<A> obj1, obj2;

  obj1 = make_shared<A>();
  obj2 = make_shared<A>();

  obj1->someCallback = [] {
    cout << "obj1 callback called!" << endl;
  };

  obj1->f();

  obj2->someCallback = [obj2] {
    cout << "obj2 callback called: " << obj2->someProperty << endl;
  };

  obj2->f();

  return 0;
}

#include <iostream>

#include <functional>

#include <memory>

using namespace std;

class A

{

public:

~A()

{

cout << "object freed!" << endl;

}

std::function<void()> someCallback;

void f()

{

if(someCallback)

{

someCallback();

}

int someProperty = 100;

};

int main()

{

shared_ptr<A> obj1, obj2;

obj1 = make_shared<A>();

obj2 = make_shared<A>();

obj1->someCallback = [] {

cout << "obj1 callback called!" << endl;

};

obj1->f();

obj2->someCallback = [obj2] {

cout << "obj2 callback called: " << obj2->someProperty << endl;

};

obj2->f();

return 0;

}

Obj2 essentially is the same as obj1 right? All we do is add a little printout of obj2′s someProperty value from within the lambda… and of course, C++11 dictates that we must explicitly declare all external objects that we access within our lambda (a capture list).

So what is the output then?

obj1 callback called!
obj2 callback called: 100
object freed!

Exited with status code: 0

obj1 callback called!

obj2 callback called: 100

object freed!

Exited with status code: 0

Note the alarming lack of “object freed” being displayed for obj2. That’s right, the above code memory-leaks obj2. You see, obj2 owns someCallback, and someCallback owns obj2 via a capture. This is a retain cycle and it permanently causes obj2 to be “forever retained” and therefore leaks. Once a retain cycle is made, you can consider things to be more-or-less, too late.

This type of bug is a dangerous silent killer of the night. Why? Because it can be so easy to miss when developing. Objective-C and swift have evolved over time to add compiler and IDE warnings about this kind of “retain cycle” problem. C++11 assumes that you just wouldn’t write this type of code. But when porting code over from Objective-C, this kind of code can pop up ALL the time.

So what do we do?

Solution 1 :: The ObjC way

Well Objective-C handles this problem by introducing the concept of “weak pointers”. These are special pointers that ensure that they do not own their reference, they only keep track of whether or not it is allocated (if any shared pointers to it exist). Luckily, C++11 brings the weak pointer to us as well. A solution that uses them looks like so:

  auto weakObj2 = weak_ptr<A>(obj2);
  obj2->someCallback = [weakObj2] {
    cout << "obj2 callback called: " << weakObj2.lock()->someProperty << endl;
  };

  obj2->f();

auto weakObj2 = weak_ptr<A>(obj2);

obj2->someCallback = [weakObj2] {

cout << "obj2 callback called: " << weakObj2.lock()->someProperty << endl;

};

obj2->f();

You’ll note now that running this code will solve the leak. It’s also important to note that, while this is an ugly solution, this is/was the only way to solve a retain cycle of this nature in Objective-C.

However, for this explicitly unique case, in C++, there’s another way.

Solution 2 :: The tempting C++ way

What if I told you there was a way to solve the original problem by adding one character to the original code? Here it is:

  obj2->someCallback = [&obj2] {
    cout << "obj2 callback called: " << obj2->someProperty << endl;
  };

  obj2->f();

obj2->someCallback = [&obj2] {

cout << "obj2 callback called: " << obj2->someProperty << endl;

};

obj2->f();

See that little ‘&’ added before obj2 in the capture? This allows us to capture a reference to the smart pointer and not a value. Why does this matter? Well when a smart_pointer value is copied, it essentially increments its retain count of the object in question, and when it finally is destroyed by going out of scope, it’ll decrement the retain count, restoring the natural balance to the universe. However, as I mentioned before, a retain cycle prevents the retain count from ever reaching zero in this case. A reference to obj2′s smart pointer value allows us to forego the copy-increment mechanism and essentially stop obj2 from being over-retained. Its essentially like taking a pointer to a pointer….

However, this solution can be dangerous, as it assumes the coder understands the rammifcations of doing things like capturing a reference to a stack variable, an argument, or anything that may be “dead” by the time the callback is ran. Almost always, this “solution” may give unintended consequences.

The “correct” way

Here’s where things get somewhat opinionated. The “correct” way to handle this really depends on the situation at hand. I suggest that if you a working with a team, you might want to use the safer and more verbose weak pointer method since it is more explicit in its intentions and helps explain why strong ownership could be a problem. Furthermore, if you are working in a smaller system where shared pointership is not necessary, you might want to avoid using shared pointers altogether (assuming you fully understand the ownership semantics of the objects being used in the captures).

Either way, the golden rule is to understand the side effects of the code you are writing and when using shared pointers, keep that little retain cycle demon in the back of your mind… always

A taste of the game’s intro

Leave a reply

Parallax Occlusion Mapping – Advances in shader land

Leave a reply

Today was cool… at least the first part of it was, before I spend 5 hours tracking down a bug in some of the game code.

I stumbled onto a pretty cool little article geared towards a very awesome technique called “parallax occlusion mapping”. This is essentially bump mapping with some extra samples taken at each pixel to provide a realistic depth (or parallax) effect to the surface. This provides the illusion of realistic surface detail all while shading a single polygon. The article was somewhat dated and targeted for Direct3D instead of OpenGL, but I was able to port it over to GLSL 150 for OpenGL 3.2.

Here’s the original article:
http://www.gamedev.net/page/resources/_/technical/graphics-programming-and-theory/a-closer-look-at-parallax-occlusion-mapping-r3262

And here’s my port of the shader code for anyone who is interested.

in highp vec4 position;
in mediump vec3 normal;
in mediump vec2 texcoord0;
in mediump vec3 tangent;

out mediump vec2 va_texcoord;

out mediump vec3 va_eye;
out mediump vec3 va_normal2;
out mediump vec3 va_light;

uniform mat4 modelViewProjectionMatrix;
uniform mediump mat4 modelViewMatrix;
uniform mat4 modelMatrix;
uniform mat4 modelMatrixInverse;
uniform mediump vec2 textureScale;
uniform mediump vec2 bumpScale;
uniform mediump mat3 normalMatrix;
uniform float shadowBiasFactor;
uniform vec3 cameraPosition;

uniform bool doNormalize;

#define LIGHT    4

struct Light
{
  mediump vec4 worldPosition;
};

#ifdef GL_ES
highp mat3 transpose(in highp mat3 inMatrix) 
{
  highp vec3 i0 = inMatrix[0];
  highp vec3 i1 = inMatrix[1];
  highp vec3 i2 = inMatrix[2];
  //highp vec4 i3 = inMatrix[3];

  highp mat3 outMatrix = mat3(
                 vec3(i0.x, i1.x, i2.x),
                 vec3(i0.y, i1.y, i2.y),
                 vec3(i0.z, i1.z, i2.z)
                 );
  return outMatrix;
}
#endif

uniform Light lights[8];

void main()
{
  vec3 P = (modelMatrix * position).xyz;
  vec3 N = normal;
  vec3 E = P - cameraPosition;
  vec3 L = -lights[LIGHT].worldPosition.xyz - P;

  //Compute transformed normal
  vec3 eyeNormal = normalize(normalMatrix * normal);

  //Pass transformed texcoord.
  va_texcoord = texcoord0*textureScale;

  vec4 nNormal = vec4(normalize(normal), 0.0);
  vec4 nTangent = vec4(normalize(tangent), 0.0);
  vec4 nBinormal = vec4(cross(nNormal.xyz, nTangent.xyz), 0.0);
  mat3 tangentToWorldSpace;
  tangentToWorldSpace[0] = (modelMatrix * nTangent).xyz;
  tangentToWorldSpace[1] = (modelMatrix * nBinormal).xyz;
  tangentToWorldSpace[2] = (modelMatrix * nNormal).xyz;

  mat3 worldToTangentSpace = transpose(tangentToWorldSpace);

  va_eye = E * worldToTangentSpace;
  va_normal2 = N * worldToTangentSpace;
  va_light = L * worldToTangentSpace;

  //Pass GL-transformed to vertex down pipeline
  gl_Position = modelViewProjectionMatrix * position;
}

in highp vec4 position;

in mediump vec3 normal;

in mediump vec2 texcoord0;

in mediump vec3 tangent;

out mediump vec2 va_texcoord;

out mediump vec3 va_eye;

out mediump vec3 va_normal2;

out mediump vec3 va_light;

uniform mat4 modelViewProjectionMatrix;

uniform mediump mat4 modelViewMatrix;

uniform mat4 modelMatrix;

uniform mat4 modelMatrixInverse;

uniform mediump vec2 textureScale;

uniform mediump vec2 bumpScale;

uniform mediump mat3 normalMatrix;

uniform float shadowBiasFactor;

uniform vec3 cameraPosition;

uniform bool doNormalize;

#define LIGHT 4

struct Light

{

mediump vec4 worldPosition;

};

#ifdef GL_ES

highp mat3 transpose(in highp mat3 inMatrix)

{

highp vec3 i0 = inMatrix[0];

highp vec3 i1 = inMatrix[1];

highp vec3 i2 = inMatrix[2];

//highp vec4 i3 = inMatrix[3];

highp mat3 outMatrix = mat3(

vec3(i0.x, i1.x, i2.x),

vec3(i0.y, i1.y, i2.y),

vec3(i0.z, i1.z, i2.z)

);

return outMatrix;

}

#endif

uniform Light lights[8];

void main()

{

vec3 P = (modelMatrix * position).xyz;

vec3 N = normal;

vec3 E = P - cameraPosition;

vec3 L = -lights[LIGHT].worldPosition.xyz - P;

//Compute transformed normal

vec3 eyeNormal = normalize(normalMatrix * normal);

//Pass transformed texcoord.

va_texcoord = texcoord0*textureScale;

vec4 nNormal = vec4(normalize(normal), 0.0);

vec4 nTangent = vec4(normalize(tangent), 0.0);

vec4 nBinormal = vec4(cross(nNormal.xyz, nTangent.xyz), 0.0);

mat3 tangentToWorldSpace;

tangentToWorldSpace[0] = (modelMatrix * nTangent).xyz;

tangentToWorldSpace[1] = (modelMatrix * nBinormal).xyz;

tangentToWorldSpace[2] = (modelMatrix * nNormal).xyz;

mat3 worldToTangentSpace = transpose(tangentToWorldSpace);

va_eye = E * worldToTangentSpace;

va_normal2 = N * worldToTangentSpace;

va_light = L * worldToTangentSpace;

//Pass GL-transformed to vertex down pipeline

gl_Position = modelViewProjectionMatrix * position;

}

#ifdef GL_ES
#extension GL_OES_standard_derivatives : require
#extension GL_EXT_shader_texture_lod : require
#endif

precision mediump float; 

in mediump vec2 va_texcoord;
in mediump vec4 ec_pos;

in mediump vec3 va_eye;
in mediump vec3 va_normal2;
in mediump vec3 va_light;

out vec4 fragColor;

#define LIGHT    4

//Prototypes
vec4 computeLight(in mediump vec3 normal, in mediump vec4 ecPosition, in lowp float alphaFade, out lowp vec4 otherSideColor, out lowp vec4 secondaryHighlight);

//Lights
struct Light
{
  mediump vec4 worldPosition;
};

uniform vec3 cameraPosition;
uniform Light lights[8];
uniform vec4 lightModelProductSceneColor;
uniform lowp sampler2D texture0;
uniform sampler2D bumpTexture;
uniform sampler2D heightTexture;

//uniform bool lightingEnabled;

const float fHeightMapScale = 0.02;
const int nMaxSamples = 32;
const int nMinSamples = 8;

void main()
{
  // Calculate the geometric surface normal vector, the vector from
  // the viewer to the fragment, and the vector from the fragment
  // to the light.
  vec3 N = normalize(va_normal2);
  vec3 E = normalize(va_eye);
  vec3 L = normalize(va_light);

  float fParallaxLimit = -length( va_eye.xy ) / va_eye.z;
  fParallaxLimit *= -fHeightMapScale;

  vec2 vOffsetDir = normalize(va_eye.xy);
  vec2 vMaxOffset = vOffsetDir * fParallaxLimit;

  int nNumSamples = int(mix(float(nMaxSamples), float(nMinSamples), dot(E, N)));
  float fStepSize = 1.0 / float(nNumSamples);

  vec2 dx = dFdx(va_texcoord);
  vec2 dy = dFdy(va_texcoord);

  float fCurrRayHeight = 1.0;
  vec2 vCurrOffset = vec2(0.0);
  vec2 vLastOffset = vec2(0.0);

  float fLastSampledHeight = 1.0;
  float fCurrSampledHeight = 1.0;

  int nCurrSample = 0;

  while(nCurrSample < nNumSamples)
  {
    fCurrSampledHeight = textureGrad(heightTexture, va_texcoord+vCurrOffset, dx, dy).r;
    if(fCurrSampledHeight > fCurrRayHeight)
    {
      float delta1 = fCurrSampledHeight - fCurrRayHeight;
      float delta2 = ( fCurrRayHeight + fStepSize ) - fLastSampledHeight;

      float ratio = delta1/(delta1+delta2);

      vCurrOffset = (ratio) * vLastOffset + (1.0-ratio) * vCurrOffset;

      nCurrSample = nNumSamples + 1;
    }
    else
    {
      nCurrSample++;

      fCurrRayHeight -= fStepSize;

      vLastOffset = vCurrOffset;
      vCurrOffset += fStepSize * vMaxOffset;

      fLastSampledHeight = fCurrSampledHeight;
    }
  }

  vec2 vFinalCoords = va_texcoord + vCurrOffset;
  vec4 vFinalNormal = texture(bumpTexture, va_texcoord + vCurrOffset);
  lowp vec4 vFinalColor = texture(texture0, vFinalCoords); //vec4(1.0);

  vFinalNormal = vFinalNormal * 2.0 - 1.0;

  vec3 vAmbient = vFinalColor.rgb * 0.1;
  vec3 vDiffuse = vFinalColor.rgb * max( 0.0, dot( L, vFinalNormal.xyz ) ) * 0.5;

  vFinalColor.rgb = vAmbient + vDiffuse;

  fragColor = vFinalColor;
}

100

101

102

103

104

105

106

107

108

109

110

#ifdef GL_ES

#extension GL_OES_standard_derivatives : require

#extension GL_EXT_shader_texture_lod : require

#endif

precision mediump float;

in mediump vec2 va_texcoord;

in mediump vec4 ec_pos;

in mediump vec3 va_eye;

in mediump vec3 va_normal2;

in mediump vec3 va_light;

out vec4 fragColor;

#define LIGHT 4

//Prototypes

vec4 computeLight(in mediump vec3 normal, in mediump vec4 ecPosition, in lowp float alphaFade, out lowp vec4 otherSideColor, out lowp vec4 secondaryHighlight);

//Lights

struct Light

{

mediump vec4 worldPosition;

};

uniform vec3 cameraPosition;

uniform Light lights[8];

uniform vec4 lightModelProductSceneColor;

uniform lowp sampler2D texture0;

uniform sampler2D bumpTexture;

uniform sampler2D heightTexture;

//uniform bool lightingEnabled;

const float fHeightMapScale = 0.02;

const int nMaxSamples = 32;

const int nMinSamples = 8;

void main()

{

// Calculate the geometric surface normal vector, the vector from

// the viewer to the fragment, and the vector from the fragment

// to the light.

vec3 N = normalize(va_normal2);

vec3 E = normalize(va_eye);

vec3 L = normalize(va_light);

float fParallaxLimit = -length( va_eye.xy ) / va_eye.z;

fParallaxLimit *= -fHeightMapScale;

vec2 vOffsetDir = normalize(va_eye.xy);

vec2 vMaxOffset = vOffsetDir * fParallaxLimit;

int nNumSamples = int(mix(float(nMaxSamples), float(nMinSamples), dot(E, N)));

float fStepSize = 1.0 / float(nNumSamples);

vec2 dx = dFdx(va_texcoord);

vec2 dy = dFdy(va_texcoord);

float fCurrRayHeight = 1.0;

vec2 vCurrOffset = vec2(0.0);

vec2 vLastOffset = vec2(0.0);

float fLastSampledHeight = 1.0;

float fCurrSampledHeight = 1.0;

int nCurrSample = 0;

while(nCurrSample < nNumSamples)

{

fCurrSampledHeight = textureGrad(heightTexture, va_texcoord+vCurrOffset, dx, dy).r;

if(fCurrSampledHeight > fCurrRayHeight)

{

float delta1 = fCurrSampledHeight - fCurrRayHeight;

float delta2 = ( fCurrRayHeight + fStepSize ) - fLastSampledHeight;

float ratio = delta1/(delta1+delta2);

vCurrOffset = (ratio) * vLastOffset + (1.0-ratio) * vCurrOffset;

nCurrSample = nNumSamples + 1;

}

else

{

nCurrSample++;

fCurrRayHeight -= fStepSize;

vLastOffset = vCurrOffset;

vCurrOffset += fStepSize * vMaxOffset;

fLastSampledHeight = fCurrSampledHeight;

}

vec2 vFinalCoords = va_texcoord + vCurrOffset;

vec4 vFinalNormal = texture(bumpTexture, va_texcoord + vCurrOffset);

lowp vec4 vFinalColor = texture(texture0, vFinalCoords); //vec4(1.0);

vFinalNormal = vFinalNormal * 2.0 - 1.0;

vec3 vAmbient = vFinalColor.rgb * 0.1;

vec3 vDiffuse = vFinalColor.rgb * max( 0.0, dot( L, vFinalNormal.xyz ) ) * 0.5;

vFinalColor.rgb = vAmbient + vDiffuse;

fragColor = vFinalColor;

}

Some of this code is “massaged” by Verto Studio’s code converter when ran on mobile. It’s definitely a pretty cool effect! It requires 3 texture maps to work currently, a standard diffuse (color texture) map, a normal map, and a displacement or height map. Lucky for me, there’s a sick little program called “crazy bump” for mac that can generate both normal maps and displacement maps from any standard diffuse texture map!

WebGL demo

For those who want to see the shader effect in action, I got a WebGL demo which runs on chrome and safari (firefox and IE don’t work).

It’s an expensive effect however, so I’m not sure yet if I can work it in for Driveby Gangster. Either way, it’ll definitely be a nice little addition to the shader arsenal. If I do actually put it into use, I hope to eventually add self-shadowing effects as well.

What I’ve been doing

Leave a reply

Tons of stuff has been accomplished as of late. I’ll just list a few of em here.

I programmed a cool “breaking glass” shader for the street scene shop window so that when you shoot at it, it breaks
I made a “spark” particle shader for when you shoot at metal, and also programmed separate bullethole decals to appear depending on the surface you strike.
I bit the bullet and grabbed maya for finishing up some of the character animation stuff including animations for aiming the gun and getting hit by bullets
I started on and nearly finished a very awesome “beach” scene for the game. This broke one of my cardinal rules about keeping the stuff simple, but I’m accepting the risk because a game with the same level over and over again is a pretty bad user experience.

I apologize again for the lack of quality updates on here, but I’ve definitely been prioritizing working on and finishing the game over keeping this blog up to date. It’s a tradeoff I guess.

For the beach scene I modeled a cool sailboat for it entirely from scratch using verto. I have big plans for this sailboat that I’m going to keep secret for now.

One thing that I’m really excited about was a small effort that took me two days. I managed to compile the entire C++ code base to JS using emscripten. This means that I can display simple verto studio scenes in the web browser! A quick demo of this, for people running on webgl enabled browsers, should display below…

Parse Error

Michael L. Farrell's Game Dev Blog

Monthly Archives: December 2014

OpenGL point light shadows & Atomspheric FX hacks

Point Light Shadows

Light-Fog Interaction (Volume FX)

How to 3D sound in windows (hint: it’s not OpenAL)

Screenshot of the arbitrary time period

So much progress

Just screenshots today

C++11′s take on retain cycles

Example 1 :: All is well

Example 2 :: Not so much

Solution 1 :: The ObjC way

Solution 2 :: The tempting C++ way

The “correct” way

A taste of the game’s intro

Parallax Occlusion Mapping – Advances in shader land

WebGL demo

What I’ve been doing