Man Man was featured on an Underrated Games to Try in December list.
https://gameskeys.net/underrated-games-to-tryout-in-december-2020/
This entire game was programmed during a coding in C series on my youtube channel.
Disclaimer, I’m not a low-level network coding expert, so I’m sure there are easier/better architectures for how to handle this sort of thing. I’ll study them someday
To keep my mind off the world going to shit recently, I redid parts of my game’s networking code last night to add “sync” support to my packet transmissions (make sure a packet was received). Mini case study below.
Basically since UDP does not guarantee delivery of packets, packets can arrive late, never, out of order, or duplicated, you’re royally fucked if a certain important packet such as (the unit was split in two) arrives twice. So for packets that aren’t just player position, you have to handle the sync properly. Each packet starts with the same header
|
1 2 3 4 5 6 |
struct Header
{
uint16_t type; //state type
SeqType seq; //packet number (sequence)
SeqType ack;
}; |
The first step is to add seq and ack to all outgoing packets where seq is the packet number of this packet and ack is the highest number of any packet we’ve received back. For example, if the client is sending the 100th packet, seq will be 100. If the server receives this packet, upon it’s next update to this client, it’ll send a packet with the ack containing 100, so the client can know it was received. This is a super basic well known mechanism for reliability (TCP does something like this but considerably more complex)
So, the issue last night was, how the fuck do I handle a missed ack. Architecturally it got a bit messy. Basically my game works on a heartbeat type mechanism where the app is set into a certain state, (like a battle match), and the same packet type is sent over and over again from the client with updated data (the player positions). If they get lost, no big deal, because another one is coming right away. The receiver only considers the highest ordered seq packet, and disregards anything older than that.
For packets that must be received once and only once,
I had to start marking these critical packets as “sync” packets, meaning they are important and nothing else will be sent by the client until the server reports back that it got the sync packet, and that it successfully sync’ed it to every client connected to it.
The devil was completely in the details on doing this right. For example, for a sync packet I had to ensure the seq number was the same during all retransmissions, so they would be processed only once, etc etc.
Key parts of this in the client are below (the server was considerably more complex, since it has to sync to multiple clients where the seqs/acks are different for each one)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
class Client
{
public:
Client(const std::string &serverAddressStr, const std::string &playerName) { ... }
~Client() { ... }
void poll()
{
if(sendOnPoll.load())
{
sendState();
sendOnPoll = false;
}
while(connection->isDataAvailable())
{
size_t bytesRead = 0;
if(connection->receive(packetData, MaxPacketSize, &bytesRead))
{
auto sender = connection->getLastReceivedFromAddress();
auto state = ServerState::stateFromPacketData(packetData, bytesRead, sender);
uint32_t ack = state->getPacketAck();
if(state->isStale())
{
//ignore out-of-order older (stale) packets
continue;
}
if(syncState)
{
if(syncSeq && ack >= syncSeq)
{
currentState->setSync(false);
currentState->syncComplete();
currentState->setAdvanceSeqBeforeSend(true);
syncState = false;
}
}
switch(state->getType()) { ... }
}
}
//...
}
///Send our state to the server
void sendState()
{
currentState->sendToConnection(connection.get(), serverAddress.get());
if(syncState)
{
//make note of the per-client seq for this packet sent to the server
syncSeq = currentState->getPacketSeq();
}
currentState->setAdvanceSeqBeforeSend(false);
sentPacketSeqs.push_front({ currentState->getPacketSeq(), vgl::System::system().globalTime() });
}
void setSyncState(bool b) { .... }
private:
//.....
vnet::UDPConnection::Pointer connection;
vnet::SocketAddress::Pointer serverAddress;
std::atomic_bool sendOnPoll;
bool syncState = false;
uint32_t syncSeq = 0;
}; |
..and the actual packet transmission code is below..
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
void ClientState::sendToConnection(vnet::UDPConnection *conn, vnet::SocketAddress *destAddress)
{
if(dataLen > 1024)
{
throw std::runtime_error("ClientState::sendToConnection() packet too large!");
}
assert(data != nullptr);
assert((StateType)type != StateType::None);
Header *hPtr = (Header *)data;
auto addr = conn->getFromAddress()->getAddr();
uint64_t ipPort = vnet::SocketAddress::ipPort(ntohl(addr->sin_addr.s_addr), ntohs(addr->sin_port));
SeqType &sendSeq = packetSeqSend[ipPort];
if(sync && advance)
{
sendSeq++;
}
if(!sync)
{
hPtr->seq = sendSeq++;
}
else
{
hPtr->seq = sendSeq;
}
auto destAddr = destAddress->getAddr();
uint64_t destIPPort = vnet::SocketAddress::ipPort(ntohl(destAddr->sin_addr.s_addr), ntohs(destAddr->sin_port));
hPtr->ack = packetSeqRecv[destIPPort];
conn->send(destAddress, data, dataLen);
} |
Currently, any ClientState (my packet base class) can set itself as a sync packet, once the Client encounters one of those, it’s in syncState mode (and retransmits the same packet over and over) until it’s resolved with an ack as shown above. In the higher level game code, if the client is in a sync state, I treat the game as if it’s paused (can’t move).
..and lastly, the pertinent sections of the actual UDPConnection class follows below. This is where my game actually touches the OS sockets API. Generally, this API is the lowest level of networking programming available to most applications.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
bool UDPConnection::send(SocketAddress *destAddress, void *data, size_t length)
{
if(sendto(socket, (const char *)data, length, 0, (sockaddr *)&destAddress->addr, sizeof(destAddress->addr)) < 0)
{
return false;
}
return true;
}
bool UDPConnection::receive(void *data, size_t dataLen, size_t *bytesRead)
{
sockaddr_in remoteAddr;
socklen_t addrlen = sizeof(remoteAddr);
memset(&remoteAddr, 0, sizeof(remoteAddr));
int64_t ret = recvfrom(socket, (char *)data, dataLen, 0, (sockaddr *)&remoteAddr, &addrlen);
if(ret >= 0)
{
if(!lastReceivedFromAddress)
{
lastReceivedFromAddress = make_shared<SocketAddress>(&remoteAddr);
}
else
{
lastReceivedFromAddress->setAddr(&remoteAddr);
}
}
*bytesRead = ret;
return (ret >= 0);
} |
So.. I’ve been making a new game and have been posting to social media occasionally some of the graphical updates around these efforts. However, may return to this blog from time to time to post more interesting technical stuff.
So I kinda never mentioned that Driveby Gangster got greenlit on steam. And I may have neglected to mention that it was LAUNCHING TODAY! HOLY CRAP GET YOUR HANDS ON IT IMMEDIATELY!
More info and demo download available here
This last month, DirectX 12 launched alongside windows 10 so I decided to see what would happen if I ported my engine backend to run on something besides OpenGL. Because I’m crazy enough to attempt something like this, I dove into what limited documentation there was on D3D12 and went at it. I wanted to get a head start on some of the concepts that I know are coming when Vulkan launches later this year, and to get myself off of a strict dependency of OpenGL for my engine graphics. One crucial decision that I made early was to continue using just one shader language (GLSL) for even the D3D12 engine backend. I did this because Verto Studio files often contain embedded shader code that is written against the Verto Studio shader standard, and I wasn’t about to break that part of the platform. Luckily, the Angle project’s shader translator proved to be a perfect solution to auto-translate my GLSL code to HLSL and interoperate with the rest of my D3D12 layer quite nicely.
I had worked at this low of a level before and I was amazed at how hard I had to bust my ass to get performance that would first match, and eventually beat my windows NVIDIA OpenGL drivers in some cases. Whats more interesting, is that I had managed to double the framerate performances that I was getting on OpenGL on OSX, which really speaks to how bad the Apple OpenGL drivers really are in comparison to windows. Mechanisms such as pipeline state caching, frame latency planning, and large ahead-of-time heap allocations were all necessary to bring my performance up to par. There still are some hiccups but for the most part, I’m really pleased with what amounted from about a week and a half of messing around with D3D12.
Windows 10 demos are below. Keyboard shortcuts aren’t that great but namely arrow keys revolve the camera, and shift+alt combinations allow you to move forward, and vertically. These demos don’t push the bar graphically and they certainly come nowhere close to pushing D3D12 to what it can do, but if you just want something to run that uses DirectX 12, well here you go.
OpenGL equivalent demos are provided in the zip archive as well for comparison..
…so I made this…
So it’s out!
The mobile port of Driveby Gangster is available to the masses as a free-to-try download! Here is the link to download: https://appsto.re/us/3cNg7.i
If you have been wanting to try the game but do not own a Mac, now is your time to shine!
After putting roughly 6 months into this project, it seemed stupid not to go the extra mile and release this game for iOS if it was possible to be ran on it. So that’s what I spent the last few weeks doing, and I eventually reached success (I submitted it to the store for review this weekend). Disclaimer, much of this post was written using voice dictation so it might not flow as well as a typed document
Now even though mobile was in the back of my mind, this game was never really designed to run on mobile. The final desktop game took up roughly 1 GB of disk space and 1 gigabyte of RAM running which would slowly rise to 4 GB over time which I suspected and later confirmed to be caused by memory leaks.
To get the game to run on my iPhone 5s, The first thing I did was make some minor modifications to the graphics engine code base to support running OpenGL 3.2 code on an iOS device. The easiest way to do this, was to simply target the OpenGL ES 3.0 API. The differences between OpenGL 3, and GLES 3.0 are so minor, that this was the best bet that I had for an effortless port of the game. This sure beat the hell out of the alternative which was to laboriously rewrite my dozens of now-forked shaders in GLSL for OpenGL ES 2.0. The downside to this however, was that I basically was cutting off any iOS device that could not run OpenGL ES 3.0 (pre-iPhone 5s, pre iPad air). I still decided to go for it despite all this.
OpenGL ES 3.0 supported so many things that I needed for my game (such as multiple render targets, shadow maps, occlusion queries, etc) that the whole process worked out really well. Things that used to be near impossible to do on mobile OpenGL were now surprisingly easy to accomplish.
Still, there were some serious challenges that I had to overcome
Off the bat, the most serious and challenging problem presented itself almost immediately after running a few levels of the game on my iPhone: RAM consumption. Almost immediately, loading the first level resulted in memory warnings and the eventual memory crash caused by allocating too much RAM on the device (this equates to roughly 512 MB on an iPhone 5s). This almost made me cancel the idea immediately thinking that there’s no way the game could fit as designed on a mobile device. When I ran the instruments tool, A utility that comes with Xcode to help you trace allocations and such, I discovered that my sound effects alone were taking up well over 100 megs of RAM. I also discovered that the heap was growing at an alarming rate between level reloads resulting in the eventual memory crash. Other assets such as textures and animation data were taking up another hundred megabytes or more (GPU memory is not easily trackable on an iOS device so I could only speculate how much the standard high-resolution desktop textures were taking up).
I used the invaluable “generations” tool of the instruments utility to track down an eliminate my biggest causes of memory leaks. The funniest one I can remember is the silent failure of the OpenAL alDeleteBuffers call resulting in the leak of ALL sound effects used in the game, including the very large “radio music” buffer which held roughly 1+ minutes of music audio. Leaking this was wasteful on desktop, but downright devastating on mobile. Discovering that I had to dequeue the auduo buffer first, solved that issue. Other stupidities such as bad pointer casting and issues related to C++ 11 smart pointer retain cycles I counted for the other memory leaks.
Apart from the memory leaks, there was a very serious issue that almost crippled the release of the entire game on mobile. This one was not my fault, and it’s related to a arcane bug in the iOS OpenGL 3.0 driver and how it handles OpenGL occlusion queries. Leave it to me to always find GL driver bugs in just about every platform I target for my games. This particular bug caused repeated usage of occlusion queries to eventually crash the whole app inside the driver code. I frustratingly posted this on the Apple developer forums, and got one of the engineers (one of the awesome engineers) to track down the issue and provide a workaround for me which fix the issue. For those that are curious, the workaround was to run the occlusion queries without an active color buffer attached to the default frame buffer object, bypassing the issue that leads to the crash.
Fixing all these issues however, still would not of been enough to get the game to run on iOS
To get the game to work “fit” on my iPhone 5s and iOS in general, I had to make the following optimizations
I also added support for MFI game controllers, a supported API by iOS 7 and above to allow players with physical game pads to get the best gaming experience possible. This was not a big stretch considering the original game was designed for physical buttons in the first place.
After doing all of this crazy stuff, I finally got the game to run reliably without crashing on my iPhone 5s and iPad air.
Victory.
My game is now released on the mac app store!!! For only $4, you can
Store link is here and also at the top of the blog page.
Windows users, sorry for the delay. Keep voting on greenlight! It’s the only way!