Monday, June 18, 2012

New world rendering scheme

So I said I'd talk about how I reworked my Scene class (which manages the world and all objects in it) to allow a lot of flexibility in drawing. Previously, drawing the Scene was as simple as calling Scene::DrawScene(). This would draw the world, then call Draw() on all GameObjects. This was really simple and intuitive, and someone using my engine could get stuff on the screen quickly. Little is more frustrating than not being able to even get started with something because you can't figure out the basics. Unfortunately, this puts a lot of control out of the user's hands; once they call the function they're limited to what Scene is hardcoded to do.

New features in my engine often come out of want or need while I'm developing ProjectFPS, and this time was no different. I was looking to implement toon shading, and wanted to try the post-process version. In this version, you render normal and depth information to separate buffers, then render the scene normally. Finally, you use the normal and depth info to detect edges and add the black outlines. However, my Scene class didn't support this at all. While it was possible to put a post-process effect on the final color image, there was no way I was getting normal or depth info.

There are a couple solutions to this problem. The simplest method is to modify Scene code, maybe through adding a bool called m_bToonShading or something that specifies whether you want your scene toon-shaded. But now I'm bringing game-specific stuff into a general purpose engine, and that's no good. Plus, you're still stuck if you wanted a different post-process effect, or even a different toon-shading algorithm. And if you try to add support for multiple special effects this way, the Scene code could get messy fast.

Another is to allow Scene::DrawScene() to be overloaded (i.e. make it a virtual function). This solution certainly gives you all the power over rendering you'd need. Doing so, however, would be very complicated. You're asking the user, who did not design nor program the engine, to delve into all the messy details of how Scene does what it does. As you can imagine, it is not a small class. Rendering a scene efficiently takes a lot of steps, and if the user forgets any when writing their own function, they get a black screen with little indication of what went wrong. And if they plan to support multiple special effects, they either have to overload the function multiple times (which is frustrating and time-consuming), or be faced with the messy code branching problems Solution 1 had.

Solution 3 is to tell Scene what kind of extra rendering information you want when rendering the Scene. The actual post-process part, however, is not done by Scene. So if you wanted color, normal, and depth info rendered, you might say:

m_Scene.DrawScene(SCENEDRAW_COLOR | SCENEDRAW_NORMALS | SCENEDRAW_DEPTH);

This is a fairly good solution. It gives you info you need without assuming what you intend to do with it, giving the developer flexibility without making things needlessly complicated. This is not perfect, however. Maybe you wanted to exclude transparent objects from showing up in the normal buffer. Maybe you wanted linearly-distributed depth, as opposed to standard non-linear. Maybe you wanted your own special type of data that Scene doesn't natively support. Still, this is a great step in the right direction.

The solution I settled on was to give high-level rendering control to the Camera class. You call Scene::DrawScene() as normal. Scene then iterates over all currently enabled cameras and calls Camera::DrawScene() on them, a virtual function that can be overloaded in derived camera classes. Scene then provides a couple of high-level functions (such as DrawLevelGeometry() or DrawOpaqueObjects()) that Camera can call at will. In between, Camera can switch render targets or techniques or set parameters or whatever it needs to do. This lets the developer only render what they want, how they want, while sparing them from all the gritty details.

To give the user further control, Camera can set tags that have special meaning to the application. For example, you could define a tag that, when set, excludes all objects that are not players. You set this tag to the camera and call Scene::DrawOpaqueObjects(). Scene will then call GameObject::DrawOpaque() on all GameObjects. The GameObjects will check for the tag, and react appropriately.

If you need to switch the special effect you're using during gameplay, you could configure the camera, or you could just derive a different class and swap cameras instead. Doing the latter would be much less daunting a task than overloading Scene::DrawScene() multiple times would be. Plus, having Scene just pass control to all enabled cameras makes doing effects like picture-in-picture and "arena cam" simple to implement.

The base Camera class provides a default implementation, so for those who just want to get up and running, they can just call Scene::DrawScene() like before. In this way, the simplicity and intuitiveness of the original design is maintained.

This solution is still not perfect. Currently, my main issue is a lack of precise control over how exactly level geometry is rendered. I can swap techniques and set parameters, but that requires knowledge of all shaders used by level geometry. And if I set a parameter on a shader, it will apply to all level geometry that uses that shader. But maybe that's not what I want. Maybe I want wall A to have a different parameter than wall B. Level geometry can't check tags, either.

I intend to solve these problems, but I'm holding off until the level format I use is more finalized and robust. Currently, I'm using a level editor called GILES that comes with courses at a site called the Game Institute. This editor has a number of limitations, however, that prevent me from fully committing to it. They're working on a new level editor that's going to be released alongside their Carbon game engine. I expect huge data format changes, so when that comes out, I'm going to take a look at it and revisit these issues.

Saturday, May 12, 2012

Finally, some screenshots.

A lot has happened in these past five months for ProjectFPS. I'm not even sure where to begin recounting the changes. I apologize for not giving more updates as I've done it. On the bright side, I am now going to actually show some screenshots. So I'm going to show those first, then rattle off all the changes to the engine itself that I can remember off the top of my head. Note that none of the art is final - it's just one of many things on my to-do list.


You'll notice that I've gone for a toon-shaded look. I feel this gives the game a nice visual appeal without straining my (non-existent) artistic skills. It also sets a less serious tone, which I like; I may have mentioned this before, but this game is going to be designed such that people of various skill levels can all play together in a more friendly sort of competition. As such, there's less focus on hardcore competition and more focus on just having a good time. Going for a non-realistic look helps set that tone.




The gun you're currently looking at is called the Flame Pillar Gun. It shoots projectiles that burst into a jet of flame upon contact with the level. This jet lasts for a few seconds before disappearing. It's good for covering your escape or trapping your enemies. Skilled players can also use it to hit areas from otherwise impossible angles.




This is the Magnet Gun. Again, the gun model is temporary - I'm actually thinking of replacing it with a glove. Anyway, the left mouse button shoots out that big spiky ball you see there. You can then use the right mouse button to push or pull it towards you (it toggles between the two). The color of the glow indicates which way it's going - blue is pull, red is push. The balls will bounce when they collide with something.




You can have multiple of those spiky balls out at once, and they will all get pushed or pulled simultaneously. That's when the real fun starts, especially when two or more players get in a Magnet Gun fight.

There's also the Rocket Launcher, but since I'm in the middle of some big engine-side changes to the way objects are rendered, that gun's not showing up correctly.




I also got a cool menu system going. This runs on top of a GUI system I developed for the engine.

As for the engine-side changes... where do I start...

- The aforementioned GUI system. It's extendable and skinnable, too!
- Major overhaul to the way the scene is rendered. This topic is a post by itself, and will come later.
- Dynamic lighting! Supports directional, point, and spot lights. No shadows yet, though. It's amazing how much even a little lighting can improve your game's look.
- Improved netcode! I overhauled this, too. It still uses the NetRep component, and the concepts mentioned in my post back in August hold, but now you can tell NetRep exactly what variables you'd like to send over the network. In addition, the whole thing is delta compressed. This means that the server only sends what changed, cutting down on bandwidth requirements.
- Shaders! I've written a lot of code designed to make using the programmable pipeline as easy as possible. You can use semantics recognized by the engine to tag variables, and they will automatically be filled with the data you need.

There's a lot of other stuff too, but that's the most exciting to me, anyways. So where am I going from here?

- More maps and weapons, and improved art, of course. As far as this goes, the engine is in enough of a finished state to make some good progress here.
- Lightmap support. Dynamic lights are expensive, so it's better to go static wherever you can.
- Shadowing support.
- Physics component overhaul. I don't mean Source engine style physics, but reworking it such that objects have easier access to collision information and can set up complex collision volumes. It also needs to play nicer with my Transform component.

And I'll try to be more punctual with my updates next time.

Wednesday, December 21, 2011

More project progress

I can't really think of a particular topic to talk about with my engine here, so I'll just give you another progress update.

- Improved the netcode a lot. Not surprisingly, using only two computers to test netcode leaves a lot of bugs hidden. I gathered five of my friends to test out the game, and there were crashes and desync everywhere in a game that worked perfectly with only two people. I think I've worked those bugs out, but I won't be able to test that for a while.

- Animation! I've got an animation component in there now, and the player can play different animations on different parts of the body. Very useful for when you want to play a firing animation while running. Also, the player's torso now rotates based on how far they're looking up (previously, the whole mesh rotated).

- Rocket launcher! The mesh is incomplete at best, but it's there, and the rocket launcher itself is fully functional. The cool thing is that stuff like rocket jumping worked without any code explicitly for it.

- Visual effects! Head bob, camera shake, weapon bob, cool explosion effect!

- Other stuff I can't think of!

While it was depressing to see the netcode I was so proud of fall on its face with larger player counts, my friends still were having a lot of fun, despite the map being a test map and only a rocket launcher available. This is a good sign. Other than fixing bugs, of course, my next main goal is to get more weapons and some real maps in there. Then I can focus on improving the visuals. My engine could seriously use some lighting, and the player/weapon models I made, frankly, suck. So I still have a lot of work to do.

Thursday, October 6, 2011

Working with the 3DS Max SDK

I haven't had too much time to work on ProjectFPS specifically lately, due to work required by other projects. I did actually get around to writing a 3DS Max exporter for meshes and animations, however. I've taken the approach of writing the mesh data + vertex bone weights out to one file, with extension .msh, and skeleton data + animations out to another file, with extension .skl. 3DS Max, like a lot of 3D modeling programs, exports only one animation at a time, so I can't use the .skl file directly. Instead, I concatenate all animations for the skeleton into one, export that to .skl, and then put that file through an animation splitter I have to write, which will break it up into smaller animations and write it out to a separate file, which I will give extension .pfs. The mesh file, when loaded, will reference a .pfs skeleton that will be loaded along with it. This allows me to use different skins on the same skeleton, and to only have to do the animation splitting process once for every .skl file I create. Making assets isn't really my thing, so having a system like this is a big boon.

Now for a bit of a rant, if I may.

Once I figured it out (with a lot of help from video tutorials) 3DS Max itself was easy enough to use to create a rigged test mesh. The SDK, however, was much more painful to work with. It seemed like I had to grapple with it to get the data I needed. I guess, to an extent, that's unavoidable when working with a system that's written to support anything under the sun (the feature set, level of abstraction, and modularity is mind-bogglingly extensive). So it's natural that my needs are very specific, and represent only a small subset of what 3DS Max needs to do. However, considering how many games require writing an exporter (if you're writing your own asset pipeline, you HAVE to write an exporter for your modeler of choice), and the establishment of 3DS Max as an industry standard for game 3D modeling, I would expect more effort to be put into supporting developers who have to write them.

The main issue I had was lack of documentation. Yes, every function declaration was commented, but I would read the description for a function and still have no idea what it did. I punch some of the words I didn't understand into the documentation. Sometimes I'd get results, sometimes I'd just get reverted back to the C++ file-generated documentation. When I did get results, I often didn't understand what THAT page was talking about, either, or it didn't contain information relevant to my purposes. So I'm being lead on a wild goose chase all over the documentation just to find out what the heck that one function even did. My Google-fu also failed me in this regard, which puzzles me; you'd think that help would be all over the internet for something like this.

Then there were the gotchas and surprises. Surprises are never good when you're trying to work with an API. Often times I would write a function only to find that the API worked completely differently than I expected, and then I had to significantly rework my exporter to accommodate (I'm looking at you, keyframe exporting). Other times, it did things that I didn't expect with my data (I was stumped as to why my mesh, which consisted of two boxes, had so many faces and vertices until I found out it was converting my bones to triangle meshes too).

Then the random crashes and unexpected behavior. My program kept crashing whenever I tried to allocate memory, and crashed during random fwrite calls (which I later found allocate memory). The program cited heap corruption, but nothing else. Turns out the problem is that when building in Debug, you're using the debug C library while 3DS Max still uses the release version; these two allocate and free memory differently. The solution is to use a special Hybrid build, which uses the release library. The only reason I know that is because it was in a PDF found through some very lucky Google-fu, and that PDF was not written by Autodesk. For something as critical and hard to trace down as this, the SDK should have made this gotcha abundantly clear to me, but it didn't. Working with the Win32 interface in 3DS Max is also causing all sorts of problems for me, and to this day I have not been able to fix them. My money's on the existence of a similar gotcha. Sure, all the stuff in this and the previous paragraph make sense now, but to the programmer just getting familiar with the API, these kinds of things need to be properly explained.

The main saving grace I had in this was the two DevTV: Introduction to 3DS Max Programming videos. These videos walked you through the steps of creating a basic ASCII exporter. These were invaluable in the creation of my exporter, and I don't think I would have been able to do it without them. My final exporter design closely mimics the one in these videos. They walk you through step-by-step the extraction of the needed data, the concepts underlying the SDK, and any associated gotchas. The problems I had with coding my exporter started only once I wanted to do things the ASCII one didn't. I think the SDK's usability would benefit greatly from a larger pool of well-documented example plugins. XNA took this approach with its documentation, and it shows - it's very simple to figure out and adapt to your specific needs. The SDK's samples folder does contain a more comprehensive ASCII exporter than the one in the videos, but it's sparsely documented and hard to follow.

In short, the SDK is too hard to figure out for me to consider working extensively with it. I don't think the problem is due to bad design decisions or hack-and-slashed functionality; far from it. I simply feel that the developer is left hanging to figure things out on his own far too often.

Wednesday, August 31, 2011

Getting game objects working over a network

First, a quick progress update on the project:
- I've got most of the game logic/high-level network code finished. There's still a kink or two I have to work out, and I need to implement a map rotation system, but everything that's needed for a good ol' deathmatch game is there. From here, it's mostly improving the look and feel of the game, and getting new weapons, maps, and stuff in.
- Still nothing in the way of animation, but I intend to change that soon.
- I've decided to give the game a wacky sort of theme. This achieves two goals. First, it makes the game fun to play for people of varying skill levels, one of my original goals in making this game. Second, it makes my game not "just another deathmatch shooter", and makes it different from all the Call of Duties, Halos, and Unreal Tournaments out there.

I'd like to describe my system for managing game object replication. In making any kind of network game, you'll need some way to instruct the client to create objects, destroy objects, and change the current state of objects. We also need to make sure that only certain game objects are replicated. For example, there's no need for the server to spawn little particle effects and replicate that to the client if it's not gameplay-critical. And the process of network replication should be as simple as possible. To solve all these problems, I created a component called NetRep (short for Network Replication).

Any object that can be replicated over the network will need a NetRep component. This component takes the current state of the object as input and produces what is called a "proxy" version of the object. A proxy is a stripped down version of the object, containing just enough information that the client can mimic the server's version. For example, when replicating a player, you can get away with just transmitting his position, velocity, look direction, health, current weapon, and a couple of state flags. That sounds like a lot, but this will be much smaller than the full blown Player! Since the client will only see these other players from the outside, you don't need to transmit everything.

If you add an object server-side to the Scene class, which I use for object management, it will check to see if the object has a NetRep component. If it does, it will automatically inform the server that this object needs to be replicated. The server will then tell the NetRep component to generate a GOCreateInfo structure. This is a superset of the proxy and contains additional information needed to recreate the object from scratch on the client, such as the object's class name. The server then sends all clients a MSG_CREATEOBJECT message, including the creation info. From then on, the server will continue to send clients a proxy of the object at regular intervals (I've found 50 ms to work well). Likewise, when the object is removed from the scene, a MSG_DELETEOBJECT message is sent to all clients.

NetRep contains a couple of default input fields, such as position, velocity, and health, but it also contains a user data field where you can specify your own custom state data. You can also set which input fields you're actually using, so if your object doesn't need its velocity replicated, you can leave it out of the proxy and reduce bandwidth usage.

Once the client receives a proxy of an object, it grabs the NetRep component of that object and calls ApplyProxy() on it. Depending on how it's configured, NetRep will then perform default proxy application, call a custom function specified by the object, or both.

This whole system allows objects to not have to worry about netcode (for the most part) when doing their stuff. For example, if my rocket launcher spawns a rocket, I don't need to keep track of the rocket or inform the server or anything. It's all done for me automatically. I just add it to the scene like it was a single-player game.

There is one aspect of netcode, however, that objects need to be aware of. They must be aware of which side of the network they exist on: server, client, or both (a listen server). This will affect their behavior. For example, firing a gun means very different things between client and server. Both will need to cast a ray, but the server will use it to determine damage, while the client is only concerned with sounds and pretty effects, like sparks and bullet holes. This is simply achieved by including a little field in the object that says which side of the network it's on. This field is set by Scene when the object is added to it.

Tuesday, July 19, 2011

Wow, a month and a half since my last update. It hasn't been due to lack of work on the game, though.

I've been putting almost all of my effort into the code and almost none into the assets, so I still don't have much to show you. I can tell you where I currently am in the project, though

Up until recently, most of my work has been on a generalized engine that I can hopefully use for a variety of 3D games. Granted, it's not going to be the next Unreal engine, but most of the big stuff (object management, low-level network code, collision system, etc.) should be usable across a number of different games. At some point in the future, I'll have to see just how good a job I did. =P

Anyway, now I'm mostly working on stuff relating to a (still generalized) multiplayer FPS. This means network code, and a lot of it. My current goal is to get the game to be awesome over LAN, and at least decent over the Internet. The latter is actually the harder part, since now you have to deal with higher latencies and all sorts of bad connections. To that end, you have to put in a lot of code to smooth things out. The two big ones are interpolation and prediction.

Interpolation means smoothing out an object's transition between two states. There's not nearly enough bandwidth to send an object's state over the network every frame. Currently, I only send an object's state every 50 ms (that's 20 states per second). But if your game's running at 60 FPS, objects' movement will appear to be choppy. So you have to smooth it out. When a client receives an object's state, instead of immediately updating the object, it waits for the next state, then linearly moves between the two. Naturally, your code must also count on states never arriving, or arriving off-beat.

Prediction allows instant reaction to player input. You don't want to have to wait 200 ms after the player presses forward before they actually move forward. They want to move forward NOW. Prediction allows the player to see response to his actions before the server acknowledges it. This means that you're going to have times when the server and client disagree on what's happening. For example, the client may move forward normally when he presses forward, but the server comes up with a different figure because of that big ol' rocket that sent the player flying server-side. So you're going to need code that corrects prediction errors on the client based on states sent by the server. This correction code must also compensate for latency, since that will make the state the server sends reflect some point in the past.

So that and other high-level network stuff has been where most of my time has been going. I'm only now working on main game logic (like the rules of a Deathmatch game). This may seem like an awfully late point in the project to be implementing something like this, but it was necessary to achieve the kind of generalization I was looking for. What I mean by this is the separation (for the most part) of low-level network code, high-level network code, and game logic.

The low-level network code's only job is to send and receive messages, and maintain connections; it can therefore work with pretty much any game. The high-level network code's a little more specific - it contains code necessary to update the world of a multiplayer FPS. This means clients send their inputs to the server, and the server sends states to the clients here. Interpolation and prediction code also goes here. Finally, at the top we have game logic - code that (ideally) works only off the game state, and doesn't touch any network code. So if I wanted to add a CTF game type, I would only have to do three things: make a flag object, give players the ability to pick up flags, and make a new game logic module. None of that involves changing the network code. That's the ideal I'm shooting for.

Players can currently shoot and kill each other, and on the server side the score updates appropriately. I just have to replicate the changed score to clients, and add in some logic for ending the game, and I have a FPS deathmatch.

Wednesday, June 1, 2011

Game Objects (Part 3)

So as mentioned, my laptop's hard drive remains broken and I haven't been able to get a new one yet. Fortunately, all the code for ProjectFPS remained perfectly intact, and I was able to get it off and continue where I left off.

My animation system has been put on hold for now. There are things I'd like to accomplish (such as more efficient resource memory usage, separation of skeletons and the meshes that skin them, and per-object callbacks) that just don't jive with the D3DX animation system, nor any X file exporters. In fact, the version of Blender I have doesn't even have an X file exporter, and when I threw in one written for an older version of Blender, meshes would not export correctly.

So I looked into the COLLADA format. I knew Blender had an exporter for it, and I had heard that 3D modeling programs like Maya and 3DSMax commonly used it to pass data from one program to another. I set out trying to use the OpenCOLLADA library to write a converter from COLLADA to a proprietary skeleton format and mesh format. It took me 2-3 days just to get the OpenCOLLADA part of the converter to compile and link, thanks to cryptic and nonsensical errors. Of course, when I tried using Visual Studio 2008 instead of 2010, these errors magically went away.

In my infinite wisdom, however, I had failed to do research on Blender's COLLADA exporter to see if it actually worked okay. Turns out it doesn't. My vertices had no weights on them at all (which you can't do skeletal animation without), and from what I've read on the web, animation export is kinda funky too. I could write my own Blender exporter, or fix the COLLADA one, or get a student version of 3DS Max. All these I either can't do right now or they require significant effort, and I felt that it just wasn't worth it yet, considering I still don't have much in terms of a game that people can play with each other.

So I've decided to work on something else: networking. You can have a multiplayer game without animation, but you can't have a multiplayer game without multiplayer. So far, I've got a server that multiple players can connect to. They can see each other and shoot the walls. As usual, a topic for another post. Now, back to game objects.

So as mentioned in the last post, the hierarchical model has its fair share of problems, mainly because of implication. If an object is able to do X, it is implied that it can also do Y and Z, which is often not the case, especially when you're defining new objects that weren't in the original design. What we need, then, is a way to precisely specify what objects can and can't do where the sheer variety of them defies hierarchical classification.

The key to doing this lies in converting what is called "is-a" relationships to "has-a" relationships. What this means is instead of saying Player is an AnimatedObject, you say he has a AnimationComponent. Thus, one could define a player like this instead:

class Player : public GameObject
{
    void Update();
    void Draw();

    AnimationComponent m_Animation;
    PhysicsComponent m_Physics;
    NetworkReplicationComponent m_NetworkReplication;
    // Other components, variables
};

Note that we inherit directly from GameObject and nothing else. That's because all the functionality we need has been put into these components, which act like plug-ins. We stick only necessary components into the object, and that gives it the functionality it needs, nothing more. So if we need a StatuePlayer class or something that can't animate, we do this:

class StatuePlayer : public GameObject
{
    void Update();
    void Draw();

    PhysicsComponent m_Physics;
    NetworkReplicationComponent m_NetworkReplication;
    // Other components, variables

};

Nothing's changed, we're still inheriting from GameObject. We just omit the AnimationComponent. This system offers us a lot of convenience. Even two completely unrelated objects can have the same components, and thus do the same things. Building new objects is also easy, and thanks to the plug-in nature of components, code duplication becomes mostly unnecessary.

An object's updating might involve updating its components, changing their settings, etc., then acting off the results. So it would, for example, tell the animation component to update its current animation in Update(). Then, in Draw(), the object would grab the matrices generated by the component and use that to draw itself.

This doesn't mean that inheritance completely goes away, however. Like before, if we wanted to update everything in the scene, since they all still inherit from GameObject, we can have a list of GameObject pointers that we call Update() and Draw() on. And if you had, say, a WeaponInventory component, it could store a collection of Weapons, which all weapons would inherit from. Note, however, that this kind of inheritance is different - we're more working off what objects are, rather than what they can do. This kind of inheritance is typically very shallow.

Then there's the problem of inter-object relationships. Don't you hate that? The issue is that since implication is now gone, unless we have the object's specific type, we can't tell exactly what components it has. So you can't iterate over a collection of GameObjects and modify their physics, because not all of them will have it. There are multiple ways to tackle this problem, and some of them could be done at the same time.

One simple way is to put a query function in GameObject that all derived classes must overload. You give it a component name, and it either gives you back a component pointer or NULL. Simple, but slow, since the requested type must go through a switch statement or something to that effect every time. So to modify the physics of all relevant GameObjects, you might do this:

for (int i = 0; i < numObjects; i++)
{
   PhysicsComponent* pPhysics = gameObjects[i].GetComponent(COMP_PHYSICS);
   if (pPhysics) pPhysics->ChangeSomeSetting();
}

A more efficient but complex approach is to create lists of components for the more important component types. When a PhysicsComponent is created, then, it would register itself with the appropriate list. The code then becomes:

for (int i = 0; i < numPhysicsComps; i++)
{
   physicsComps[i]->ChangeSomeSetting();
}

In my project, I use both approaches, depending on how much benefit I would get from the latter vs. the convenience of the former. Inter-component communication (e.g. between Physics and Transform) is a very similar problem, only now you have to be careful not to hurt the re-usability of the component in other kinds of objects.

And that's the basis of my game object design. Hopefully I didn't forget to say anything important. =P My next topic will probably be networking, since it has been the sole thing occupying my mind for weeks now. It also plays a big part in how you design your objects.