среда, 10 февраля 2010 г.

Design components, subsystems and modules...

Hi all,



Unfortunately, at the last couple months I haven't enough time to proceed with development of 3D game engine, and appears to me I wont have enough time in nearest future, but... I really need to continue work on this project, otherwise my IQ/skills/ goes down ... in other words without such work I feel myself really upset...




Brief: Today I'll provide a list of useful design components and subsystems, which will be implemented or used. Also I'll specify the data format which will be used for storing meshes and animations.

It is obvious thing that we need to store and manage either internal 3D engine data (meshes, skeletons, events, and so on) and Game Core data (gameplay entities, in-game events and so on) somehow. So the common used design components are:
  • linked list -- e.g. used for queues;
  • indexed array;
  • Z-ordered list (or array indexed by Z-value);
  • ring-buffer;
  • whatever?
We will need following subsystems either for 3D engine layer and for Game Core layer to parse *.ini files, printout variety of debug-purposed info (which could be filtered somehow in run-time)... So the list of common subsystem:
  • debug/log subsystem -- is used to provide debug messages output;
  • *.ini file parse subsystem -- is used to load configuration and other game data from *.ini files;
  • whatever?
3D engine specific subsystems and components:
  • mesh loader (+ skeleton loader if mesh file format requires it);
  • texture loader;
  • animation loader;
  • scene management subsystem;
  • rendering subsystem based on OpenGL (3D scenes, 2D scenes, fonts, textures etc..);
3D engine external data format:




So, with new approach it make sence to provide a bit more detailed diagram of data flow from game core to 3D renderer layer. I need to figure out the minimum set of data, which has to be passed from game core layer to 3D renderer layer.



to be continued...

WBR,
Vadim

пятница, 5 февраля 2010 г.

Transfer data from Game core into 3D engine

Hi all,

Hope you still with me... =)

Today I'll try to figure out one of the possible ways of passing data from Game Engine to 3D engine for drawing.

=== Separated Threads architecture ===

First of all I' have to note that each time when Scene Manager is called to draw the scene it will draw only one single frame. So, it means that until this routine isn't called the 3D drawing subsystem awaits. May be blocked on some semaphore, whatever. Before calling scene rendering method we should prepare the data for drawing from the Game Core layer. There are some questions here:
  • Q: Should we split this routine into two threads one for updating game core data, game objects and another one strictly for processing the 3D scene and drawing elements or this work may be done within one thread?
  • A: Yes, we separate the game updating process (objects state, game state, user events etc) and scene rendering;
  • Q: How we should pass data from Game Core layer to 3D Engine?
  • A: We need to pass strictly limited set of data from Game Core layer to Scene Manager layer via Scene Queue -- fill a small structure with necessary set of data and place it into Scene Queue;
  • Q: which data we should pass each time when we need to update scene?
  • A: strongly limited set of data related to each game entity - entity position and rotation (quaternions), mesh id, texture id, skeleton id and animation state id (in case we have skeletal animation);
  • Q: how should we store all of 3D related data in the meantime?
  • A: we don't need to store any graphical data at the Game Core layer. All graphical related data is stored at the 3D engine layer. At the Game Core layer we should store only links (some kind of IDs) to resource(s) used by each particular game entity;
  • Q: does the Scene Manager flush its queues after rendering scene?
  • A: the Scene Manager should flush Active Scene Queue from memory after scene gets rendered and switch Active and Inactive Scene Queues between each other. Then Scene manager will proceed rendering of next frame based on data from Active Scene Queue;

Let's see to the design architecture for two threads implementation (game state update and render) - refer to the picture below:



Briefly description of the design components:
  • Game Core (GC) -- update game world and place visible entities (according to current position and rotation data) into Inactive Scene Queue and notify Scene Manager about it;
  • Scene Manager (SM) -- picks up the new elements from Active Scene Queue and renders them;
  • Scene Queue (SQ) -- the abstract object which is implemented to let threads be running simultaneously; Scene Queue contains two FIFO lists - Active Scene Queue and Inactive Scene Queue; Active Scene Queue - contain elements which currently renders by drawing subsystem; Inactive Scene Queue is used by Game Core update thread to store (describe) new frame data and pass it into Scene Manager; Scene Queue contains mutex and counting semaphore also to manage access to lists;
Briefly description of Scene Queue component:

1. Scene Mutex is used for:
  • preventing simultaneous access from Game Core side to Active Scene Queue which is currently being rendered;
  • preventing switching of scene queues by Scene Manager until Game Core has not completed updating of Inactive Scene Queue;
2. Counting semaphore is used for:
  • blocking of rendering thread in case there are no any elements for rendering in the Active Scene Queue;
  • waking of rendering thread in case new elements have been added into Active Scene Queue by Game Core;
  • blocking of the Game Core update thread in case previous frame hasn't been rendered yet (means that the Active and Inactive queues have not been switched yet since last update operation);
The maximum value of counting semaphore is set to 1 when it has been initialized;
The activity diagrams of Game Core Update and Render threads are provided below:


Some comments about activity diagrams:
  • Scene Manager flushes the Active Scene Queue each time when frame has been rendered;

=== Single Thread Architecture ===

This design approach is actually the same as previous just without mutex, counting semaphore, Scene Queues, and now we have only one single thread to process game world and 3D scene. The all operations called to add node into the scene for rendering applies directly on Z-ordered Render Queue. So, each time when Game core adds new node into the scene, this node goes directly into Render Queue and takes place there according to node Z-axis value.

Activity diagram of "sceneUpdate" routine for single thread architecture is provided below:


Note: before this routine will start we need to have some list of all in-game objects to process them. Actually we will process this list and collect from it visible objects.

From the data transfer perspective -- there is no significant changes, we still need to pass limited set of data (mesh id, texture id, skeleton id, animation state id/number, position and rotation). Also we don't need to store graphical data at the Game core layer, just links to them -- all 3D stuff is managed by 3D engine core (Scene Manager, Resource Manager etc).

Unfortunately, in case we have single thread for processing all in-game stuff, someday (e.g. in case of game world with large number of entities) we could face an significant time delay issue which causes FPS degradation.

WBR,
Vadim

понедельник, 1 февраля 2010 г.

Initial design architecture

Hi all,

Time to begin. My major intent here is to get practice and experience of game development, so I won't try to create something complex. Just simple 3D arcade, with some limitations and features.

First of all I would like to specify some requirements for the subsystems of the game engine.
  • graphical output subsystem - openGL;
  • physical engine;
  • user events processing;
  • target platform - OS Linux;
  • engine has to be pluggable - ?;

Initial 3D engine structure is provided at the picture below.


Brief description of 3D engine components:
  • Scene Manager (SM)- abstract entity targeted to manage content of the scene; Scene Manager implements the set of API required to add/remove scene nodes, groups, lights, animation states for nodes; prepares the render queue, fills it with user viewable objects only, and call the API of render subsystem to proceed with pre-processing and drawing objects;
  • Resource manager (RM) - abstract entity to store meshes, textures, bumpmaps, animations etc., and provide links to them for in-game objects;
  • Render queue (RQ) - Z-ordered queue; This queue is prepared by scene manager and passed further into the rendering subsystem; The queue contains Z-ordered renderable objects. These objects contains data required only for drawing them via OpenGL layer;
  • Engine rendering subsystem (ERS)- performs pre-processing (apply mesh transformations, rotations, animations, whatever? without calling to openGL API ) at the render queue and after that draw the renderable items using openGL API;
  • game manager (GM) - suppose to be major entity of game core, implements manipulating of all in-game objects. GM implements calls to SM, in case if new node needs to be added into the scene or remove from it, to set up links between scene node and mesh, mesh and texture, etc;
In other words the one frame rendering design overview could be represented as follows:



==
From the high level perspective the 3D engine has two separate threads - one for drawing scene, another one for capturing events from user. The activity diagrams for both threads is provided below:


At this moment I plan to both threads be blocked:
  • drawing thread is blocked until updateScene method isn't called by Game Manager (or ?);
  • event capturing thread blocked until new event comes; this thread is waiting for new event by calling XWindowEvent; After event is received it calls the game core event handle callback function (see picture below);
Q: Why event event capture thread is located and running at the 3D engine layer?
A: The 3D engine layer besides drawing also creates render target - window (X-Window) and stores its data inside of internal data structures. So it seems logical to run this thread inside of 3D engine, and use callback function to pass events to the game core.

And one more picture, now about high level overview of message processing architecture:

The ring-buffer is used to pass event data from 3D engine core into game core. Here we have two threads:
  • event capture thread, running at the 3D engine layer and waiting for event from X-server for our window;
  • event processing thread, running at the game core layer, also waiting for new events; but this thread waits until new event appears at the ring-buffer;
==
couple comments about realization:
  • event handling subsystem requires callback function (pointer) to be provided before start event processing loop; I'm not sure about using ring-buffer for passing user events from 3D engine to game event processing layer, since such design obviously implements some timings;
  • 3D drawing subsystem requires callback to drawing function(s). In this case 3D engine becomes more flexible, but such realization seems to me unnecessary right now;

WBR,
Vadim

p.s. eventually it is a not such easy work to describe design and provide documentation to it; but I'll try..

at the very beginning...

Hi,

First of all let me make some comments before I start writing.
My English is a bit... don't know how to say that, but I have strong feelings that my English skills aren't like I want them to be. Especially in grammar.

So, I just would like to ask you to excuse me if my English appears incorrect to you -- feel free to let me know about my mistypo, gramar issues and so on - and I'll correct them.

==
It's my first game development experience. I plan to post some thoughts here, in blog to summarize ideas, and possibly to discuss them. As a software developer, I have an 5 years experience at the IP-telephone area.

Actually, I always wanted to create something by myself, create my own world. Yes, I know it may appears insane or foolness, but it is my insane and lameness. Eventually, it became an idea to develop a simple 3D game.

Moreover, before opening this blog I've spent about 1.5 year investigating of game design basics and OpenGL, and you know, I haven't found any step-by-step tutorial article about designing simple game. Yes, there are fine books such as "3D Game Engine design", "3D game engine architecture" by David H. Eberly, "3D game engine programming" by Stefan Zerbst and Oliver Duvel etc, but for me it makes sense to use them as reference books only. These books contains a lots of useful info, but it is a bit hard to read these books from start till end. Also all of them describes a development on C++, for MS Windows platform using DirectX library. In my case I don't need description of "how abstract class ideology works", and so on. I need short description of design architecture, of design components and their interoperability to each other.

My idea is to develop application on C language, using openGL and based on POSIX standards, which means that such application could be compiled for Linux and Mac (may be for MS Windows too).

WBR,
Vadim