More about Component Systems
Posted: 2014-01-02, 13:05
When I first posted about Component Systems exactly one year ago, things were still at the very beginning. Since then, the efforts to introduce Component Systems to the GUI window hierarchies of 2D and 3D GUIs, and to the game entity hierarchies of game maps made very good progress. In fact, I was so busy with and dug into my work, that I didn't get down to actually talk about them. ![Embarassed :oops:](./images/smilies/icon_redface.gif)
The good news is that it is now perfectly clear that the new Component Systems are a very important step for the future of the Cafu Engine. They give joy and pleasure every day that I work with them.
So I thought that this was a very good time to briefly reflect on what has happened during the past year, and to highlight a few related aspects in greater detail.
The problems with class hierarchies
Before it sinks into oblivion for good, I would like to summarize where we came from and what the problems were, as this helps to understand what the Component Systems actually achieve.
At the time that it was written, even though we were aware that inheritance is not a means for code reuse, we thought that a class hierarchy was a good idea, because it naturally seemed to group entities that have related features. And even though the implementation actually worked in the beginning, it soon turned out that there were plenty of shortcomings.
For example, at some point we wanted to augment entities to play 3D sounds, complete with Doppler effects and settings to control the volume, emission cone, and several other details. Our natural approach was to derive a new
It took me a while to realize it, but when I did, the result was pretty shocking: Even though the Cafu Sound System allows for minimal code in the game entities, not only were we left with several pieces of code that all essentially did the same, scattered across many entity classes. It was also very cumbersome to properly document all this (in fact we never did), to keep it in sync with the Map Editor and tools, and to deal with any special cases that occurred.
You can find another account on the the principal problems with class hierarchies in the blog post Cowboy Programming: Evolve Your Hierarchy by Mick West.
In hindsight, I'd say that many of these shortcomings were so severe that they effectively hindered or even halted the further development of the Cafu Engine. The old code just didn't scale, as adding new features caused the combinatorial complexity to grow worse and worse. Also, the code in the Map Editor, the map compile tools and the Cafu Engine was not the same, and keeping it in sync turned more and more into a pain. Not to speak of the special-cases that were needed here and there, and which we never managed to totally avoid.
These issues also burdened game developers with unnecessary complexity, e.g. the requirement to master C++ in order to make even small changes or to add small features, a burden which many game developers understandably did not want to concern themselves with.
Two class hierarchies, two Component Systems
Game entities are the objects that populate our game worlds, such as the human player, monsters, weapons, items, ...
GUI windows are the basic rectangles of which our GUIs are built. GUIs are used both in 2D (for the Main Menu, the in-game console, etc.), as well as in 3D, where they are a part of the game worlds and used e.g. as lift controls, computer terminals, and so on.
These two systems are designed to work together, but regarding code design, they're largely independent of each other. By coincidence however, we find that both are structurally very similar to each other and share the same ideas of hierarchical setup and organization of concrete instances.
So both systems suffer the same problems as described above, and both happen to be candidates for migration to a Component System. In fact, as the GUI windows hierarchy was so much smaller than the game entities hierarchy, I started the initial tests and the actual migration with the GUI windows. This went very well, and now all the efforts are focused on the game entities. Read on for details.
Generally, all that is said above and below holds for both game entities and GUI windows, although I may often talk about and provide examples for only one, but not the other.
The new design
After careful research and planning, I found that the new design can and should be as simple as this:
Entities
All entity instances are now of the same common class:
A new aspect however is that entities can now have children, or "sub-entities". In fact, entity instances (not classes, don't get confused!) can now be hierarchically organized.
For example, consider a car that consist of a chassis and four wheels, or a complex lift system that consist of a moving platform and a set of moving doors at each floor. It is now possible to have an
The same is found (and possibly feels a bit more natural) in GUI window hierarchies: We often have a window that acts as a dialog frame, and child windows of it are used for adding details like the dialog message, text input fields and the "OK" and "Cancel" buttons.
Components
Each component implements exactly one feature, and each feature is implemented in exactly one component.
For game entities, we have (at the time of this writing) these components:
![Thumb Up :up:](./images/smilies/thumbup.gif)
Generally, components are similar to a code design pattern: They naturally group things together that belong together, and encapsulate code and features in a manner that is concise and clear. Above all, components eliminate the combinatorial complexity that occurs with adding features in classic class hierarchies. As we will see in the next section, components also break up other old weaknesses and bring very worthwhile improvements.
Variables
Each component has a list of variables. This may sound like a note of trivial notability, but in fact I came up with a very nice scheme from which our Component Systems draw much of their power: Entities can iterate over their components, and components can iterate over their variables. On this basis, it was possible to implement features such as:
Isn't that absolutely rocking cool??![Cheesy Grin :cheesy:](./images/smilies/icon_cheesygrin.gif)
These virtues were achieved by employing the "Visitor" code design pattern for implementing the variables (which, when you look at the source code, are actually small classes of their own).
Note that the motto "everything can be interpolated" is twofold: One aspect is "real" interpolation, where for example we move a platform from A to B, without concerning us with the intermediate positions along the track. Another aspect is the interpolation on the client, where we interpolate over client frames to fill the larger gaps between server frames, achieving a movement that is as smooth as possible, and independent of network latency that possibly affects the arrival of server updates.
The Component System for GUI Windows
When I was done in March 2013, there was only one problem left: I was eager to get started doing the same with the game entities. So eager in fact, that I didn't want to waste time with anything else, such as expanding the GUI system (adding new components was so easy then that it was very tempting to add ton of new features!), or mundane tasks like writing proper documentation. (The new reference documentation was of course a piece of cake, you can see it here. I'm talking about the introductory documentation in the Wiki, with screenshots and well-phrased prose texts...) Well, I'm sorry to say that these things are still not done, and unless someone is willing to help, are likely to stay that way at least until I'm done and happy with the Component System for game entities.![Wink ;-)](./images/smilies/icon_wink.gif)
More benefits
Component Systems remove the complexity from combining features, providing both more flexibility and more capabilities to game development.
But there is more: With our implementation of Component Systems, we can finally use the same code everywhere: from the Map Editor over the map compile tools to the Cafu Engine, they all use the same code now! For example, in the Map Editor, we must support editor-specific features such as "selections" (objects clicked with the mouse for further editing). But now, such support is only a matter of adding another Component to the selected entity! That's all. All other code is the same as that in the map compile tools and in the Cafu Engine.
A similar aspect allows for unified script states. Previously, script states had a tendency to get fragmented, that is, it could be difficult to access the state of the game world from the script code of an embedded 3D GUI. My implementation of Component Systems unifies script states as well.
Important for many game developers, there is no longer a need to write C++ code (if you don't want to). There is full flexibility and full power available with scripts. (And those who love C++ get their money's worth as well – hacking Cafu in C++ is more fun than ever!)
And finally, we're able to create very fine scripting reference documentation mostly automatically. As indicated above, we can iterate over components and their variables. This allows to generate complete Doxygen documentation automatically. To improve this even further, it is of course also possible to augment the generated documentation by hand.
The strings that are used to document the components and their variables are available in CaWE (e.g. the Map Editor) as well! That means that the documentation is not only available as a nice online web resource, but is readily available in CaWE, too! You can see an example of this in the "Window Inspector" screenshot at the bottom ("hor. Align"). It is even possible to query the help string for a variable at the in-game console in Cafu, or from any other script code, if you wish.
References
For historical reference, here are some related forum posts where Component Systems were previously discussed:
) and who participated in discussions and provided suggestions!
Ongoing work
However, we're not done yet: the Component System for game entities has made excellent progress, but there is still work to do with porting old DeathMatch game code to the new system. There is also a lot of documentation to write (those introductory and prose parts that don't automatically write themselves...), and many pages of the website to update (Features, Gallery, ...). And once that's done, when the new Component Systems are in place and all obsolete code is gone, a new release is due, and the real fun begins: We can then add new features that we couldn't add before, add new scripts that we didn't have before, and so much more.
If you're interested in the details, have a look at the Cafu source code repository, where currently the active work is in the
I wish everyone a Happy New Year and all the best for 2014!![WOW :wohow:](./images/smilies/wohow.gif)
Carsten
![Embarassed :oops:](./images/smilies/icon_redface.gif)
The good news is that it is now perfectly clear that the new Component Systems are a very important step for the future of the Cafu Engine. They give joy and pleasure every day that I work with them.
So I thought that this was a very good time to briefly reflect on what has happened during the past year, and to highlight a few related aspects in greater detail.
The problems with class hierarchies
Before it sinks into oblivion for good, I would like to summarize where we came from and what the problems were, as this helps to understand what the Component Systems actually achieve.
Until about a year ago, we used to use "classic" C++ class hierarchies both for GUI windows as well as for game entities. You can see an excerpt of the entity class hierarchy as it used to be found in the (now revised) C++ API documentation in the image to the right.
At the time that it was written, even though we were aware that inheritance is not a means for code reuse, we thought that a class hierarchy was a good idea, because it naturally seemed to group entities that have related features. And even though the implementation actually worked in the beginning, it soon turned out that there were plenty of shortcomings.
For example, at some point we wanted to augment entities to play 3D sounds, complete with Doppler effects and settings to control the volume, emission cone, and several other details. Our natural approach was to derive a new
EntSoundT
entity class from the BaseEntityT
class, and have it implement all the features that we wanted. However, it soon turned out that we not only wanted dedicated EntSoundT
entities to play sounds, but many other entities (of different entity classes) as well. For example, an item that is picked up by the player and respawns a few seconds later makes a sound at both occasions. So we re-implemented the sound functionality in the item's entity code as well. And soon in the code for monsters, for human players, and several others.It took me a while to realize it, but when I did, the result was pretty shocking: Even though the Cafu Sound System allows for minimal code in the game entities, not only were we left with several pieces of code that all essentially did the same, scattered across many entity classes. It was also very cumbersome to properly document all this (in fact we never did), to keep it in sync with the Map Editor and tools, and to deal with any special cases that occurred.
You can find another account on the the principal problems with class hierarchies in the blog post Cowboy Programming: Evolve Your Hierarchy by Mick West.
In hindsight, I'd say that many of these shortcomings were so severe that they effectively hindered or even halted the further development of the Cafu Engine. The old code just didn't scale, as adding new features caused the combinatorial complexity to grow worse and worse. Also, the code in the Map Editor, the map compile tools and the Cafu Engine was not the same, and keeping it in sync turned more and more into a pain. Not to speak of the special-cases that were needed here and there, and which we never managed to totally avoid.
These issues also burdened game developers with unnecessary complexity, e.g. the requirement to master C++ in order to make even small changes or to add small features, a burden which many game developers understandably did not want to concern themselves with.
Two class hierarchies, two Component Systems
Before we proceed, be aware that in Cafu we have two class hierarchies that are important for game development:
- game entities
- GUI windows
GUI windows are the basic rectangles of which our GUIs are built. GUIs are used both in 2D (for the Main Menu, the in-game console, etc.), as well as in 3D, where they are a part of the game worlds and used e.g. as lift controls, computer terminals, and so on.
These two systems are designed to work together, but regarding code design, they're largely independent of each other. By coincidence however, we find that both are structurally very similar to each other and share the same ideas of hierarchical setup and organization of concrete instances.
So both systems suffer the same problems as described above, and both happen to be candidates for migration to a Component System. In fact, as the GUI windows hierarchy was so much smaller than the game entities hierarchy, I started the initial tests and the actual migration with the GUI windows. This went very well, and now all the efforts are focused on the game entities. Read on for details.
Generally, all that is said above and below holds for both game entities and GUI windows, although I may often talk about and provide examples for only one, but not the other.
The new design
After careful research and planning, I found that the new design can and should be as simple as this:
Entities → Components → Variables
Let's look at each of these elements in greater detail:Entities
All entity instances are now of the same common class:
EntityT
. No inheritance, no class hierarchy. Essentially, entities are now nothing but lists (or "containers") of components.A new aspect however is that entities can now have children, or "sub-entities". In fact, entity instances (not classes, don't get confused!) can now be hierarchically organized.
For example, consider a car that consist of a chassis and four wheels, or a complex lift system that consist of a moving platform and a set of moving doors at each floor. It is now possible to have an
EntityT
instance that represents the "whole thing", the entire car or the entire lift, possibly with a set of components that define features for the whole, and to have child entities that represent the parts, where each part is another EntityT
instance that can contain components of its own.The same is found (and possibly feels a bit more natural) in GUI window hierarchies: We often have a window that acts as a dialog frame, and child windows of it are used for adding details like the dialog message, text input fields and the "OK" and "Cancel" buttons.
Components
Each component implements exactly one feature, and each feature is implemented in exactly one component.
For game entities, we have (at the time of this writing) these components:
- Basics (name, status),
- Transform (origin, orientation),
- Model,
- Collision Model,
- Point Light,
- Radiosity Light,
- Particle System,
- Player Physics,
- Script (custom scripted behavior),
- Sound.
- Basics (name, visibility),
- Transform (position, size, rotation),
- Border (frame),
- Text,
- TextEdit,
- Image, Model,
- Choice, ListBox.
![Thumb Up :up:](./images/smilies/thumbup.gif)
Generally, components are similar to a code design pattern: They naturally group things together that belong together, and encapsulate code and features in a manner that is concise and clear. Above all, components eliminate the combinatorial complexity that occurs with adding features in classic class hierarchies. As we will see in the next section, components also break up other old weaknesses and bring very worthwhile improvements.
Variables
Each component has a list of variables. This may sound like a note of trivial notability, but in fact I came up with a very nice scheme from which our Component Systems draw much of their power: Entities can iterate over their components, and components can iterate over their variables. On this basis, it was possible to implement features such as:
- automatic dialogs in the Map and GUI Editors (see example below),
- automatic serialization to and from disk,
- automatic serialization and synchronization over the network,
- automatic generation of documentation,
- automatic binding to Lua (all variables are accessible by script),
- automatic ability to interpolate all variables (of type
float
,double
, or composites thereof).
Isn't that absolutely rocking cool??
![Cheesy Grin :cheesy:](./images/smilies/icon_cheesygrin.gif)
These virtues were achieved by employing the "Visitor" code design pattern for implementing the variables (which, when you look at the source code, are actually small classes of their own).
Note that the motto "everything can be interpolated" is twofold: One aspect is "real" interpolation, where for example we move a platform from A to B, without concerning us with the intermediate positions along the track. Another aspect is the interpolation on the client, where we interpolate over client frames to fill the larger gaps between server frames, achieving a movement that is as smooth as possible, and independent of network latency that possibly affects the arrival of server updates.
The Component System for GUI Windows
As mentioned above, I started with the Component System for GUI windows. The goal was to re-implement the then existing functionality (and optionally to improve and expand it) in terms of the new Component System. This goal has long been reached, and it was a huge success (see ticket #125 for details).
When I was done in March 2013, there was only one problem left: I was eager to get started doing the same with the game entities. So eager in fact, that I didn't want to waste time with anything else, such as expanding the GUI system (adding new components was so easy then that it was very tempting to add ton of new features!), or mundane tasks like writing proper documentation. (The new reference documentation was of course a piece of cake, you can see it here. I'm talking about the introductory documentation in the Wiki, with screenshots and well-phrased prose texts...) Well, I'm sorry to say that these things are still not done, and unless someone is willing to help, are likely to stay that way at least until I'm done and happy with the Component System for game entities.
![Wink ;-)](./images/smilies/icon_wink.gif)
More benefits
Component Systems remove the complexity from combining features, providing both more flexibility and more capabilities to game development.
But there is more: With our implementation of Component Systems, we can finally use the same code everywhere: from the Map Editor over the map compile tools to the Cafu Engine, they all use the same code now! For example, in the Map Editor, we must support editor-specific features such as "selections" (objects clicked with the mouse for further editing). But now, such support is only a matter of adding another Component to the selected entity! That's all. All other code is the same as that in the map compile tools and in the Cafu Engine.
A similar aspect allows for unified script states. Previously, script states had a tendency to get fragmented, that is, it could be difficult to access the state of the game world from the script code of an embedded 3D GUI. My implementation of Component Systems unifies script states as well.
Important for many game developers, there is no longer a need to write C++ code (if you don't want to). There is full flexibility and full power available with scripts. (And those who love C++ get their money's worth as well – hacking Cafu in C++ is more fun than ever!)
And finally, we're able to create very fine scripting reference documentation mostly automatically. As indicated above, we can iterate over components and their variables. This allows to generate complete Doxygen documentation automatically. To improve this even further, it is of course also possible to augment the generated documentation by hand.
The strings that are used to document the components and their variables are available in CaWE (e.g. the Map Editor) as well! That means that the documentation is not only available as a nice online web resource, but is readily available in CaWE, too! You can see an example of this in the "Window Inspector" screenshot at the bottom ("hor. Align"). It is even possible to query the help string for a variable at the in-game console in Cafu, or from any other script code, if you wish.
References
For historical reference, here are some related forum posts where Component Systems were previously discussed:
- The original suggestion by midix.
- Additional discussion in these two threads by MatejZajacik.
- My first News posting (introduction, first summary).
- Some related tickets: #113, #125.
![Wink ;-)](./images/smilies/icon_wink.gif)
Ongoing work
In summary, the new Component Systems provide elegant solutions to old problems. They make game development much easier and much faster, a lot more flexible, and more fun than ever.
However, we're not done yet: the Component System for game entities has made excellent progress, but there is still work to do with porting old DeathMatch game code to the new system. There is also a lot of documentation to write (those introductory and prose parts that don't automatically write themselves...), and many pages of the website to update (Features, Gallery, ...). And once that's done, when the new Component Systems are in place and all obsolete code is gone, a new release is due, and the real fun begins: We can then add new features that we couldn't add before, add new scripts that we didn't have before, and so much more.
If you're interested in the details, have a look at the Cafu source code repository, where currently the active work is in the
entity-component-system
branch (which will be merged back to master
once done).I wish everyone a Happy New Year and all the best for 2014!
![WOW :wohow:](./images/smilies/wohow.gif)
Carsten