Changing EntityStateT for flexibility

Get technical support about the C++ source code and about Lua scripts for maps, entities, GUIs, the console, materials, etc. Also covered are the Cafu libraries and APIs, as well as compiling, linking, and the build system.
Post Reply
User avatar
Haimi
Posts:85
Joined:2011-11-23, 09:28
Changing EntityStateT for flexibility

Post by Haimi » 2012-02-27, 13:09

Hi @ll,

I was reading through the code and I thought if it would be helpful to change the engine code to make the EntityStateT independent from the Engine. Every game has other Options and so it makes not very much sense to hardcode the EntityStateT Options into the Engine code.

So i thought about changing the server code to recognize the members of EntityStateT by itself.
The other Idea is to create an extra class with the Options of EntityStateT, so the Server and Client may call specific methods is used (e.g. changing the health is calling the class). This may help to e.g. auto-assign the Field masks.

Do you have any ideas of a best practice with this?
Thanks alot!
Project Status: Code architecture definition
6 Programmers, 1 Photographer, 1 Architect, 1 Game designer
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Changing EntityStateT for flexibility

Post by Carsten » 2012-02-28, 01:10

Every game has other Options and so it makes not very much sense to hardcode the EntityStateT Options into the Engine code.
Yep, that's absolutely right.

My rough plan / idea outline was as follows:
EntityStateT should not exist at all. As it is, it just is a member of the BaseEntityT class, and all entities have to use it. This was done in the ancient past so that the engine "knows" the state of each entity, and can serialize, deserialize, and manipulate it at will / as required.

One way to overcome this is to let each entity have its own data members, just as one would expect with any normal C++ class. Each entity class would be free to define as much or as little data members as required and desired.

Of course, the engine still has to know the state of the entity, e.g. for sync'ing it across the network etc.
This can be achieved by adding pure virtual methods with roughly this signature to BaseEntityT:

Code: Select all

virtual void Serialize(SpecialOutputStreamT& Stream)=0;
virtual void Deserialize(SpecialInputStreamT& Stream)=0;
Each entity had to implement these methods, and the engine would call them whenever it wanted to ask the client to write or read its state from or to the special Stream objects. For example, and entity had to make sure that in it's implementation of Serialize() it writes all it's relevant state into the stream, and to make sure that Deserialize() does the inverse: Read all relevant state data from the stream

Each stream would be a bitstream, and the engine would not be able to do something with it's contents, but it would be able to delta-compress them, run-length encode them, send and receive them over the network, etc. etc.

This is something that I consider very important (in fact, even more important than the OpenGL 3.x renderer).
Alternative ideas, discussion, help, patches, ... all very welcome! :-)
Best regards,
Carsten
User avatar
Haimi
Posts:85
Joined:2011-11-23, 09:28

Re: Changing EntityStateT for flexibility

Post by Haimi » 2012-03-11, 21:55

The night starts at 20:45, and so does the realisation of this topic :D

I am just wondering with which kind of dynamically-typed variable I should start, it needs to be something like cf::GuiSys::WindowT::MemberVarT
Project Status: Code architecture definition
6 Programmers, 1 Photographer, 1 Architect, 1 Game designer
User avatar
Haimi
Posts:85
Joined:2011-11-23, 09:28

Re: Changing EntityStateT for flexibility

Post by Haimi » 2012-03-11, 23:50

Okay, i am thinking about the following:
We create a template class like this:

Code: Select all

template <typename T> class EntityVarT
{
    private:
    T* Member;           ///< Pointer to the member variable.
    unsigned long Mask;  ///< Mask for this EntityVarT for client/Server Messages

    // Construct each EntityVarT with a value and a Mask value
    EntityVarT(T &v, unsigned long Mask_) {
        Member = &v;
        Mask = Mask_;
    }

    // Value setter
    void SetValue(T v) { this->Member = &v; }
    // value getter
    T GetValue() { return *this->Member; }
};
and give every entity an array of EntityVarT Objects. The Serialize() and Deserailize() Methods, who are Added to BaseEntityT iterate over all of those Elements. Every Server Action on an entity uses an instance of that enitity, so the methods for a special treatment of the values can be implemented in the entity code in the game itself.

To maintain the overview, every Entity needs to be holding a Set of definitions for its Var-Masks (maybe as constants). So there is not sooooo much to be changed.
My Idea for (De-)Serialization was like the following:

Code: Select all

    // Serialisation for the Entity for client-server transfer
    virtual void Serialize(NetDataT& Stream);   // Write all data into NetDataT
    virtual void Deserialize(NetDataT& Stream); // Read all data from NetDataT
And the Events are a part of the Entity itself, no longer of the EntityStateT.

Do you think this is the right way or are there some things I did not see?
Project Status: Code architecture definition
6 Programmers, 1 Photographer, 1 Architect, 1 Game designer
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Changing EntityStateT for flexibility

Post by Carsten » 2012-03-12, 01:07

Hi Haimi,

I'm currently quite busy with - finally - finishing the Model Editor, so I've not yet looked into this very thoroughly. But to pick up my previous suggestions, I'd add methods like

Code: Select all

virtual void Serialize(SpecialOutputStreamT& Stream)=0;
virtual void Deserialize(SpecialInputStreamT& Stream)=0;
to class BaseEntityT.

Class SpecialOutputStreamT (and SpecialInputStreamT) would newly be created as well, its public methods being mostly operator << for the most common POD types (int, bool, float, etc.).
It would store the data in a "bit-field", i.e. an array of uint32_t that is considered a series of bits.

For example (pseudo-code!):

Code: Select all

class SpecialOutputStreamT
{
    public:

    SpecialOutputStreamT& operator << (int i);
    SpecialOutputStreamT& operator << (bool b);
    SpecialOutputStreamT& operator << (float f);

    const ArrayT<uint32_t>& GetBitField() const;


    private:

    ArrayT<uint32_t> m_BitField;
    unsigned int     m_BitsWritten;
};
And an implementation of some entity that implements Serialize() and Deserialize():

Code: Select all

virtual void SomeEntityT::Serialize(SpecialOutputStreamT& Stream)
{
    Stream << m_myHeading;
    Stream << m_myPosX;
    Stream << m_myPosY;
    Stream << m_myBoolFlag1;
    Stream << m_myBoolFlag2;
    // ...
}


virtual void SomeEntityT::Deserialize(SpecialInputStreamT& Stream)
{
    Stream >> m_myHeading;
    Stream >> m_myPosX;
    Stream >> m_myPosY;

    const bool OldFlag1=m_myBoolFlag1;
    Stream >> m_myBoolFlag1;

    if (OldFlag1!=m_myBoolFlag1)
    {
        // Value changed (i.e., an event occurred)
        //   -- time to take some action...
    }

    Stream >> m_myBoolFlag2;
    // ...
}
Now, the interesting and more difficult part is probably the server side processing of the so obtained bit fields.
Typically, we have two such bitfields: the current one, and an older one that we are comparing to.
In order to find the "delta" to send over the network to update the other party, I would first build the XOR of both bitfields in order to obtain a bitfield that expresses where they are different: There will be all 0's where the two bitfields are the same, and 1's otherwise. Probably there'll be a lot more 0's than 1's.
The result can be RLE-compressed and be sent over the wire.

But again, I've only started looking into this. Finishing the Model Editor requires me to take a set of screenshots for the gallery and especially to write some documentation for it (all programming and all tickets that are important for now are done), and to update the Worlds.zip resource file as I've been updating some model paths.
As soon as that done, I'll take another look into the details of the above. :up:
Best regards,
Carsten
User avatar
Haimi
Posts:85
Joined:2011-11-23, 09:28

Re: Changing EntityStateT for flexibility

Post by Haimi » 2012-03-13, 13:47

Okay, I understand how you plan to do it. My Problem is: When i shift the values into The Special Streams, how will I be able to separate them again if any Enitity may have other Option fields?
Project Status: Code architecture definition
6 Programmers, 1 Photographer, 1 Architect, 1 Game designer
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Changing EntityStateT for flexibility

Post by Carsten » 2012-03-14, 00:43

Haimi wrote:When i shift the values into The Special Streams, how will I be able to separate them again if any Enitity may have other Option fields?
Ah, good question. I should have said more about it right away:

Obviously, for each concrete entity, the two methods Serialize() and Deserialize() must closely mirror each other: The order of serializing data in one must match the order of deserializing in the other.

If you want to have optional data, e.g. additional data that is needed only when the entity is in state "alive" but not when "dead", then there are two options:
  • Make whatever is written to and read from the stream dependent on a flag that is always written. That is, if flag == "alive", then extra data fields follow. Otherwise no or different extra data follows. In a sense, this would be a mini protocol layer that is specific to the entity class.
  • A possibly(?) better alternative is to simply write all data always, even if it is not always needed. That is, write the extra data fields for state "alive" also when being in state "dead". This is efficient because of the way the bitfields that are wrapped by the stream classes are handled: We only ever send the difference between to bitfields across the wire. When unused data doesn't change, the difference will consist of all zero's in the unnecessarily transferred part. But the subsequent RLE-compression will reduce that extra payload quasi to nothing, wasting no bandwidth at all.
While I'm at it, two additional details about the stream classes come to mind:
  • Contrary to my example above, we may provide the interface only for data types whose size is known in advance. Specifically, int and long int can be 32 or 64 bits wide, so it would be better to use e.g. int32_t instead, such that the proper signature is:

    Code: Select all

    class SpecialOutputStreamT
    {
        public:
    
        SpecialOutputStreamT& operator << (int32_t i);
        SpecialOutputStreamT& operator << (bool b);
        SpecialOutputStreamT& operator << (float f);
        // ...
    };
    
  • Then, transferring boolean flags could be optimized by "pooling" them, e.g. eight at a time in a single int8_t's...
But these are really details for later... :up:
Best regards,
Carsten
User avatar
Haimi
Posts:85
Joined:2011-11-23, 09:28

Re: Changing EntityStateT for flexibility

Post by Haimi » 2012-03-14, 16:24

Okay, I checked out a completely new copy of cafu :D

The start runs rather good, The default values (public) in BaseEntityT are:

Code: Select all

    VectorT               s_Origin;
    VectorT               s_Velocity;
    BoundingBox3T<double> s_Dimensions;
    unsigned short        s_Heading;
    unsigned short        s_Pitch; 
    unsigned short        s_Bank;
I prefixed the "State values" with s_. In Physics.cpp I am passing the Reference of the State-providing Instance of EntityStateT instaead of EntityStateT... I am currently rewriting the Entities... more later :D
Project Status: Code architecture definition
6 Programmers, 1 Photographer, 1 Architect, 1 Game designer
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Changing EntityStateT for flexibility

Post by Carsten » 2012-03-14, 17:58

Haimi wrote:I prefixed the "State values" with s_. In Physics.cpp I am passing the Reference of the State-providing Instance of EntityStateT instaead of EntityStateT... I am currently rewriting the Entities... more later :D
Uhhhhh.... what?
Sorry, I didn't really understand this.

Very importantly, I'd strongly suggest to add this new feature in small steps and in parallel to the existing networking code.

That is, the new streaming classes, entity (de-)serialization methods, and all that can be introduced and added without demolishing or even interfering with the existing code.

While the new code is added, the entity data is just transferred as before, using the old code.

Then, for a selected subset of entities (initially only one, then more), we can transfer both the old as well as the new data.

Finally, if all that works and the new code is in use, can we remove the old code, possibly followed by more cleanup.


In other words:
Please don't attempt a change of such magnitude and with such impact (we're discussing the fundamentals of Cafu Engine operation, after all ;-) ) in one big attack. The old code was severely limited in flexibility, but it worked over years without any known bug or issue.
Thus, please let's try to keep the individual steps as small as possible, and implement constructive steps first before removing or mass rewriting old code. :cheesy: :up:
Best regards,
Carsten
Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests