Renderers and Meshes

Get technical support about the C++ source code and about Lua scripts for maps, entities, GUIs, the console, materials, etc. Also covered are the Cafu libraries and APIs, as well as compiling, linking, and the build system.
thomasfn
Posts:14
Joined:2011-11-07, 22:48
Renderers and Meshes

Post by thomasfn » 2011-11-07, 22:58

Hi, I'm currently looking into making an OpenGL 3.2 renderer for Cafu. The current OpenGL renderer is for 1.x, and that works by pushing vertices one-by-one to the graphics card, each frame. As such, the renderer implements a DrawMesh call with a Mesh object - a sort of "here's a list of vertices, now draw it" contract. While this is fine for OpenGL 1.x and indeed DirectX, GL3.2 requires you to store persistent data per mesh (VBOs).

One option would be to add some kind of misc data variable to the mesh class, and have the GL renderer (or any other renderers that need to store persistent data on the mesh) use that - I could add this if needs be.

A better option would be to handle mesh object creation through the renderer - that is, the engine asks the renderer for a new mesh whenever it needs, rather than creates it's own. The renderer has the option of just giving it a new MeshT object, or implementing it's own class which inherits MeshT and giving it that. The advantage of this is that the renderer has full control over what it can store on the mesh object, through the magic of inheritance, while the base mesh object and any other place in the code which references the mesh remains renderer-agnostic.

The tricky part with the latter is going through and finding everywhere in the code where "new MeshT..." is used, and replacing it with some kind of renderer controlled mesh factory - something which someone with way more experience with the engine could do faster than I.

Any thoughts or opinions? Perhaps Cartsen could take a look at this, he could probably do this in 10x less time than me!
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-08, 11:39

Hi Thomas,
welcome to the Cafu forums! :welcome:

Your proposal sounds very good, and I'd be very happy about adding renderers that support VBOs.
Obviously, your suggestion about having the renderer creating and managing the meshes is much favorable over patching the existing MeshT class (which has potential for being overhauled/revised anyways).

Btw., Ronsaldo started a similar effort a while ago, see http://trac.cafu.de/ticket/31 for details. Unfortunately, he himself said: "I decided to leave my work in an incomplete state and leave it just as a small proof of concept, which is in the attached patch.". Well, I don't believe he's still working on it, and as I've been busy with so many other pressing things, so far I didn't complete digging through his patch (also because it's so big and monolithic).

Anyways, if you'd make the required changes to the renderer and meshes, I'd be happy to do the tedious work of updating all the "user" code that uses the old MeshT classes.

For "strategy", I think it would be worthwhile to attempt to do this in a series of non-destructive steps:
For example, if we first added basic VBO support besides the existing immediate mode, then tried to make it work in a portion of the user code that is small enough to be manageable (e.g. the Model Editor would be a very nice test field), then grew and expanded everything as needed, and only then switched the rest of the Cafu code base over to the new mesh classes (even this possibly subdivided into substeps, if necessary), I believe that the switch to VBO would work very well.

Well, let me know what you think!
I'm happy to do anything I can to assist you, including doing the tedious parts of the work. :up:
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-08, 14:49

Thanks for your quick reply.

The VBO specific implementation would be the responsibility of the renderer - the base mesh class, or any other renderer, won't care or require any VBO related stuff. As such, fallback to immediate mode if VBOs aren't available can be done using the existing renderer fallback mode - if GL3.2 isn't supported, the old GL1.x renderer can still be used and will still work as before.

All that needs to be changed in the main engine is virtualising the MeshT class, and having the renderer control mesh creation and destruction. (I'll have a go at doing that myself today, and let you know how it turns out).

The only problem I can foresee is animated meshes. Is the mesh object immutable, or are animations constantly updated in the same mesh object?
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-09, 00:28

Hi Thomas,
The VBO specific implementation would be the responsibility of the renderer - the base mesh class, or any other renderer, won't care or require any VBO related stuff. As such, fallback to immediate mode if VBOs aren't available can be done using the existing renderer fallback mode - if GL3.2 isn't supported, the old GL1.x renderer can still be used and will still work as before.
Yes, although I guess that besides the user code, also the non-VBO renderers must be changed to deal with the new MeshT interface, at least at the very end, when the old immediate-mode interface is eventually removed in favor of the new renderer-managed MeshT class. (Whereas their implementation, i.e. how they process the mesh data, will remain the same.)
All that needs to be changed in the main engine is virtualising the MeshT class, and having the renderer control mesh creation and destruction. (I'll have a go at doing that myself today, and let you know how it turns out).
Sounds very good!
(Just let me re-phrase my comment above: I'd suggest not to change the existing MeshT class, as the expected ripple effect would require to update all existing non-VBO renderers and all user code, unconditionally and immediately. Instead it's probably better to introduce a new set of classes (base class and concrete renderer-derived class) that can be introduced in parallel and simultaneously to the old class.)
The only problem I can foresee is animated meshes. Is the mesh object immutable, or are animations constantly updated in the same mesh object?
Currently, for animated meshes, the same mesh is just updated whenever the pose changed from one frame to the next. Creating a new mesh instance for each animated object for each frame is probably too expensive, so I guess that there must be some way to update the mesh even after it has been registered with the GPU...
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-09, 00:59

Unfortunately I managed to mess up my version of the engine, so it seems to randomly show obscure error messages and crash in places I haven't even edited - so I'm going to need to start over on the mesh interface front.

This was the sort of thing I was experimenting with though:

Code: Select all

virtual void BeginUpdate();
		virtual void FinishUpdate(bool updateall);

		void AddVertex(VertexT v);
		void SetVertex(VertexT v, int loc);

	protected:
		ArrayT<int> AffectedVertices;

	};
This was to be appended on the end of MeshT. The idea being, when the user code wants to build the mesh, it calls FinishUpdate(true) which, in the renderer-specific implementation, will upload the entire set of vertices to the GPU. When it's time to update the mesh for animation, the SetVertex and AddVertex calls are wrapped inside a BeginUpdate and a EndUpdate(false) pair - these keep track of which vertices changed (AffectedVertices array) and the renderer can just upload changed vertices to the GPU. If this turns out to be too slow, some way of storing all frames of animation in a set of meshes and precaching them will be required.

The meshes would still get pushed to the renderer in the usual DrawMesh way. Non VBO renderers can just not implement their own mesh class, and use the default.

Also I'll be on the IRC channel pretty much all day, probably a bit faster than talking on the forum.
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-09, 13:05

Well, as mentioned in my earlier post, I'd suggest to start like this (pseudo code for demonstration only, not tested, not complete):

To class RendererI, add these methods:

Code: Select all

        /// Creates a new mesh for rendering.
        /// It's up to the user code to set the vertices, normals,
        /// colors, etc. in the returned UnivMeshT instance.
        virtual MatSys::UnivMeshT* CreateMesh()=0;

        /// Deletes a previously created mesh.
        virtual void DeleteMesh(MatSys::UnivMeshT* Mesh)=0;
UnivMeshT would be a completely new mesh class that is given to the user code and that renderer implementations use derived versions of:

Code: Select all

    class UnivMeshT
    {
        public:

        /// Sets m_Modified to true.
        virtual void SetVertex(unsigned int VertexNr, const Vector3fT& Pos);

        // virtual void Set...(...);


        private:

        /// This is set to true, whenever a user method modified this mesh
        /// (geometry only? i.e. in a way that requires further action regarding VBO?).
        /// The renderer (or it's UnivmMeshT derived class) will take notice of this
        /// flag to update any related VBO resources etc., then reset it to false.
        bool m_Modified;

        // Type, winding, vertices, ...
    };
One big issue besides animated meshes (how does everyone else deal with them??) is the question who controls the rendering.
Currently, the user code calls Renderer->RenderMesh(...);
That is, rendering control is explicit in user code, and in fact the user code must even care about the rendering phases (ambient, stencil, lighting).

It's possible to keep it like that, but a (long-term, and probably better overall) alternative is to move all rendering control into the renderer:
The renderer knows all meshes that it ever created through the new CreateMesh() calls, it could "just" loop over them and render them all at it's own discretion.

This has several advantages / consequences, but some open questions as well:
  • Each mesh might need a flag that says "Render me (in the next frame)", so that the user code can choose to have registered meshes omitted from rendering.
  • The renderer would be able, for example, to sort all meshes by material before rendering, or implement any other "global" optimization as it sees fit.
  • The renderer might even be able to render all stencil shadows on it's own, without requiring user code for computing the details. Again, each mesh would need a "this mesh casts stencil shadows" flag to turn this feature on and off.
  • More generally, all rendering phases (ambient, stencil shadows, lighting) would be handled by the renderer, not affecting the user code any more at all.
  • An open question is how each mesh would be positioned. Right now, user code directly modifies the relevant transformation matrices via the related RendererI methods before calling RenderMesh(). We would have to figure out if instead - for example - each mesh should have its own model-to-world positioning matrix, and/or a pointer to a "parent" mesh (e.g. the weapon model that another player holds is positioned relative to the player mesh), or something entirely else (e.g. callbacks?).
Well, this is a rough overview of my ideas and suggestions about it.
What do you think?
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-09, 16:43

Handing over complete control of meshes to the renderer, and letting the renderer render the meshes at it's own discretion, like you describe, is probably the best idea. It's more or less what I had in mind. I'll try and answer/respond to your bullet point list.
  • Visibility control from the user code is a good idea.
  • Letting the renderer go about rendering meshes in it's own way is a good idea too, since in different graphics APIs it's often better to do things in a slightly different order or method.
  • Shadows are kind of a tricky one. In my opinion, it should be up to the renderer how it does lighting and shadows at all - a certain renderer might not want to implement stencil shadows at all, but rather some other technique. For example, if a renderer decides it wants to do all it's lighting using a deferred shading technique and shadow mapping, as opposed to forward rendering and stencil shadows, this should be entirely possible.
  • Answered this above - the renderer should be able to render the scene in any way it likes, independent of the user code.
  • A good way of handling transformations is having a projection-view-world matrix setup. This is how XNA works. The projection is obviously the projection matrix, the view is the camera transformation matrix, and the world is the object specific matrix. The modelview matrix is composed of both the view matrix and the object matrix. Coordinates would go in this order: Object space -> world space -> view space -> screen space. The modelview matrix can be calculated in software, and the transformation (proj * mv * vertex) is performed on the GPU. Hence, each mesh instance needs it's own world matrix.
Whilst typing all this, I realised that as well as the renderer allocating meshes, it also needs to allocate mesh instances - suppose you have 3 players with the same model. It would be a big waste to create 3 seperate mesh objects containing an identical set of vertices.

As for animations. Are all animations in Cafu bone-based? If so, it may be possible to integrate a bone system with the mesh object, and let the renderer handle the details of the animation itself.
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-10, 01:35

Well, ok, having the renderer control all rendering seems to be the right approach, but please note that we should keep this change entirely (or at least as much as possible) separate from the changes required for VBO support -- this makes things much easier to understand, develop, debug, etc.

About shadows, well, there is another school of thought: I'm not sure if the renderer should make the decision about the shadow technique, because it seems to me that stencil shadows and projective shadow maps can/could exist next to each other (that is, both be used at the same time in the same scene, if desired). So alternatively, it would be possible to just have the user choose if he wants to have a mesh cast: no shadows, stencil shadows, or projective shadows.

The transformation: Well, yes, I'm very aware of the matrix setup. From class RendererI:

Code: Select all

  enum MatrixNameT { MODEL_TO_WORLD, WORLD_TO_VIEW, PROJECTION /*VIEW_TO_CLIP*/ };
However if the renderer is in control of rendering the list of all meshes known to it, how do you tell it that some of these meshes have matrices that are different from others? I'm sure there are good solutions to this problem, but from the current abstract point of view, I cannot quite see which one would be the best.

I don't quite get your point regarding "mesh instances": It's not up to the renderer to check for duplicate meshes. Instead, e.g. our new (already finished) model code goes to great length to ensure that the same resource is only loaded and allocated once.
Also note some conflict of interests: "When the same model appears in the scene several times but at different poses, should each model have it's own mesh, or should the models share a common mesh that is updated and re-used for each pose in turn?"

Final point, about animations: While "vertex-skinning" (computing the mesh vertices in the vertex shader on the GPU) seems to be a much-loved topic nowadays, I don't think that we should try to integrate this as well (at this time). This is mostly because it sacrifices quasi all flexibility.
For example, I'm right now working on animation channels and blending, which is very well organized and encapsulated right now, and moving it into the renderer would essentially break all this and cause chaos. Even if it worked, what if we wanted to add inverse kinematics at a later point? Again this would break everything. It's similar with the implementation of swaying branches of trees and lots of other subtle features.
In summary, I'd rather continue to have the user code compute the mesh, and have the renderer render them. :-)
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-10, 14:27

The thing I was getting at about mesh instances (I'm bad at explaining things like this), is basically this. The mesh object should contain vertices and nothing else. It should literally just be the data required to draw a model. No matrices, nothing todo with how the model is used, just raw data. In mesh objects, you store all the models the game requires, as well as the world etc. But how do you tell the renderer to "draw 3 of this mesh, one here, one there, and one over there, oh and make the 3rd one red, and the 2nd one not have shadows"? You'd need some kind of other object which the renderer controls, which contains a pointer to the mesh, as well as colour data, transformation matrices etc.

So the user code asks the renderer for a mesh object. The renderer creates the mesh object, stores it away for later use, and returns it back to the user code. The user code populates it with data from file - lets say a player model. Now, a player connects to the server - so a new player needs to be renderered. The user code asks the renderer for a mesh instance (needs a better term?). The renderer creates the mesh instance, stores it away for later use, and returns it back to the user code. The user code sets properties on this mesh instance, such as the mesh it points to, what colour to shade it, the transformation matrix, whether or not shadows are turned on etc.

Now, it's time for the renderer to draw. It will loop all mesh objects, and see if there are any updates it needs to process and transmit to the GPU (this is where VBOs come in). It will then loop all mesh instances, prepare the graphics state to draw that instance, go get the VBO IDs from the mesh object that the instance points to, and draws it.

This is essentially the system I'm visualising at the moment, and would probably be the best system for me as a renderer programmer to work with.

As for animation, I've not had that much experience with animation programming so I can't really say much about it - you know more than I do in this case, and you know the bottleneck is pushing new vertices to the GPU and this should be minimised as much as possible.

In the end, it's your engine - I've never written or maintained a full engine before so I'm not exactly in a position to tell you what to do or anything, and I can understand you not wanting to commit big changes everywhere just for GL3.x support :wink:
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-10, 23:49

Hi Thomas,

your description of meshes vs. mesh objects/instances is a very good one, and the point you're making is obviously right!

When I read your description, the term "mesh object" made me think of a scene graph, because we can use matrices to hierarchically position meshes in the coordinate systems of other meshes (/ mesh objects).
The term "scene graph" is unfortunately pretty overloaded already, but I think we could still name these things e.g. "mesh nodes". Each mesh node would have a list of meshes (the vertex data), a set of matrices (model-to-world, world-to-eye, projection), boolean flags (they would affect all meshes in the list), and a list of child mesh nodes (if the user e.g. wanted to set a different flag only, these matrices would be the identity matrices (if we choose to inherit them) or those of the parent (if we choose to not inherit)).

(It would also be possible to not design mesh nodes hierarchically. In this case, we could simply call them "scene", which would just be a list of meshes with attributes.)

About the strategy, commits, etc.: Well, it's not my intention to exercise overly much control over everything. :cheesy:
It's just my goal to have things well planned, and to break large changes such as those we discussed above into small steps. It's ideal if each of these steps is a separate patch that is reasonably small, concise, and easy to understand. Then, it's also well possible to review each such step/patch, determine whether it's correct, see if the next step can be started, etc.

For example, all of what we have discussed above falls at least into two big steps:
  • Adding VBO support
  • Refactoring the code to have the renderer control the rendering
I think that these two steps are independent of each other, can be done in any order, and each has to be refined into sub-steps. As long as we agree on the steps that are to be taken / the design / the masterplan, I'm open to anything. :-D

If I had to do this all by myself, I'd probably start with reading the VBO extension spec, browsing through the code to get an up-to-date picture of the most important things that need to be accounted for (whatever comes to my attention), then try to come up with the steps to implement it.

Sorry if this all sound a bit abstract. In short, if you can avoid sending me huge patches that change the world and that I don't understand, everything will be all right. :up:
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-11, 16:18

If it helps, I can describe the requirements of the VBO implementation, it'd save you spending hours looking through complicated specs and such like. I don't know how far your graphics programming knowledge extends, so I'll go over everything in a bit of detail.

Basically, I (the programmer) ask GL for a VBO, and I tell it what kind of VBO it is (does it store vertex data, or index data?). In this case, since the meshes don't work with indices at all, we'd only use vertex VBOs. GL comes back with an int, which is a unique ID for that VBO. Now, for GL3+, the way it works is this. For each "channel" of data, I have a seperate VBO. For example, I might have a VBO for all the positions, a VBO for all the normals, a VBO for all the colours etc. So each mesh can have an arbitrary number of VBOs assigned to it.

Now, for each "material", I create 2 shader objects (1x fragment shader, 1x vertex shader), compile them, and link them to a program object. This ties it neatly with the existing shader system that Cafu uses, and in fact I already have such a system working. The shader language I've used is GLSL. Now, there's yet another object called a VAO - a vertex array object. Again, represented by an int. This describes to GL how each VBO binds in to the shader. For example, in the vertex shader, I might have 3 input attributes called "inNormal", "inPosition" and "inColour". The VAO tells GL "hey look, I want this VBO here bound to inNormal, that VBO there bound to inPosition and this other one to inColour". Hence, each mesh will have just 1 VAO assigned to it. Drawing is nice and easy - bind the shader, bind the VAO, and tell GL to draw - it does the rest.

The nice thing about this system, is all attributes are generic and optional. If I want to add in UV coordinates later, it's easy - add a new VBO containing UV coordinates, and tell the VAO to map it to "inUV". Or I could have a VBO for some other thing which the fixed function pipeline wouldn't contain, like binormals or tangents or something. So it's really quite flexible.

As for matrices, they're set as uniforms - which are like globals to the shader. The vertex shader uses inPosition (or whatever you like really) and the uniform matrices to do the transformations.

So basically, each mesh needs to have an int for the VAO and some kind of array of ints for the VBOs. Obviously it would be a bad idea to just go ahead and add those straight to MeshT, since only the GL3.2 renderer will use it, nothing else - this is where the abstracted mesh interface comes from. The renderer would implement it's own mesh class inheriting MeshT, and add in the ints itself, along with convenience functions for building the VBOs etc.
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-11, 22:59

Wow, thank you very much for the overview about VBOs, it was very helpful!

The kind of problem that I sometimes have is that I used to be very deeply dug into (in this case) OpenGL at the time I wrote the currently existing renderers, but when they were finished, I went on to other topics in the Cafu source code world. When then enough time goes by, I have to re-familiarize myself with everything again, especially if - like in this case - the subject itself (OpenGL) evolved and underwent huge changes in the meanwhile.

So I'm quite familiar with OpenGL up until roughly 2.x, with NVidia's Cg, and with reading the specs of interesting extensions, but I'm not (yet) up to speed with the latest 3.x+ versions of OpenGL (and unfortunately it seems that neither the 7-th edition of the Red Book nor the Orange Book help with VBOs or any new parts of OpenGL 3.x+).
So while I find you description very helpful, it is probably a good idea if I read the text about the details, too.
Are you using http://www.opengl.org/registry/specs/AR ... object.txt, or something else?

Btw., something else: I'm considering changing the build scripts, for linking everything dynamically to the C/C++ runtime libraries under Windows as well (are you using Windows??). Right now we're linking to the C/C++ runtimes statically (under Windows only), which makes binary distributions easy (no need for an installer for making sure the users system has the proper runtime libraries installed), but there always was and still is the horrible problem of having multiple copies of the CRT in one program (the DLL Boundary).
I think this change will make our lives a lot easier (I've spent already too much time with related issues in the past), and will probably make the change early next week, or maybe even sooner. To let you and everyone else know, I'll make a separate forum post about it when done, with any additional information.
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-12, 01:36

To be honest, I've only worked with OpenGL in a managed environment (C# + OpenTK), so all the nasty extension stuff was done for me - I could forget about spending hours making a window appear and loading extensions, and get on with coding OpenGL stuff. But that spec you linked looks correct, and if you cross-reference between it and what I wrote, you'll notice we both said the same things (even though I've never seen that spec before - I don't like reading specs much, I prefer reading actual documentation :wink: ).

Since I'm more of a C# programmer, I don't really know much about the linking/compiling process of unmanaged code so I can't really advise there. Dynamic linking has never played ball with me during my attempts, so if you can get it working them great :cheesy:

I'm building on Windows 7 btw, using VS 2010.

Edit:
Oh the other thing, that spec talks about vertex array object in the notes at the bottom. Just to make sure there isn't any confusion, a vertex array object (VAO) does exist and is not the same as a vertex buffer object (VBO). It's all confusing stuff :wink:
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Renderers and Meshes

Post by Carsten » 2011-11-12, 21:48

Hi Thomas,

thanks for the info!

I've already implemented the change regarding dynamic linking, please see http://trac.cafu.de/changeset/420 for the details (you have to run the rd /s ... commands if you upgrade).

What OpenGL 3.2 documentation (if not the specs) do you read?
If you have already begun with VBO code, what OpenGL loading library do you use? GLEW?
Best regards,
Carsten
thomasfn
Posts:14
Joined:2011-11-07, 22:48

Re: Renderers and Meshes

Post by thomasfn » 2011-11-13, 00:48

I haven't started VBO code, but I have started the shader code - I already have GLSL compiling (can't remember if I mentioned that or not). I'm using your OpenGLEx thing you used for the other GL renderers, I've just added all the other extensions I needed to that. It's kind of a laborious task though, so I'm tempted to use GL3W or something.

As for OpenGL documentation, I just picked up what I know from various tutorials and code examples around the internet rather than a central source. A good reliable OpenGL3+ tutorial which covers everything is difficult hard to find, so cross-referencing is required.
Post Reply

Who is online

Users browsing this forum: No registered users and 24 guests