Happy New Year!

News about the Cafu Engine. Subscribe to the related ImageCafu News feed to stay informed.
Locked
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:
Happy New Year!

Post by Carsten » 2011-01-01, 17:41

I wish you a happy New Year, and all the best! :groupwave1:
P1012141_800.jpg
The New Year fireworks at my home town.
We've been busy working on Cafu even over the Christmas and New Years holidays, making very good progress with bug fixes and many enhancements, but the efforts in the first few months of the new year will certainly be focusing on the upcoming Model Editor, for which the work is progressing very well!

I've largely completed a loader for the Wavefront Object (.obj) file format the day before yesterday: "obj" is a very common file format that most 3D modeling software can load and save, and as such it allows us to exchange (static, non-animated) models with almost any other modeling program.

Regarding collaboration with other 3D programs, we also need support for a similarly universal and popular file format that can also handle (key-frame or general) animation.

This is where I'd love to hear your feed-back:
Which 3D model format that supports animation should the upcoming model editor be able to import?
The format should ideally be widespread and supported by several or most of the well known 3D programs (but of course nothing stops us from having the Model Editor support for importing multiple such file formats, or to add more formats over time).
Please post your comments as replies to this post! :up:

In the meanwhile, we continue our work on the list of open tickets and Project Ideas -- and much else that is not (yet) mentioned in the lists! :wohow:
Best regards,
Carsten
scott
Posts:173
Joined:2004-08-23, 09:11

Re: Happy New Year!

Post by scott » 2011-01-02, 02:24

Happy new year!!

Good to hear about the .obj loader. Will we be able to export map geometry out of the level editor as well? This just makes it handy if you can bring stuff in to your modelling app for reference (Scale, spacing etc etc)

With the model editors formats, I'd say my first choice would be Collada .dae, Just about every package has a plugin for it, and its used in games. .fbx may also be a good choice, a lot of apps have plugins for it, not sure if any games actually use it.

Another alternative would be the Ogre formats,
http://www.ogre3d.org/tikiwiki/tiki-ind ... +Exporters
theres a lot of exporters to use.
Rich
Posts:5
Joined:2010-12-24, 19:43

Re: Happy New Year!

Post by Rich » 2011-01-03, 20:03

Happy new year to you too :)

Yeah I'm really looking forward to the model loader.
I think .fbx or .X could be nice formats to support.

Hope the new year is great for you all :)

Regards,
Rich
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Happy New Year!

Post by Carsten » 2011-01-03, 23:52

Hi Scott, hi Rich,

many thanks for your replies!

I looked into each of the formats that you suggested (in addition to some preliminary work with Collada several months ago), and they all look pretty good. :cheesy:
  • About Ogre, while I have great respect for the project, I'd prefer to focus first on file formats that are more "neutral", and for which the licensing terms are as light as possible. I'd be open though to add a loader for Ogre if someone contributed it, or maybe add one later myself, but right now let's consider the alternatives first.
  • FBX seems to meet the criteria quite well, however I still have to download and familiarize myself with the SDK to check the details. Key questions are the terms of the related license agreement, and how "accessible" the related source code is (e.g.: Can we redistribute it with Cafu? Does it work on Linux? and so on). I'll make sure to check it out though as soon as I can.
  • Finally, Collada looks very promising for the obvious reasons: open, well specified, widespread application support, etc. I experimented a bit with the Collada DOM about a year ago, but then something else cropped up and I never finished. In addition, the Collada DOM is not exactly lightweight, but I'm very motivated to resume and finish the previously begun work. :up:
    (By the way, there is a nice and gentle prose introduction to Collada: http://thecansin.com/2010/10/collada-lo ... r-directx/ )

In summary, I guess that I'll just add support for loading Collada models first, then see what else we still need.
In this regard, I'm currently also pondering over MilkShape, which can read and convert a huge number of file formats. If the new Model Editor can read the most important formats that the large Digital Content Creation programs create, and MilkShape could be used for converting any "existing" models, this would cover a lot of ground already -- what do you think?
Will we be able to export map geometry out of the level editor as well? This just makes it handy if you can bring stuff in to your modelling app for reference (Scale, spacing etc etc)
Ahh, yes. Iirc, Kai asked for Obj export for the Model Editor as well.
Could you please open a ticket for it (covering Obj export for both the Map and Model Editor)?
Best regards,
Carsten
scott
Posts:173
Joined:2004-08-23, 09:11

Re: Happy New Year!

Post by scott » 2011-01-04, 08:11

http://trac.cafu.de/ticket/57

Ticket submitted


I had no idea that milkshape was still being developed, its been years since I've seen it.

Are you thinking of buying a copy to convert the current models over to the newer formats? If so, I dont see why not.

But beware supporting too many formats, thats a lot of code for you to maintain ;)

Collada and .obj will cover 90% of whats out there for now.
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Happy New Year!

Post by Carsten » 2011-01-04, 15:19

scott wrote:http://trac.cafu.de/ticket/57
Ticket submitted
Thanks!
Are you thinking of buying a copy to convert the current models over to the newer formats?
Not really, as I "converted" the existing, current mdl loader code into a proper, "native" loader/importer for HL1 mdl files for the Model Editor. That is, I'll soon use the Model Editor with the existing models in mdl format to convert them to the new Cafu-specific cmdl format -- this is also a very good opportunity for me for initially "testing" many aspects of the Model Editor: If this conversion works, it's a good indication that its most important concepts and features are complete and working.

MilkShape I had in mind as an "external resource", that is, users who have models in a format that we don't support natively could be referred to MilkShape for conversion into one of the formats that the Model Editor can eventually read.

Using MilkShape should rarely if ever be necessary for authors who create entirely new models, as they would be able to save them as Collada, FBX, Obj, etc. (something the Model Editor can load directly)

But models for which the source files are no longer available or were originally made for a different purpose could be converted with MilkShape into a format that can be loaded into the Model Editor.
But beware supporting too many formats, thats a lot of code for you to maintain ;)
Yes, MilkShape occurred to me when I was thinking about how the number of supported formats can be kept at a reasonable minimum, and I hope that the above outlined "split" of cases (original content authors use e.g. Collada for direct import, foreign formats are converted with MilkShape) makes sense to meet that goal. :-)
Collada and .obj will cover 90% of whats out there for now.
Yes, that's very good, all the more reason to add the Collada loader soon. :D
Best regards,
Carsten
Rich
Posts:5
Joined:2010-12-24, 19:43

Re: Happy New Year!

Post by Rich » 2011-01-05, 19:49

Cool yeah, I think collada and obj would do the trick nicely.

From an engine user point of view, its easy enough to convert files to either of those formats too so importing models would be easy to do :)
User avatar
Kai
Posts:177
Joined:2004-08-19, 15:56
Location:Germany
Contact:

Re: Happy New Year!

Post by Kai » 2011-01-26, 05:34

Im adding my thoughts to this:

First of all the model editor will act as central tool to combine and manage asset components (mesh, textur, shader, animation file, scripts) into a native, internal cafu format.
Regardless of which format it would be, it should transport relevant information from a 3d package into the Cafu Model Editor where it will be further processed and de-cluttered of unneeded data.
  • Ideally it should allow for import of as much information as possible:
  • Mesh, uv channel and textures for static components
  • Skinweight, skeleton structure and animation sequences for animated models.
Additionally, physical attributes, LOD Sets, and collision meshes should also be supported.

Export is much less important, textures would be already accessible, shaders have no need for export. Animation may be complex and you better ask for the original file, which would be more accurate, same goes for weighting of skin clusters. Would be nice to have it later but i have no real need for it.

But i would like to see an export ability for bsp elements, including multiple UV channels (color, lightmap channel) so one could export a bsp group and use external renderer software to generate lightmaps, then import it back into cafu.

Regarding the format:
Collada is open scource, made by sony, so its pretty big and accepted as modern standart for fully featured assets (mesh,skin,joint structure, animation data...). I found that most Autodesk packages are having more and more serious trouble of updating good plugins for collada's .dae export. Reason for this could be that Autodesk would like to promote its own fbx format.
However there is a converter (on autodesk site) for "fbx to dae" which is said to be much better than exporting dae out of-the-box.
I have no experience yet using it, but heard that it seems to be quite good so far, and can also contain physic informations.

FBX is property of Autodesk, therefore almost any of their packages support it natively (Maya, 3dsMax, XSI, CAD ..) and can contain a lot of information, like collada. Sometimes its a bit naggy about scaling between packages but it retains all data as it should.
Some packages like blender use a (python) script for working with fbx.
Autodesk provides a nice SDK
http://usa.autodesk.com/adsk/servlet/pc ... eID=123112

Personally i had not much problems using fbx, but never came to the point of using collada. Both formats act as interchange format, so they carry all of the relevant information. But both should be processed for perfomance issues either before exporting or after importing into the model editor.


Overall i read through many forum threads and found out that there is still a split between both formats ;)
In this (rather new, November 2010) game developers blog, the author explains why they drop collada and prefer fbx, as it seems to be more accurate: http://www.mobilebits.de/Blog/post/2010 ... r-FBX.aspx
However scoll down and find a comment of some Okino coder (Okino polytrans is THE software for converting almost any format into any another), who claims that collada is a
much better format than FBX (few people really know the difference at the core technical level)
. This conversations extends further down so might be a good source to read.

It's hard to say which one is better until different exporter for different packages are tested for how they maintain the relevant information.
Another alternative would be the Ogre formats,
http://www.ogre3d.org/tikiwiki/tiki-ind ... +Exporters
theres a lot of exporters to use.
I heard the same, ogre's mesh and skeleton files are said to be pretty good and already have mature exporter for almost any valuable package. Maybe worth a try..
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Happy New Year!

Post by Carsten » 2011-01-26, 20:13

Ok, many thanks to you and everyone else for your input!

I've spent the afternoon with getting familiar with the Autodesk FBX SDK, which is looking pretty good but doesn't come with source code, and with some additional research about file formats in general.

In summary, I think that the best strategy is to proceed as follows:
  • Let's first complete the support for Collada, using the OpenCollada library. I started using it a while ago, then experienced some problems (but much smaller ones that I had with Collada DOM a few months ago), but today also found how these problems can be overcome, so things look very promising for Collada right now.
  • In the next step I'll be happy to add an FBX loader as well. Their (official?) FBX SDK FAQ makes is clear that the source code is not available, and that we cannot bundle the FBX SDK with Cafu in any way. Thus,
    • our source code distributions (and especially the build scripts) will use the FBX SDK if it is there, but happily continue to work when it isn't. That means that the FBX SDK will be optional: source code users that install it will get FBX support built in, otherwise support for FBX will be missing but everything else will work as normal.
    • our binary distributions will, once done, have FBX support built in, we'll get and install the FBX SDK for all our development platforms.
  • You're certainly right regarding your assessment of Ogre. I agree that it is worthwhile to support, too, but right now I'd like to leave open if we'll add support for it before or after we finish the Model Editor. (As usual, your feedback has the power to change our priorities. ;) )
Kai wrote:[...] and use external renderer software to generate lightmaps, then import it back into cafu.
Hmmm. An interesting thought!
(Can you please file a ticket for it?)
I found that most Autodesk packages are having more and more serious trouble of updating good plugins for collada's .dae export.
Yes, I think this one of the central issues is what the OpenCollada team is determined to fix.
But both should be processed for perfomance issues either before exporting or after importing into the model editor.
Yes, the Model Editor importers will clean-up meshes as they see fit, and if reasonably possible, even without user intervention.
Overall i read through many forum threads and found out that there is still a split between both formats ;)
In this (rather new, November 2010) game developers blog, the author explains why they drop collada and prefer fbx, as it seems to be more accurate: http://www.mobilebits.de/Blog/post/2010 ... r-FBX.aspx
However scoll down and find a comment of some Okino coder (Okino polytrans is THE software for converting almost any format into any another), who claims that collada is a
much better format than FBX (few people really know the difference at the core technical level)
. This conversations extends further down so might be a good source to read.
Yes, thanks, I've seen this before, but even having read it all, it's hard to form an opinion on which is "better", thus I guessed it would be reasonable to support them both.

So in summary, it turns out that Collada and FBX (and Ogre) are the most relevant file formats, and even better, from today's perspective it seems that the effort to implement each does not require us to choose one in favor over the others, but allows us to eventually support them all. :up:
Best regards,
Carsten
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Happy New Year!

Post by Carsten » 2011-01-29, 14:31

Just a quick update with very good news:

I started working with the FBX SDK, and it turns out that it is very well and carefully done. It comes with good documentation and is relatively easy to use.

Most interestingly, the FBX SDK supports, besides FBX itself, also the DXF, 3DS, OBJ and (can you believe it?) Collada DAE file formats!
That is, at no extra effort we get support for all these, and I've now changed my priorities to finish the FBX importer first. :cheesy:

Also see our new (C++ developers) documentation section Using the Autodesk FBX SDK for details. :book:

:up:
Best regards,
Carsten
User avatar
void
Posts:8
Joined:2011-01-23, 23:09
Location:Germany

Re: Happy New Year!

Post by void » 2011-01-30, 20:29

Great news!

A good working art pipeline is probably crucial for any engine.

Here is an interesting article on that topic on Gamasutra: The All-Important Import Pipeline
As the name suggests, the article's author prefers an import-based pipeline to an export-based pipeline, too.
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Happy New Year!

Post by Carsten » 2011-01-31, 17:50

void wrote:Here is an interesting article on that topic on Gamasutra: The All-Important Import Pipeline
As the name suggests, the article's author prefers an import-based pipeline to an export-based pipeline, too.
Interesting, thanks for posting this!
Best regards,
Carsten
Locked

Who is online

Users browsing this forum: No registered users and 5 guests