Database connections

[PatM]

Hi! Just found Cafu and I'm about to go diving in this weekend. From what I've seen it looks like Cafu has pretty much everything required for a non-persistant FPS style game and that's fantastic, particularly since its GPL. However, I want to work on a persistant RPG (MO, not MMO - nothing massive about what I want!). The basic idea is a 3D multiplayer RPG scaled for a home based server that will probably never see more than 30 players (depending on the server's network bandwidth in each direction etc). I've played and loved MMORPGs for many years, starting with Meridian 59 back in the 90s - before I ever heard of Ultima Online or Everquest.

While I loved the MMOs I always wanted something that I could run at home and ... "adjust" to suite my play preferences (e.g. faster or slower advancement, more or less powerful items/skills etc). The closest product I've ever seen was Neverwinter Nights and it was fun in it's own way but I've never liked D&D rules and it was a 2.5D world which doesn't fit my ideas.

I realize that Cafu has no built in features for inventory or quests or other RPG style features but I can script and if absolutely required I can program in C++ to some degree so I'm looking forward to experimenting with Cafu (I've seen some forum posts about inventory etc which look promising).

What I haven't seen is anyone implementing a connection to an external detabase. There have been a few general questions without definitive answers but the questions were somewhat vague so the answers couldn't be very definitive. I'll try to be clearer

What I'd like to know is if it's possible to make a database connection (Say MySQL or PostGres) from Cafu scripts or would some C++ coding be required to allow connections and more C++ coding required to allow scripts to use that connection? Security on those connections, of course, would be my responsibility.

Also, can scripts access entity property values i.e. if I add a Strength property can a script set/retrieve that value?

My intent would be to save character stats, character inventory, character quest states, game entity states and locations, those sort of things required for a persistant RPG world. Of course I realize I have to implement the systems for the above, I just want to know if I can access and save all the data to a database.

P.S. I know scripting is based on Lua and Lua can indeed connect to many RDMS but being based on Lua doesn't necessarily mean being able to do everything that stand-alone Lua can do.

Thanks!

[Carsten]

Hi PatM,
welcome to the Cafu forums!

What I'd like to know is if it's possible to make a database connection (Say MySQL or PostGres) from Cafu scripts or would some C++ coding be required to allow connections and more C++ coding required to allow scripts to use that connection?

At this time, no support for databases is built into Cafu. That means that definitively some C++ code is necessary to add it. While I'm relatively familiar with database programming from my PHP (website) and Python (Django-based) projects, I don't know ad hoc how connecting to a database is done in C++, but I'm sure that there a ready-made libraries for this.
A separate coding step is necessary to allow scripts to use that connection, which essentially consists in providing the proper script bindings to whatever database library has been used in C++.

However, I honestly wonder why you're looking for database support at all? Assuming that we have a central server that is authoritative for a game world, how is saving game state in a database different from saving game state in a file? (For completeness I should add that I know very little about persistent RPGs...)

Also, can scripts access entity property values i.e. if I add a Strength property can a script set/retrieve that value?

Yes, of course.

Again for completeness, let me add that Cafu is currently undergoing massive changes regarding scripting. See thread More about Component Systems for details. My reply above, "Yes, of course", relates to both the "old" as well as the "new" situation, but the new situation everything is really a lot better.

Last but not least, we should consider how we obtain the data that describes the persistent state, i.e. the very data that is saved to a file or a database: The key idea is to implement a serialization / deserialization interface for all game entities and their components. This is already implemented in the new Component System, and also used for sync'ing data over the network in the live game. Although the same technology could also be used for a "save game" feature (and to record games as well), this is not yet implemented -- but it should be easy to do.

(There are also inherent limits. For example, if you capture the game state immediately after an explosion has occurred, in the captured state the explosion itself is not recorded, because it took place a small amount of time before the capture was taken. But the smoke and other lingering after-effects of the explosion -- may not be captured as well, because effects such as smoke and other graphical details may only be available on the client systems as an immediate consequence of the preceding explosion event that triggered the effects.)

My intent would be to save character stats, character inventory, character quest states, game entity states and locations, those sort of things required for a persistant RPG world. Of course I realize I have to implement the systems for the above, I just want to know if I can access and save all the data to a database.

As described above, yes, I think that that works. All that leaves me unsure is that I cannot see the difference between a database and a file in this regard.

[PatM]

For an FPS, saving game state in a file makes sense because there isn't much information to save. In most cases saving anything other than player standings doesn't make sense at all.

in an RPG it's not just about saving game states between games it's about accessing and changing information on the fly. The game itself would not shut down at all, just periodically save information about the players and the world.

Another reason to use a database is for run-time modifications to the world. You don't want to have to re-compile programs or scripts if you want to add or modify a spawn, quest, or whatever.

There's also the sheer scale of information involved. An FPS might have a couple dozen weapon spawn points which spawn items from a small list of non-varying weapons and gear. An RPG world likely has at least hundreds of spawns for at least dozens of monster types with dozens of variants for each type. Instead of a dozen or two items for an FPS an RPG will likely have hundreds of items at a minimum. You could use files but you'd have to write your own RDMS to be able to manage everything anyway and the established ones are a heck of a lot better at it than anything I could write.

Another benefit of database vs files is speed of access. Databases handle caching of data while files do nothing special unless you write the code to do caching of frequently accessed information. Doing an SQL query drawing from several tables (i.e. to generate loot on the mob someone just killed) would probably be at least 100 times faster that me writing the code to accomplish the same thing. Even if I just used a flat data scheme I'd still be a lot slower using random access text files than an established RDMS.

My first foray into databases was with PHP and Postgres waaaay back doing a purchasing and inventory control system for myself (I was supervising several sites and paper based was getting insane). I've done a bunch of other things with MySQL and SQLite as well. It's lots of fun playing with those.

I've also played with MMO Emulators for Everquest and WOW and got to appreciate how beneficial it is to have a real RDMS to handle organizing all the information. I couldn't even begin to see how to do the same with just regular files.

As I said above, Lua has sql functionality through things like http://www.keplerproject.org/luasql/ and I'm wondering if Cafu's Lua implementation could use these for database connectivity?

[Carsten]

Hi PatM,

[...]

I've also played with MMO Emulators for Everquest and WOW and got to appreciate how beneficial it is to have a real RDMS to handle organizing all the information. I couldn't even begin to see how to do the same with just regular files.

As I said above, Lua has sql functionality through things like http://www.keplerproject.org/luasql/ and I'm wondering if Cafu's Lua implementation could use these for database connectivity?

Thanks for your detailed explanation!

I think that it would indeed be possible to integrate LuaSQL with Cafu, and the integration would probably not be particularly difficult to implement.

However, there is still Cafu's own “view” of a game world, i.e. the datastructures used and the (script) API exposed to access and change it. On the other hand, there is a database connection. I honestly have no idea how these two would best be combined… I don't think it's easy, and it would probably take some dedication (that is, some time and efforts) to achieve it.