Migrating Cafu to distributed version control – Part 1

News about the Cafu Engine. Subscribe to the related ImageCafu News feed to stay informed.
Locked
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:
Migrating Cafu to distributed version control – Part 1

Post by Carsten » 2012-10-15, 09:49

For the Cafu Engine project, we have almost from the very beginning used Subversion as our version control system, and Trac as the bug and issue tracker. Both systems have served us very well in the past, and in fact until today there is nothing fundamentally wrong with either of them.

However, over the years, the requirements of developers and users have evolved, as has the Cafu Engine and available version control systems. Increased flexibility and ease of work for everyone who participates in the Cafu Engine are more important than ever.

With DVCS – distributed version control systems – being widely spread and well known today, most prominently Mercurial and Git, and with related hosting and collaboration platforms such as Bitbucket and Github readily available to everyone, it's time for change.

In this post I will summarize the current state of affairs regarding the migration of Cafu to Mercurial or Git.


Why DVCS?

Why should we migrate from Subversion to Mercurial or Git at all?
What are the goals?
  • First and foremost: Our users use the Cafu source code to develop their own games and applications, but would also like to integrate the latest enhancements and bug-fixes from the official Cafu code base. Keeping up-to-date by applying the latest changes of Cafu into your own code base can be cumbersome with Subversion, but is easy to do with Mercurial or Git.
  • Similary, developing with and contributing to Cafu should be easy. Anyone who wants to contribute an enhancement, a bug-fix or a new feature should be able to do so in the simplest manner possible.

    This will later be described in greater detail in the wiki, but in brief, the primary method would be to create a pull request at Bitbucket. Fallbacks and alternatives would be posting pull requests at the issue tracker or the mailing-list, or to submit patches by uploading them to the issue tracker or sending them to the mailing-list.
  • Better branching than Subversion. Branching is a vital tool to us even with Subversion.
    • We use a special flavor called "vendor branches" to manage our external libraries.
    • In our website, we use vendor branches as well, for separating "live" from "pristine" content.
    • Last but not least, the use of branches in Cafu as "feature branches" was existent, but underutilized. Employing a more sophisticated branching model like the one described here will only be possible with Mercurial or Git.
  • Implicitly contained backups. As everyone always has a full copy of everything, the worries about creating backups for the repository are virtually eliminated. This is important for our actively used Cafu source code and website repositories, but also for legacy repositories that we no longer use, but still keep around.
  • Repositories that are no longer in use and have been archived require no special measures to inspect, use or even revive them at any later time. No server is necessary to do research and archeology even in long archived repositories.

Mercurial or Git?

The internet is full of related debates already. As I do not want to start another one, I'll be brief:

In my opinion, Mercurial is easy to learn, well documented, and powerful enough for 99% of all use cases and users. For new projects with no pre-existing history, Mercurial is my clear favorite.

Git is technically more sophisticated, but comes with a horrible command-line interface. These days it is easier to learn than it used to be, but developing a proper mental model and becoming a confident (admin) user still takes a very big effort.

A fair and competent opinion about Mercurial and Git was expressed by Sinbad, the founder and former lead of the Orge3D project. I much agree with his points in this discussion. (Continue reading on the subsequent pages 13 and 14, where he also links to his detailed review and comparison.)

Besides studying all the existing, prevalent sources, during the last few weeks and months I also spent huge amounts of time with evaluating both Mercurial and Git myself: The minimum that I wanted to accomplish was gaining a good understanding of each system, import the existing Cafu Subversion repository, and run some real-live tests that resemble daily routine Cafu development.

My opinion about Git:
  • Regarding the negative aspects, I have to agree with Steve Bennett's blog post "Ten things I hate about Git". And he forgot to mention that Git under Windows only exists due to a small group of volunteers who are not involved in Git core development.
  • BUT: Git is widely supported by web hosters. All hosters that I have a contract with (LCube and 1und1) now support Git without any further ado (none of them supports Mercurial).
  • AND: Git has with git-svn a very good interface to Subversion for both parallel use or one-time conversion.
Despite my original preference for Mercurial, I must say that I spent more time with Git (maybe because it took me three printed books to get a good understanding of it), so probably in the end I was more biased than I now like to admit.

As we will see later, the initial importing of the Subversion repository turned out to be the crux. After a long battle, it turned out that I was able to complete the conversion to Git in exactly the clean, complete and careful way that I wanted, whereas my attempts at converting to Mercurial were unfortunately much less fruitful. Things had probably gone better for Mercurial if I had invested more time into it, but then I have still no idea how I could have done with Mercurial what I did with Git.

With this experience, I eventually chose Git to proceed with the next steps.

However, I'm confident that importing the now clean Git repository into Mercurial is much simpler than the original attempt of importing the Subversion repository into Mercurial, so the door to Mercurial is still open.


The next steps

After all of the above, I was ready to ponder the concrete next steps that must be undertaken to actually migrate the Cafu Engine to Git.

But as this matter in all its details is very complicated and involved, I finish this post at the overview level.

In my next post, I'll provide more details on the requirements and goals of the conversion, and why it was so difficult to implement. Things will get more technical (and practical) then, and I'm going to provide the exact steps on how the conversion was eventually implemented. As the bug and issue tracker is closely integrated with Subversion, we'll have to have a look at it, too.


Please let me know what you think, I'm looking forward to your feedback! :up:
Best regards,
Carsten
HWGuy
Posts:95
Joined:2011-05-31, 07:37

Re: Migrating Cafu to distributed version control – Part 1

Post by HWGuy » 2012-10-15, 20:46

Mercurial or Git, either way it's a good change, they're both better than SVN and very popular.

Perhaps Git could be the official and a community-run Mercurial mirror.
Git has pretty extensive web support and Mercurial can import Git projects so it works out.
sycoso
Posts:13
Joined:2012-09-18, 21:23

Re: Migrating Cafu to distributed version control – Part 1

Post by sycoso » 2012-10-15, 22:33

Yay :D

Where do you plan on hosting the git server? On cafu.de with full control over it or github with easy forking and easy pull requests but less control and a predefined (but IMO quite good) bugtracker?
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Migrating Cafu to distributed version control – Part 1

Post by Carsten » 2012-10-15, 22:58

HWGuy wrote:Perhaps Git could be the official and a community-run Mercurial mirror.
Git has pretty extensive web support and Mercurial can import Git projects so it works out.
Yes, good news is that with a Git repository at the center, we don't even need a Mercurial mirror: Thanks to
Mercurial can use a Git repository as if it was a native Mercurial repository. As described at the website, the process is lossless and thus apparently transparent to the user.

I have not yet tried this myself, but you can sure bet that I will. This Mercurial plugin seems to have to potential to unite both our Git and Mercurial users, and thus to get us the best from both worlds! :wohow:
Best regards,
Carsten
User avatar
Carsten
Site Admin
Posts:2170
Joined:2004-08-19, 13:46
Location:Germany
Contact:

Re: Migrating Cafu to distributed version control – Part 1

Post by Carsten » 2012-10-15, 23:38

sycoso wrote:Where do you plan on hosting the git server? On cafu.de with full control over it or github with easy forking and easy pull requests but less control and a predefined (but IMO quite good) bugtracker?
I'll say more about all this in Part II of my "Migrating Cafu" posts. But to anticipate things: The bug/issue tracker is an important factor, because it's closely related to and interwoven with the repository.

I've not yet decided about the issue tracker, but for hosting, I clearly tend towards Bitbucket. As it happens, they revamped their user interface just around the time I began evaluating it, see http://blog.bitbucket.org/2012/10/09/in ... bitbucket/ for details.
When I saw this for the first time, I immediately and intuitively got the catch of it, and was able to create my account and first test repo in minutes. I can actually find what I'm looking for. I love it.

Comparing this to Github, which I (once more) looked at as well: In Github, both in the past as well as in the present, the user interface never ceases to confuse me, and whenever I'm there, I have a hard time finding what I'm looking for. There are so many little buttons near the top that link to pages that don't have the expected information (where do I find a commit graph? does it even have one?), so that I usually leave frustrated. I seriously wonder how they could attract so many users over time.
What I only felt but couldn't properly express was very nicely put into words and illustrated by somebody else in this blog post: http://www.serpentine.com/blog/2012/04/ ... -stupider/

So right now, the tendency is towards Bitbucket, where I also run my continued tests right now. :up:

(Oh, and Bitbucket is made with Django, which I exhaustively use myself in a different project.)
Best regards,
Carsten
Locked

Who is online

Users browsing this forum: No registered users and 6 guests