Automation for the people: Parallel development for mere mortals

Branching, tagging, and merging in Subversion

 

It seems that most software development teams fall into two campswhen it comes to source-code branching: They don't branch at all, orthere's such a proliferation of branches (or worse, repositories) thatdevelopers are unsure where to check in changes — or find mergingchanges so painful that they riskily put it off until immediatelybefore a software release.

Terminology

The trunk (sometimes referred to as the head) is used for mainline development. A branch is a copy of a code line used for making changes that are different from the mainline development. A tag (sometimes referred to as a label) is a time-stamped copy of a codeline that identifies it so you can get back to it later in the development cycle.

In a perfect world, we would always work on the trunk. This wouldlimit the complexity of merging changes between two or more codelines.However, in the real world of software development, you might bedeveloping for a future release and at some point need to make anemergency patch to a version that's already in use. You need access toa copy of the source code for that released version — withoutdisrupting the new code that's under development.

But when teams try to use separate codelines, problems can occur.Some may choose not to create branches, which leads to delayed releasesand developer bottlenecks. Other times, developers merge lessfrequently, leading to merge conflicts, bottlenecks, and delayedreleases. A proliferation of branches can make it difficult to navigatethe project repository, causing developers to make inadvertent changesto the wrong code.

When a team develops in parallel, it's important to merge code backto the mainline (trunk) as often as feasible. If it's not possible tocommit the merged code back to the trunk frequently, test runs candetermine if any merge conflicts will occur so that merges are lesspainful when they are committed. In order to develop inparallel effectively, you can use tags and branches in Subversion(SVN), an open source and freely available source-code managementsystem. By tagging, your team can get back safely to a previous versionof the source code.

I'll demonstrate how to develop in parallel in SVN by covering:

• How to create an SVN release tag from the trunk
• Creating an SVN branch based on a release tag
• Techniques to merge changes back to the mainline (trunk)
• How to run Continuous Integration (CI) against a branch that is under development and regularly test the merge against the trunk
• Demonstrations of applying changes from branch back to the trunk
• Examples of tagging source code for a branch

About this series

As developers, we work to automate processes for users; yet, many ofus overlook opportunities to automate our own development processes.To that end, Automation for the people is a series of articles dedicated to exploringthe practical uses of automating software development processes and teaching you when and how to apply automation successfully.

Figure 1 shows a rudimentary workflow of several concurrent codelines:

Figure 1. Developing in parallel

In Figure 1, the active development occurs between versions 1.0.0and 1.1.0 off the SVN trunk. One set of developers can work on theversion 1.0.1 branch while others develop off the mainline.

Many strategies and techniques are available when multipledevelopers work on different codelines. In this article, I demonstratea common approach that I've used on projects using SVN.

Configuring Subversion for parallel development

Installing and configuring an SVN server is beyond this article'sscope. Assuming that you have access to a functioning SVN server,you're ready to run through these steps:

1. Download the SVN client software to your workstation.
2. Create standard local directories in private workspace.
3. Add directories to the SVN repository.
4. Commit directories to the SVN repository.

Download SVN client software for your operating system from the Tigris.org Web site (see Resources)and install it on your workstation. Be sure that the SVN executable isin your workstation's system path. Perform an SVN checkout of therepository using svn co URL.

Next, create three local directories:

• branches: Used to maintain software outside mainline development.
• tags: Used when you release software to identify a changeset for later use.
• trunk: Used for mainline development.

Listing 1 shows how to create these directories from the command line on Windows®, Macintosh, and *nix-based systems:

Listing 1. Creating local directories to prepare to add to Subversion

$ mkdir branches
$ mkdir tags
$ mkdir trunk

 

Now that the directories have been created in the operating system, you can add and commit them to SVN using the SVN add and commitcommands. From the directory where I created the directories in Listing1, I type the commands shown in Listing 2 (replacing the usercredentials, as appropriate):

Listing 2. Adding and committing local directories to remote SVN repository

$ svn add *.*
$ svn commit -m "Setting up standard SVN branches, tags and trunk directories" /
  --username tjefferson --password Mont!cello 

 

After I perform the actions in Listings 1 and 2, my SVN repository looks something like Figure 2:

Figure 2. Standard SVN directories created in repository

With my basic SVN repository setup in place, I'm ready to create a release tag.

---

Back to top

Creating a release tag based on trunk

The purpose of a tag is to uniquely identify a copy of a codeline ata particular point in time so that you can get back to this versionlater. Figure 3 shows a tag named brewery-1.0.0 that is created for the 1.0.0 release. (A tag can be created at any point in time, but tags are most often created when software is released.)

Figure 3. Create a unique tag for the SVN trunk

Assuming the trunk contains source code of released software, thefirst task is to create an SVN tag based on the trunk. Listing 3 is anexample of how to create this release tag:

Listing 3. Creating a release tag based on trunk

<path id="svn.classpath">
  <fileset dir="${lib.dir}">
    <include name="**/*.jar" />
  </fileset>
</path>
<taskdef name="svn" classpathref="svn.classpath" 
  classname="org.tigris.subversion.svnant.SvnTask"/>

<target name="create-tag-from-trunk">
  <svn username="jhancock" password="S!gnhere">
    <copy srcUrl="https://brewery-ci.googlecode.com/svn/trunk"
      destUrl="https://brewery-ci.googlecode.com/svn/tags/brewery-1.0.0"
  message="Tag created by jhancock on ${TODAY}" />
  </svn>
</target>

 

Securely failing

When first running an SVN server that uses Hypertext TransferProtocol over Secure Socket Layer (HTTPS), you must accept the securitycertificate. If you're connecting to a secure SVN server like this forthe first time from your Ant script, it will fail without muchdiagnostic information. So, you must run your first SVN command fromthe command line to connect to this server. You can subsequently runany SVN Ant script from your workstation to connect to this server.

Listing 3 uses the SVN Ant task provided by the Subclipse open source project (to download, see Resources). The JARs provided with the SVN Ant task — svnant.jar, svnClientAdapter.jar, and svnjavahl.jar — must be included in your classpath when you run the Ant script. The first part of Listing 3 defines the classpath. The second defines the SVN Ant task using taskdef. Finally, I perform an SVN copy from the trunk to the tags directory, providing a unique name for this release: brewery-1.0.0.

After you run the script in Listing 3 to create a new tag, your SVNrepository should look similar to Figure 4. Under the root level of therepository is the tags directory (created in Listing 2). Under it is the new tag (directory) created in Listing 3: brewery-1.0.0. It contains a copy of the trunk.

Figure 4. Create tag based on trunk

Although it is possible to modify tag contents in Subversion, you should never do so.

---

Back to top

Create a branch based on release tag

Creating a branch based on a release tag is similar, in technique,to creating a tag based on the trunk. Each involves the use of SVN's copy command. You always want to create a branch based on a tag because the tag is a copy of the code when it was released — rather than the code currently under development, which may have been modified. Figure 5 illustrates creating a 1.0.1 branch based on the 1.0.0 release tag:

Version naming

Coming up with a version-naming scheme is something people tend totrivialize, but it can be more than annoying when a slightly differentversion-naming pattern is used from one version to the next or betweendifferent projects. Many versioning patterns are possible. Choose onethat is simple but will be flexible in dealing with future releases. Asimple pattern I use is major-version.minor-version.patch. Version1.1.2 is an example version number based on this naming pattern. Someteams choose to append a build number to the version as well.

Figure 5. Creating branch 1.0.1 based on 1.0.0 release tag

Listing 4 calls the SVN copy command via the SVN Ant task to copy all files from the brewery-1.0.0 tag to the branches location:

Listing 4. Ant script to create branch from release tag

<target name="create-branch-from-tag">
  <svn username="sadams" password="b0stonM@ss">
    <copy srcUrl="https://brewery-ci.googlecode.com/svn/tags/brewery-1.0.0"
      destUrl="https://brewery-ci.googlecode.com/svn/branches/brewery-1.0.1"
  message="Branch created by sadams on ${TODAY}" />
  </svn>
</target>

 

After you run the script in Listing 4, the SVN repository should look something like Figure 6:

Figure 6. Creating branch from release tag

Remembering the trunk

Some teams take branch development to the extreme by beginning alldevelopment on a branch. Remember that it is less complex and moremanageable to develop exclusively off the trunk. The reason forbranching is to be able to support separate development efforts inparallel. Try not to misuse it by creating a branch for everything youdo.

By remembering always to use tags when creating branches and usingthe SVN Ant task, you can provide a repeatable process that gives youan easy way to maintain, and get back to previous versions of, yoursource code.

Run CI against branch

The process of CI is typically run against the repository'smainline: the trunk. This can be extended to branches as well, with theintent of integrating changes among developers working on the branchand checking the merge with the mainline.

Figure 7 shows the SVN location. From this Hudson configuration page, you can also define the Ant target to call.

Figure 7. Hudson CI server building branches and testing merges against the trunk

Running a CI server such as Hudson to test merges can give you anearly warning system that alerts you to potential merge conflicts thatcould occur later on in the development cycle.

---

Back to top

Merge changes from branch back to the trunk

One of the primary reasons to create a branch is to preventdisruption to mainline development. However, once you've completed workon the branch, the changes should be merged back to the trunk. Figure 8illustrates a merge from version 1.0.1 back to mainline, which isdeveloping version 1.1.0 of the software:

Figure 8. SVN timeline

In Listing 5, I use the merge command from Subversion. I type svn merge followed by the URL to merge to, then the URL to merge from, followed by the local directory location:

Listing 5. Using SVN's merge command to merge branch development back to the trunk

$ svn merge https://brewery-ci.googlecode.com/svn/trunk /
 https://brewery-ci.googlecode.com/svn/branches/brewery-1.0.1 /
 /dev/brewery --username pduvall --password password!

 

The SVN Ant task does not provide a merge command, so the merge command needs to be run from the command line. Or you can run it using Ant's exec task.

The results of running the command in Listing 5 are similar to those shown in Figure 9:

Figure 9. Results of merging branch back to the trunk

If the merge is successful, you need to commit the change in Subversion, as shown in Listing 6. From the command line, type svn commit along with the message description and the SVN URL for the trunk:

Listing 6. Committing merged changes to the trunk

<target name="commit-branch-to-trunk">
  <svn username="gwbush" password="IL0veTHEG00g!e">
    <commit dir="${basedir}"
      message="Committing changes from brewery-1.0.1">
    </commit>
  </svn>
</target>

 

Merge changes from the branch back to the trunk as often as possibleto prevent difficult merges and so that the different codelines don'tgrow apart over time.

---

Back to top

Create a tag based on branch

To prepare a release based on a particular branch, I create an SVNtag. This follows a similar approach to some of the previous listings.Figure 10 shows the creation of a tag called brewery-1.0.1 based on the brewery-1.0.1 branch:

Figure 10. Creating a tag based on a branch

When development is finished on a particular branch, it needs to betagged in Subversion. Listing 7 shows an example of creating this tagbased on the branch:

Listing 7. Creating an SVN tag based on a branch

<svn username="jbartlett" password="newHampsh!re">
  <copy srcUrl="https://brewery-ci.googlecode.com/svn/branches/brewery-1.0.1"
    destUrl="https://brewery-ci.googlecode.com/svn/tags/brewery-1.0.1"
    message="Branch created by jbartlett on ${TODAY}" />
</svn>

 

By creating a tag based on a particular branch, you can get back to this version later in the development cycle.

---

Back to top

Right on track with parallel development

Developing in parallel isn't brain surgery, but it can bemonumentally difficult to manage without planning and continuallyimproving based on project needs. If you remember one thing, rememberthat all roads should lead back to the mainline — eventually. Look atbranches as a temporary home for source code that could interruptmainline development. The last point to consider is to test the mergeearly and often. There are probably version-control systems thatsupport parallel development better than Subversion, but in myexperience the policies that teams adhere to when developing are muchmore important than how a tool technically solves the problem.

 

Resources

Learn

• Software Configuration Management Patterns: Effective Teamwork, Practical Integration:(Stephen Berczuk and Brad Appleton, Addison-Wesley, 2002): Learnpatterns of software configuration management from real-worlddevelopers.
  
• SCM Patterns: Access software configuration management resources at the SCM Patterns Web site, maintained by the authors of Software Configuration Management Patterns.
  
• "Branching: do it like this and nobody gets hurt"(Julian Simpson, Build Doctor, September 2008): Build and deploy expertSimpson expresses how following simple branching rules can be the mosteffective approach.
  
• Version Control with Subversion - Second Edition(Ben Collins-Sussman, Brian W. Fitzpatrick, and C. Michael Pilato,O'Reilly, 2008): Official guide and reference manual for Subversion.
  
• Continuous Integration: Improving Software Quality and Reducing Risk(Paul Duvall, Steve Matyas, and Andrew Glover, Addison-Wesley SignatureSeries, 2007): Numerous examples in the book cover how version controlworks in the context of Continuous Integration.
  
• Browse the technology bookstore for books on these and other technical topics.
  
• developerWorks Java™ technology zone: Hundreds of articles about every aspect of Java programming.
  

Get products and technologies

• Subversion: Download Subversion to manage source code versions.
  
• Ant: Download Ant and start building software in a predictable and repeatable manner.
  
• SVN Ant task: Download an Ant task for operating Subversion to provide a repeatable process for managing source changes.
  
• Hudson:Download the Hudson Continuous Integration Server to begin runningbuilds with every change to Subversion (or other SCM server).