Instructor: Introduction to SVN
What is Subversion (SVN)
SVN, or "Subversion", is a source code management system that allows a group of programmers to work together on the same collection of source files without getting in each others' way any more than necessary. It also keeps track of the change history of the collection and makes it possible to examine the way things looked at arbitrary times in the past. And finally, by keeping track of what is and is not part of a project, it helps programmers maintain the organization of large projects. If you have used CVS in the past, you will find that SVN is quite similar, although somewhat more sane. If you are familiar with CVS and concerned about the differences in migrating to SVN, there is a special appendix of the SVN Book specifically for you!
Why source management?
In most of your course work, each assignment is a distinct unit: you sit down and code something up, you hand it in, it gets graded, and you immediately forget about it or even throw it away. In this environment, a source management system is not really necessary and may, if imposed by course staff, seem like a waste of time.
In the real world, however, programs are large and expensive to develop, and so their life-cycles are measured in years or sometimes decades. Over these time scales, and with such large amounts of code, just keeping track of everything becomes a major problem.
Worse, in the real world, programs have users, who are not part of the development team and not (generally) interested in internal details of the program. Usually, someone insists that now and then a new version be made available to the users. The development team has to be able to issue these releases, and then also has to be able, for years afterwards, to handle bug reports and sometimes issue fixes. In this environment it is imperative to be able to go to some central place and get a copy of the precise release you need.
And finally, when you have a number of programmers working on the same program at once, it's essential that some mechanism be put in place to allow them to coordinate their work. Otherwise, each programmer's version slowly diverges from the others, and eventually everyone has a private version different from everyone else's... and none of them work. Once this happens, it takes an immense amount of effort to straighten out the mess.
Source management (or version control) systems are designed to help programming teams handle these issues.
In this course, the probable lifetime of your project is a few months, not a few years, and most likely no more than two people will be working on it at once. Furthermore, the OS161 source you will be working with is several orders of magnitude smaller than a large real-world project. Nonetheless, it is large enough, the time is long enough, and there are enough people involved that failure to use some kind of source management system would be an act of reckless folly.
Why SVN?
We require the use of SVN because it is reasonably powerful, freely (and widely) available, and commonly used. Many large open-source and proprietary projects are managed using SVN.
If you are familiar with another source management system (such as CVS), SVN should be easy to understand and use.
Remainder of This Document
The rest of this handout is divided into two main sections. The first explains the philosophy of SVN, its operating model, and the assumptions behind the way it works. The rest explains, in terms of how one actually uses SVN rather than its various commands, a number of basic and not-so-basic SVN operations. A small additional section lists the main SVN commands.
You do not need to remember everything in this handout. In fact, this handout was written mostly so you do not need to remember all this.
The World According to SVN
The SVN model assumes that there is one central official master copy of everything. It is called "the SVN repository." This copy is managed by SVN and is not meant to be touched directly.
Instead, when you wish to work on a SVN-managed project, you "check out" your own private copy or working tree. Since work on a program is an ongoing process, and other people may be working on the same program at the same time, you generally want changes in the official master copy to appear in your private copy. To facilitate this, the private copy is connected back to the master copy. Some people think of this as a "subscription" - the private copy is signed up to receive copies of updates that are made to the master copy.
Note, however, that these updates do not happen automatically. You must explicitly update your private copy when the master copy changes.
When you transfer ("commit") changes in your private copy to the master copy, the master copy is updated and other developers will then see those changes when they perform an update. Changes you make in your private copy that you do not commit do not officially exist and will not be seen by anyone else until you commit them.
You can have as many private copies as you want. Often you will have only one, but circumstances may arise in which it is more convenient to have two, or to make temporary ones, or whatever. Just remember that they are all independent: while they're all connected back to the master copy, they are not connected to each other except through the master copy.
Repository
The official master copy is known as the "SVN repository". SVN provides a variety of ways of accessing the repository. In this course, we are providing repository access using ssh, meaning that your SVN development repository path will look something like svn+ssh://ug250.eecg.utoronto.ca/srv/ece344s/GRP_NR/svn, where GRP_NR is your group number.
Accessing svn using ssh access generally requires using your ssh password for each svn command. However, for your convenience, you can setup ssh keys, after which you will need to use your ssh password only once per ssh session.
You should not have to change the repository by hand. If you really mess something, please let us know and we can try to help you.
Merging
Unlike older source control systems, SVN does not lock files for modification. Instead, SVN works from a model where everybody edits freely and changes to the same file are merged. SVN does not support locks at all.
In the merge-based model, anyone can edit any file at any time. This is both an advantage and a disadvantage: if two people have small unrelated changes to make in the same file, they can do so without any difficulty. Another major advantage is that when there are no locks, nobody can hold up development by leaving on vacation while holding a lock on a critical file. On the other hand, if two people make sweeping changes to the same file at once, the resulting merge becomes a nightmare.
The way SVN merging works is as follows: when you check out a working tree, SVN remembers a global version number describing the tree you got. When you update your working tree, it updates the global version number. When you go to commit, if the version your changes are based on is not the latest one, SVN aborts and tells you to update first.
Then, when you update, SVN notices that you have changed your copy and the master copy has changed as well. It then tries to merge the two sets of changes. If the changes are to unrelated areas of the file, this usually succeeds. If the changes overlap, or the merge program gets confused, SVN will ask you what to do. If you do not choose to accept the repositories changes ('tf', or "theirs-full"), push your changes entirely ('mf', "mine-full"), the safest thing to do is to simply 'p' or "postpone". This will mark affected files in a "conflicted" state and insert blocks that look like this:
@@ -1 +1,5 @@
int foo(void) {
+<<<<<<< .mine
+ bar();
+=======
+ baz();
+>>>>>>> .r32
This means that your copy of foo.c changed function foo to call bar, but that the official master copy, in revision 32, changed foo in the same place so it calls baz.
When you get merge conflicts like this, you need to resolve them before committing your new versions. You might pick your version, or the latest version from the repository, or some combination of the two, or whatever. When doing this, it's up to you to make sure you do the right thing. Once you are finished you need to run the svn resolved command to indicate to SVN that you have successfully dealt with the conflicts and whatever is left is ready to be checked in to the repository.
Some notes:
- Even when there are conflicts, the conflict blocks do not necessarily reflect all the changes associated with the merge. Some may have merged successfully. If in doubt, look at diffs.
- It is also not always the case that everything that may be involved in resolving a conflict correctly is contained within the conflict block delimiters. The merge program is only a program, not an omnipotent human being.
- While the merge system is reasonably robust, once in a while it makes a mistake, particularly if some but not all of the changes merged. It's prudent to look at diffs after an automatic merge, just in case.
- Merging is painful. Merging a big change is a lot more painful than merging the same amount of change a bit at a time. Update early and often. Commit early and often.
If you are planning to make huge changes to a file, like reordering all the functions or moving large blocks of code into if clauses (which changes the indent, making SVN think everything changed), it's a good idea to coordinate manually with anyone else who might have pending changes to the file.
Log Messages
When you commit changes to the SVN repository, SVN gives you the opportunity to provide a message explaining the change. These messages get saved in the SVN file and can be reviewed later using svn log. This can be quite useful when trying to reconstruct the thinking that led to some piece of code you wrote months previously.
The commit message should thus describe (briefly) what you did and why. There's no need to report the exact changes, as they can be retrieved using svn diff.
When to commit
The general rule for commits is that any change should be committed as soon as you're reasonably certain that it's correct and appropriate in the long term, subject to the proviso that committing many small changes in quick succession will probably annoy everyone working with you.
Remember that your partner won't see anything you don't commit, so get bug fixes in quickly and hold back a little on new features that might still have problems.
Ideally you and your partner should keep track of which tests you expect to work at any particular time, and before committing check to make sure that they all still do work.
In most cases, one should try to avoid committing changes that cause the program to stop working properly (or, even, stop compiling at all.) This rule can sometimes be profitably bent when you know your partner will not be affected by the errors introduced.
Tags
While SVN uses global version numbers, retrieving your codebase via a global version number is rarely useful. Instead, SVN supports a concept known as a "tag". A tag is a symbolic name (like asst4-debugged) that you attach to a particular version of some set of files. You can then refer to that version of those files with the name.
Indeed, tags fit seemlessly into SVN's view of the world. In SVN, tagging is accomplished simply by creating a copy of your current development directory in another part of your versioned tree.
To introduce some degree of consistency most SVN users structure their repositories into at least two areas, divided at the top level into: trunk, which contains the canonical working copy and tags which contains copies of trees saved for the purposes of tagging. Given the canonical SVN repository structure, creating a tag means simply copying trunk to another part of the repository.
A tag is normally used to identify a single consistent version of an entire project. For instance, the directions for each assignment tell you to create one tag before starting and another tag after you're done. These tags then identify the versions of all your files that were current before and after you did the assignment. This lets you, for example, ask SVN to show you all the changes in the entire system between those two points.
See below for specific directions for manipulating tags with SVN.
Use SVN Effectively
SVN is a tool, not a panacea. It helps you organize and maintain a project, but it doesn't do it by itself. It requires that you use it in a manner that makes it useful.
In order for the repository to be a useful tool for keeping track of what is really part of the project and what is not, you have to actively maintain the set of files SVN knows about. Don't add or commit temporary files, editor backups, object files, and the like to the SVN repository. In general, you should not add files that can be generated in any way. For example, executable files should not be added because they can be generated from sources.
You can also remove files from SVN that you're not using any more. Even after telling SVN to remove them, you can still get them back later, because removing them is just a change that SVN tracks.
In order for the version history to be useful, you have to add tags at important points in development, like releases. You also have to write at least minimally useful commit messages so you can look at them later and be reminded of the circumstances.
In order for the merging features to be useful, you have to avoid making sweeping changes without warning your partner, you have to update and commit regularly but not insanely often, and you have to take the trouble to merge correctly by hand when conflicts occur.
If you don't do these things, you will eventually end up in a hole, and SVN will not save you from yourself.
How do I...
The previous section explained SVN concepts in general terms. In this section we explain how to do various useful things.
How do I make a new repository?
Don't bother with this. We have created repositories for you.
How do I add files and directories to a repository?
There are two ways to add code to a SVN repository. One is to use svn add in a working directory to add files and directories one at a time. The other is to use svn import to do a bulk import of a whole unversioned existing source tree into the repository. Using svn import will be described in Assignment 0. We describe svn add here.
You can add files or directories to a repository by using svn add. To add a directory into an existing repository, create the directory in the appropriate place in a checked out tree and then ask SVN to add the directory. Adding the directory will recursively add all files and sub-directories within the directory.
% svn add dir
To add files to an existing directory, create the new files in the appropriate directory in a checked out SVN tree, and then ask SVN to add them to the repository:
% svn add newfile
Files and directories are added to your working tree. They will then show up with the "A" code on subsequent calls to svn status. They will not be visible to other developers until you update the repository by using the svn commit command described below.
How do I check the status of a file?
You can examine the status of your complete repository or a portion of it using the svn status command. This is useful to do before committing, and at any point during the development cycle. It can also help you spot files that are missing from the repository, either because they need to be added or you need to tell SVN to ignore them using the svn:ignore property discussed below.
How do I check out a working tree?
Use the svn checkout command with the name of the project (the top-level directory in the SVN repository):
% svn checkout <repository path> os161
This will create a directory called os161 that holds a working tree.
To check out a particular tag, you need to change the URL that you check out from. It's that simple. For example, if for some reason you wanted to check out the tag that you checked for the beginning of ASST1, you would simply run:
% cd some directory % svn co <repository path>/tags/asst1-begin
If you want to check out the trunk or even a tag while specifying a global revision number, explore the -r option to svn co. This can be useful if you are trying to roll back to a known good version. However, if you roll back your working copy to an earlier version, SVN will remember that you did this, and you will need to update again to the head version using a different argument to -r.
How do I update my working tree?
Use svn update. You can update whole directory trees or individual files. It's your responsibility, if you don't update everything at once, to make sure the resulting working tree you have is self-consistent.
If you don't specify what to update, SVN updates the current directory recursively. Like with svn checkout you can specify particular versions or dates with the -r option.
When you update, SVN prints one line for each file it processes, with a letter in front of it reflecting the file's status. These letters are:
- A Added
- B Broken lock (third column only)
- D Deleted
- U Updated
- C Conflicted
- G Merged
- E Existed
If you want to retrieve status information, use svn status, not svn update, since the latter will update things below you that you might not want to update yet.
How do I commit my changes?
Use svn commit. You can commit directories or individual files. You can use the -m option to supply a commit message on the command line. If you don't, SVN will invoke an editor and ask you to add a message. Like with most commands, if you do not specify anything to commit explicitly, SVN commits all changes in the current directory and all subdirectories.
% svn commit foo.c
or
% svn commit src/kern
Remember that changes that have not been committed, including added and removed files, will not be seen by other developers.
The EDITOR environment variable must be setup correctly or else you may see errors such as the follows:
svn: Commit failed (details follow):
svn: system('/usr/bin/editor svn-commit.tmp') returned 256
To setup the EDITOR environment variable (assuming emacs is your editor):
- For csh, add the following to the end of your ~/.cshrc:
setenv EDITOR emacs - For bash, add the following to the end of your ~/.bashrc:
export EDITOR=emacs
How do I ignore a file or directory shown by svn status?
When you run svn status, its output will show a question mark ? in front of a file (or directory) that is not known to SVN. You can either add this file as described above, or you can ask SVN to ignore it. Typically, generated files, such as object files should be ignored. One way to find out if a file is generated is to run make clean after you have run make, which will remove generated files. Running svn status after running make clean should only show non-generated files.
You can ask SVN to ignore a file (or directory) by adding the svn:ignore property to it. When this property is added to the file, the file will not show up when using svn status. The following example shows how you can ask SVN to ignore a set of files.
% cd lib/hostcompat % svn status ? libhostcompat.ha ? time.ho ? hostcompat.ho ? err.ho
The command above shows the ? symbol for various unversioned files. Check that they are generated files. For example, they may be object files. If we wish to ignore them, run the following:
% svn propedit svn:ignore .
A temporary file will show up in an editor. Add each of the unversioned file names to the temporary file, save the temporary file, and exit the editor.
libhostcompat.ha time.ho hostcompat.ho err.ho
Run the svn status command and these files will not show up any longer. This command will show that the current directory is modified. It is modified because the svn:ignore property has been added to the directory. When you commit the directory (you should do so), this property will be added to the directory in the repository. See this page for more details.
Is there an easier way to ignore files and directories?
Yes, we have provided an easier way to ignore files. Run the svn-ignore.sh script in ~/ece344/os161.
% cd ~/ece344/os161 % ./svn-ignore.sh
This script iterates recursively over each .svningore file found in any subdirectory, and ignores the files listed the .svningore file in that subdirectory. If you want to ignore a file F in a directory D, the easiest way to do so is to add the name of the file F (or a regular expression such as F.* or *.F) in the .svnignore file in the directory D. If the .svnignore file does not exist in this directory, then you will have to create it. Then run the svn-ignore.sh command from the os161 directory again.
How do I remove files that are versioned under SVN?
When you wish to remove a file or directory from the tree, run svn rm. Do not delete it first. SVN will do this for you. Removed files show up with a "D" code with svn status. Commit the file (or its directory) after removing it.
It's a good idea to compile the project after removing but before committing, just to make sure you aren't breaking things.
Files that have been deleted are still kept around by SVN. While they'll be removed from people's working trees by default, you can still look at them, and you can bring them back again later if needed.
How do I rename files?
Run svn mv. However, you may lose revision history upon moving the file, or get it tangled up with another file that may have had the same name previously.
How do I create a tag?
Quite simple, using svn copy. For example, assuming you have the trunk,tags structure described above and you want to create the tag FOO:
% svn copy <repository path>/trunk <repository path>/tags/FOO
Easy. Right? The versions tagged will be the version in the repository that your working tree is based on. (As always, uncommitted changes will not be processed.)
How do I make diffs?
Use svn diff. Specify the files or directory trees you wish to compare. If you do not specify anything, by default the current directory and all subdirectories are diffed with the repository version. You can diff against a specific repository version by providing an -r option (as described above under checkout), or against a tag by providing the repository URL. You can diff two specific versions by providing two such options.
If you want to see the latest commits that you haven't updated yet in your working directory, use -rHEAD as one of the arguments.
How do I find out where/when a particular line of code appeared?
The svn blame command prints each line of the file with a prefix containing the version number in which the line appeared, the date of that version, and the username of the person who committed it. You can use the -r options, or a tag URL to retrieve the file as it existed in any previous version.
This can be used in conjunction with svn diff to track down the history of individual lines of code, as long as they haven't moved around very much.
How do I back out a bad commit?
It's late at night and you foolishly/accidentally commit some immensely stupid change that breaks everything. (We've all been there; if you haven't yet, you will eventually.)
All is not lost. Part of the role of SVN is to keep track of old versions; you can extract the old version and re-commit it. Suppose that you can determine than version 342 was the last "good" version of the code. You can return your working directory to that version using the following command:
% svn update -r342
This tells SVN to update your working directory with revision 342. Next, you can commit this version to the repository.
How do I move my working directory?
In general, you can just rename the entire tree with mv. The SVN control files in the working directory are position-independent.
For more information
To get a list of SVN commands and options, use the help page for SVN (type svn help), for a particular command (type svn help <command>) or look at the online SVN documentation.