Configuring SVN

When I started using SVN 1.4.3 (r23084) with the help of TortoiseSVN, I have struck a number of quirky things and a number of must do things:

Ignoring bin, debug, etc. files in .Net development

You can use TSVN to edit the config, which is located in
"\Documents and Settings\<user>\Application Data\Subversion" and the file name is Config with no extension.

To get it to ignore these directories or files, you need to uncomment or edit the global-ignores line. In mine, I have it set to:

   global-ignores = bin obj *.suo *.obj *.pdb *.exe *.dll *.csproj.user

This will causes it to ignore the bin and obj sub-directories of a typical Visual Studio projects.

Supporting keyword substitution

Subversion supports keyword substitution even though it treats each check in as a file set. It is not as complete as that in VSS and these are the followings:
$Revision$, $Id$, $Date$, $HeadURL$ and $Author$

$HeadURL$ is particularly useful as it is expanded into the path in the repository and hence allowing one to quickly work out which branch does this file belong to. $Id$ is not as useful if you have already included $Revision$ + $Date$ + $Author$. BTW, it also supports the $Revision::$ convention to allow one to line up all the values, much like VSS.

To enable keyword substitution, you need to do:

1. Add the above keywords into the source file.
2. Edit the config file to include "enable-auto-props = yes"
3. Then in the "[auto-props]" section define the file types that support the keywords.

There is a quirky thing with SVN and when you define the supported keywords for a file type, say C#, you need to include a leading dummy keyword and a trailing dummy keywords. They are kind of sacrificial keywords to be thrown away keeping the wanted ones. Otherwise the substitution logic will ignore the leading and trailing keywords. You can specify which keywords are supported for which file type. For example to specify expansion for C# file, this is what you have to do:

[auto-props]
*.cs = svn:eol-style=native;svn:keywords="xx Revision Id Date HeadURL Author yy"

The ones in italics are the dummy ones. You can use any names.

Don't put all projects into one repository

For information, see the Pragmatic Version Control for advice.

Use symbolic link to set up repository

If you are using RANU (Run as a normal users>, you should consider putting all the repositories in "\Documents and Settings\All Users\Documents\Svn-repos".

But doing so can make it hard to use command line program like svn, which uses a modified file URL convention requiring spaces to be replaced with %20. Rather than putting it on the root, you can set up a link to this directory in the root using the LinkD.exe, a tool which is part of the Windows Resource Kit. This does not damage the security settings of target directory or the link. You can use this command to do it:

Linkd c:\svn-repos "C:\Documents and Settings\All Users\Documents\Svn-Repos

"Pragmatic Version Control - Using Subversion" by Mike Mason

Review

This is a must read book for anyone starting out to use Subversion the first time. It is not a book for a novice but for someone who has uses other VCS. SVN is like any other software tool that it has its own view of using it effectively. Failure to understand this can lead to frustration and costly time loss.

This should be treated as a supplement to the Subversion book as this book is a how-to book in using SVN properly.

SVN has its own kind of idiom because "SVN organizes everything by directory" in the repository. These directories does not necessary have corresponding physical directories in the file system. However, to help one to keep track of branches and tags, it is recommended that one tries to map these to physical directories.

These directories in the repository are virtual and does not incur real storage cost. When you issue command to a branch, nothing really is copy until files are changed. There are there to help one to organize the projects and to manage their versions.

The other issues is that the revision number is like a transaction number. Every transaction, like creating a directory in the repository, incurs a new transaction id. In other VCS, this is not so.

For those that have already had exposure to other VCS can jump pretty much into Chapter 7 "Organizing Your Repository". It is always a good idea to start off in the right foot even though SVN has commands to rescue you, see Chapter 6 for some of the operations.

In SVN, Branch and Tag, while similar, have different usages. Tag is a symbolic name for a set of files and "Tags are really useful for keeping track of important events in the life cycle of your project". A branch represents a fork in the history of the repository. Branching uses "cheap copy" of the files as they were when the branch was made. When you check in a change to the files on the branch, Subversion remembers the changes.

The book provides a very thorough coverage of using Branch and Tag properly in Chapter 8 "Using Tags and Branches". It even takes the reader through the journey through a short software life cycle showing how to use SVN properly in each stage. You are encouraged to read that.

It has provided a naming convention for using tags and branch and the author recommends one should apply this consistently across all projects.

After reading Chapter 8, don't just dive into using SVN like you used Visual SourceSafe or other VCS. You are recommended to read Chapter 9 "Creating a Project". This chapter discusses the pros and cons various layout schemes of projects, branches and tags in a non-trivial project consisting of multiple sub-projects. It also discusses a very powerful idiom in SVN to allow one project to draw in another project from a different repository.

SVN is very different than other VCS in its use of revision number. It maintains only one number beginning at 1 for the entire repository. Hence it is better to keep unrelated projects in separate repository rather than in one, which would cause the revision number to be seemingly assigned meaninglessly across different projects. In VSS, the revision number is restarted for each project within a VSS database and hence it is customary to have many projects in one database, which is kind of equivalent to a repository in SVN.

Keeping them in separate repositories may give someone an idea of difficulties in using these projects in separate repositories as dependent projects. Contrary to this belief Chapter 9 shows an example of how to use svn:externals properties to link in projects in other repositories at the particular branch or tag. Chapter 9 is definitely a must read book before one starts.

This book is published by "The Pragmatic Bookshelf"

The world without Visual SourceSafe

I am not here to bash Visual SourceSafe as I still believe that it is a great tool for simple development shop but to explore the world after leaving VSS. What are the possibilities on offer?

The primary aims are to keep as much cash in my pocket rather than putting them in some vendor's pocket and that it has to scale up.

What can scale up better than source control system that are routinely used by Internet Open-Source communities? Hence there are CVS and Subversion. I have used CVS and not at all impressed with it and since Subversion (SVN) is a replacement of CVS, I will concentrate on it instead.

The other on offer is VSTS with TFS from Microsoft. Since my MSDN subscription can let me use this and that it is a fairly common toolset, I will also explore this one too. The fact that it is becoming like a giant octopus reaching out and trying to be a tool for everything worries me. Others such as ClearCase, I can't afford and my dislike of it has subsided considerably after my discovery of TFS as revealed below.

There are enough centralised materials on TFS that need not be repeated but Subversion route is more interesting and is free!

One of the things that stunned me when looking at VSTS/TFS as a VCS is that it has taken away the facility of keyword substitution of things like $Revision:$, $Date:$ or $Log:$ (keywords available in VSS) in the source file to brand it. They are god-send when you have a few copies of them lying around.

In these days of ubiquitous USB memory drives, the chance of having a few copies of the same file is extremely high. Without any form of identification, you will be spending hours resolving the differences.

To me, this is a philosophy that someone is trying to ram that into the customer's throat and is a classic example of "Why Software Sucks...". The fact is that the repository has all these pieces of information and the software refuses to allow the users to use them as they see fit. Their developers should be reminded that "You are not the user" principle.

This is not just VSTS/TFS dogmatic approach in pursue of their philosophy but out of the box, ClearCase does not support this too. But at least in ClearCase, one can add a script that is called when one checks in a file to extract those information from the repository and injecting them into the source file. Microsoft, I hope you are listening.

It is also interesting to compare the philosophy used in Subversion in managing the versions.

• It is very similar to that used in TFS.
• It treats a set of files/folders as a tree rather than file by file as in CVS.
  
• It is also atomic - meaning that if one file in a set fails to check in, the whole set will not get in.
• The revision number applies to the entire tree not really to a file.

OK. I have left the best part last. Even though their versioning models are almost identical, SVN allows users to perform keyword substitution to inject revision number using $Revision: $, check in date using $Data: $, and the customary $Id: $ is also there too. For more information please see "Keyword Substitution" in Chapter 3 of the book, which is free. The only thing missing is to inject the change history into the file. For that I can live with it.

Good to see SVN developers that are considerate and not pushing one's philosophy down one's throat.

This alone wins me over immediately.

For Windows users, one of the disadvantage with early CVS was that there was no GUI client. SVN has fixed all that. The best is the Windows Explorer plug-in called TortoiseSVN.

For those that live day in day out inside Visual Studio, help is also available in the form of an add-in.

Finally, the Windows version of SVN with an installer can be downloaded from here free.

Armed with all the materials, I am off to explore the world without VSS. Stay tune.

RANU - Microsoft's advice

RANU is a new acronym that I have just discovered when installing SQL Server 2005 Express books online.

It stands for Run As Normal User. In other words, another acronym for LUA.

I was digging around looking for advice on this new concept called "User Instances" and this is where I discovered this acronym. Microsoft actually wants developers to RANU rather than as Administrator and has gone to great length to help by introducing this new concept called "User Instances".

On this topic, this is what Microsoft's advice to the development community:

Microsoft strongly recommends NOT running as an administrator because this reduces the amount of damage that can be inflicted by a virus.

This piece of advice can be found on the books online on "Users Instances":

Many Windows users have accounts with administrative privileges. Unfortunately, running as an Administrator makes it easy for malicious software to take over a user's computer. However, malicious software run by non-administrator users cannot make system-wide changes and therefore can do only limited damage.

It seems developers have a very poor idea of how to develop in non-admin account and they can found out the recipe from here.

Many developers thought that they are shackled down as a poor users and they all will tell you that developers need administrative rights. I was in that camp once.

I think they are very confused. No one denies that a developer need administrative rights but that rights are not required for the majority of time. There is a sharp distinction between having administrative access/privilege and running as an administrator. The latter means you have turned off the security infrastructure of Windows. The former means that you operate in LUA or RANU and gain the elevated privilege when you need it.

For example, as Keith said you do not need administrative rights to compile or debug a program. Visual Studio is quite happy RANU. The only time you need Admin rights is if you need to register COM server, restart IIS, etc.

In COM development, you do not need to register a COM component in order to build the client. All you need is their TLB. Similarly in .Net, you can generate the interop assembly manually in the build script for the server, thus avoiding the need to add a reference via the COM tab.

So if I were a developer who still insist on running everything as admin, I respectfully suggest he/she to visit Keith's instruction, give it a go and you will be amazed how little adjustment you have to make. It also sharpens your awareness of security issues when programming.

The benefit is that you can feel the security envelop within your debugger. So if there is any access violation, you will be told immediately.

On the topic of Users Instances, if you follows the example in that article to have a taste of it, rather invisible actually. You may not have user instances facility turned on by default and when you open a connection to the user instance, you will get a System.Data.SqlClient.SqlException saying that

Generating user instances in SQL Server is disabled. Use sp_configure 'user instances enabled' to generate user instances.

This is what you have to do to fix it:

1. Fire up the Sql Server Management Studio Express
2. Press New Query to compose a query
3. Follow the advice in the section for "sp_configure 'user instances enabled'" here.
   

Grep the tab - driving me mad

Most of the people would have heard of this great Unix tool called Grep. This is the authoritative site on the Grep and can be downloaded from here for Windows. This downloads the whole set of GNU tools, which are very good.

There appears to have a number of slight variants of Grep around, kind of like Linux/Unix and that the GNU Grep does not support the ability to find the TAB (0x9) character. For example, if you have a line like this:

<tab>Hello World

in a file file.txt and if you use this syntax

grep -E "\t+Hello" file.txt

You will get nothing. The reason is that according to GNU Grep, \t is not an acceptable special character. It simply looks for \ follows by t repeating one or more time followed by Hello, contrary to standard Regular Expression syntax. So grep does not use regular expression at all.

To search for white spaces, GNU Grep has this syntax: [:blank:] which indicates a space or tab. But it will not be just looking for TAB.

All is not lost! There is a variant of grep that supports the Perl Regular Expression mode. It is selected by -P or --Perl switch. You can download it from here for Windows version.

With this you can use this syntax to echo the above line:
grep -P "\t+Hello" file.txt

The only disadvantage of this version of grep is that it is not a single file program. It now requires the following DLL: libiconv2.dll, libintl3.dll, pcre3.dll and in addition, it requires MSVCP60.DLL.

It is nice to see this version of grep.exe also has the PE version information so that you can tell what version of software you have. It never amazes me why Unix/Linux never has anything like this. It is almost impossible to tell the vintage of the number of grep.exe in my machine and all of them have different MD5.

If you need to conceal the e-mail address from spammer

I have been updating my blog to improve communication but I do not want to plaster an e-mail address all over it for spammer to harvest it.

So I searched the BlogSpot help forum and found many valuable suggestions. Among them I have found using an e-mail encoder an easy way to accomplish this.

I followed one suggestion to edit this blog's layout to add a JavaScript/HTML page element to inject my contact e-mail address. According to the community is can prevent harvester. Time will tell I guess.

Agile Software Development

Many people thought agile software development is all about doing things fast and no need to do planning and design but let's not be too fast.

A recent report on Agile movement emphases that many have

mistaken belief by some adherents that no requirements planning or architecture should be devised before embarking on a project.

In Jim Highsmith's book "Agile Software Development Ecosystems", he said:

Agility means quickness, lightness, and nimbleness - the ability to act rapidly, the ability to do the minimum necessary to get a job done, and the ability to adapt to changing conditions.
[...]
They become skilled, and Agile, from long hours of training and effective mentoring. Sometimes their drills are repetitive and prescriptive, but only as part of learning.

Agility also requires discipline and skill. A skilled software designer can be more agile than a beginner because he or she has a better sense of quality.
[...]
"You need to have a very good discipline in place to be able to respond in turbulent times, yet simultaneously know when to be 'undisciplined.' I view anticipation to be actively seeking situations where the generally accepted guiding rules or principles no longer apply, or where shortcuts are the least risky approach to take to gaining some objective. To be able to understand when the rules don't apply, you need to completely understand when they do"..... He has to be skilled before he could be agile.

Agile individuals can improvise, they know the rules and boundaries, but they also know when the problem at hand has moved into uncharted areas. They know how to extend their knowledge into unforeseen realms, to experiment, and to learn.

These paragraphs in his book are so accurate in describing the agile software development that everyone considering adopting any agile approach should recite. I recommend this book to anyone serious about adopting Agile Software Development. This book lacks a lot of hypes that are frequently found in books trying to sell this movement.

However, far too many managers and developers only thought agile software development as a free ticket to experiment, and forgetting the to learn bit. Manager has forgotten about using skilled person who are more agile than a beginner and discipline to carry out the project. In turbulent times, they opt for 'undiscipline' approach rather than disciplined. No wonder so many managers have a bad run with Agile movement.

The conference pulled no punches and declared that "Transitioning to agile can be difficult". Don't try to introduce this if you have an important product to deliver on time when you have no experience in Agile Development.

How do you get skilled people? These paragraphs provide the answer. Sadly, managers have conveniently overlooked this.

These paragraphs also answer how 'pair-programming' works in agile movement. Often people would argue that you do not need to start the project with skilled people because pair-programming would lift the game. According to Jim, Agile developers are skilled "from long hours of training and effective mentoring". Otherwise, pair-programming would exploit human frailty by gyrating to a comfort zone, which may not produce the desirable result. Most likely, like many things in life, it is harder to develop virtuous habit than bad ones.