A site devoted to discussing techniques that promote quality and ethical practices in software development.

Wednesday, June 27, 2007

Two very poorly understood and appropriately staffed roles in software house

Many years ago, I met a young software engineer who was considering whether or not to take up a role as a software installation writer in a software company, which was classified in the top largest private software company of a country.

Since he was worried that it would be a mundane job with probably no future into development and to improve his knowledge in Windows, I had a long conversation with him pointing out the following key aspects:
  1. It is a role poorly understood and poorly rewarded by management
  2. It is a role that holds the make or break in the success of delivering a piece of software into customer's machine.
  3. Its creation is the first point of contact with the customer and the customer's positive or negative impression of the software largely influenced by this piece of software.
  4. The person needs more skill, thus my early comment on reward, than developers of the software, as he has to ensure he knows all the quarks and weird configurations out there. Often he has no way of knowing with precision. More importantly he needs skill to detect them to plan evasive actions. On top of that of course he also needs to be aware of the make up of the software he is installing, its requirement of OS support, its hardware/software dependency, if any.
  5. He also needs to know the right way to install the software and uninstall it cleanly and in such a way not violating the Windows security.
  6. He also needs to be a competent developer so that he can supplement the installer script. Often he needs to know C/C++ as often times, that is all he can rely upon in that very instance.
He was initially skeptical of my assertion because that was the general, and still is, trend but he did take the job. After a few years on this role, he agreed with me wholeheartedly. Over the years, I have observed the same misunderstanding being repeated time and time. The manager always thinks that it is a matter of pressing a few buttons in say InstallShield to crank out the installation package so simple that it can be given to the most junior staff. Sure, some simple application would only require simple installation package. But as the sophistication grows, such naive observation at time can prove fatal.

This is the first role that I have always maintained, a view formed from the trenches, that is extremely poorly understood and appreciated by management. I tip my hat to those installer script writers whose products flawlessly guided me through, particularly OS installer.

The second role is the Build Master. I have always thought of this but do not have the same strong convection as the Installation Writer. But the book "The Build Master - Microsoft's Software Configuration Management Best Practices" by Vincent Maraia, finally tips me over.

As Vincent says:
Many project and program managers think that the actual building of a project is a pretty trivial. Their attitude is that they can simply have the developer throw his code over the wall and hire someone to press a Build button, and everything will be fine.
[...]
I recommend that you consider the build process a piece of software that you regularly revise and deploy throughout your product team.
Many projects that I have worked on or reviewed do not even have an automated project build process. Different parts of the project are built in various machines in ways no one knows what settings are used. How can such ad hoc and haphazard arrangement manages to produce consistent and reliable product? It is just sheer luck.

Once again, management, as Vincent said in his book, does not understand the skill and reason for the Build Master.

Saturday, June 23, 2007

Outlook Clone - Evolution in Windows at last

Microsoft's Outlook has been the kind of the defacto PIM/Mail Client for a long time with no challenger insight.

The reasons why it is so popular are:
  • It is a damn good Internet Mail Client.
  • A Microsoft Exchange Server Client. Not too many program can touch the Exchange Server
  • It has extremely good synchronisation ability with Pocket PC - naturally as they come from the same stable.
  • It is a good organiser for Calendar activities and tasks.
Well, Outlook's sole dominance in this arena is now being challenged by a program called Evolution, which has been around in Linux for a few years. Now, it has been ported to Windows.

I am always on the look out for ways of not paying Microsoft therefore I obviously do not pass up the challenge.

The installation in XP is rather smooth and uneventful. What is disturbing is the lack of any entry in the Start Menu nor was there any option not to create Start Menu during installation.

After installation in Admin account, I return to RANU to test Evolution for the LUA compliance. Surprisingly, it runs fine in LUA.

The installer appears to have forgotten to create the start menu. Hence to start Evolution, you need to run the Evolution.cmd in the Evolution directory. This starts a Command console briefly while it starts the StartX.exe which then runs the Step2.cmd. The comment in Step2.cmd is kind of worrying: Yes, I know this takes a while. Sorry. So be patient.

The first time it starts in an account, it will present you with the customary wizard to set up the POP account. The first thing an Outlook user will notice is the difference in user interfaces. The MS Outlook is slick and Windows-ish naturally, while the clone has the distinct alien look, but still functional.

Once that task is completed, the main screen is up. This is where I got lost completely. I was struggling to find the "Send/Receive Mail" button so that I could pull down some mail messages, which I knew were in this account.

After a bit of struggle, I found it under the File menu. But after pressing that and seeing the momentarily displayed progress window, I still could not find the Inbox messages. The WebMail access to this POP account confirmed the disappearance of the test messages. So obviously Evolution must be working.

After some exploration of the menu, I discovered the key ones to set and they are:
  1. Press "View | Layout | Show Side Bar" as this will present you the customary Outlook side bar allowing you to select folders.
  2. Press "View | Layout | Show Toolbar" as this will present you the toolbar.
Once these two settings are set, the operations and appearance of Evolution are uncannily close to Outlook.

It is early the days to proclaim that I will flip over to this but I will definitely monitor its development.

One of the claim of fame of Evolution is that it can hook up to a Microsoft Exchange Server and the Linux people have been using this for that purpose. Since I do not have an Exchange server to test it, I will take their words for it.

There are some Windows Z-order problem and is very typical for program that has forgotten to nominate the correct owner window of the dialog box. For example, when it is prompting for the password of POP account when the current one is incorrect. This sequence of windows is very untidy.

During closing down Evolution 2.8.2, it causes an application error. At the moment, I cannot determine the cause of it but it is repeatable.

Since it is free and some of the roughness is tolerable. The next is to compare this with Thunderbird 2 with the Lightning calendar plug in. So stay tune.

Bug reporting - a lesson CodeGear can learn from Open Source

My recent usage of the GNU Win32 port of Grep resulted in my discovery of a bug and in custom with my practice, I filed a report. What happened next is a lesson that Borland/CodeGear should learn from the Open Source community.

Not only did they not whinged, throwing up excuses, calling their user and bug reporter not understanding their 'feature', they digested my bug report carefully and promptly acknowledged that was a genuine bug. Even when Borland's bug has been proven beyond doubt, yet they still refused to fix it.

Within a short time, they posted me a message advising me a bug fixed version is available. Full mark for that.

Contrasting that with CodeGear's Delphi bug reports I have filed over the years, I have never received this kind of acknowledgment and quick response. More disturbingly CodeGear's Delphi product is NOT FREE and yet providing far worse support than free stuff. Hence, price does not equate to provision of support, not even meager support.

People considering CodeGear software needs to remember this before spending your money.

Saturday, June 9, 2007

Different approaches in handling users' bug report

I have used many software libraries and software packages over the years and whenever I discover something that does not agree with the intended usage, I always take the time and effort to develop the simplest scenario to demonstrate the anomaly. Then I will submit this to the developer for comments. Since I am a developer, I know one cannot always be right and if someone can show you a scenario that can duplicate the anomaly, it is a great help as half the work is done to improve the product.

Below are a number of cases of my fault reporting experiences and the vastly different experiences I have encountered. You be the judge who should deserve the wooden spoon.

Possible Microsoft Enterprise Library bug - Range Validation configuration.

Recently, I have decided to use the Microsoft Enterprise Library 3.0 and in particular the Validation Application Block. During my experimentation with using the Configuration Console, I have discovered the console is not behaving logically. After extensive experimentation, I have formed the opinion that the validation process used in the console is faulty preventing me to save the configuration file and that a work around has been found.

So I fired a comment to the forum responsible for the Ent Lib to seek the expert opinion. With speed commensurates with Internet Time, the moderator of the forum recognized a bug and raise a work item. Full mark for their effort.

Something amiss - two minor releases of Gnu Win32 port of grep behaving differently

I am a frequent user of this great tool called grep that is rooted in Unix. Recently, I had to perform some recursive search using regular expression in some C# files. I found a version of grep in my machine that allowed me to successfully locate the lines with this command line:
 grep -R --include="*.cs" -P "^[ \t]+class" *
I looked at my grep and realized it was slightly dated and wondering if there was more recent one. So I searched Google and found the two sites on SourceForge. The GNU Utilities port to Windows (UnxUtils) and the Win32-versions of GNU tools (GnuWin32); not sure why two. To cut the story short, the UnxUtils contains a old version of grep, 2.4.1, which does not have the --include and -P (the Perl Regular Expression) switches.

In the latest version of GnuWin32 version 2.5.1a-1, it has the grep with the required switches. However, it does not function according to the man page. On that site it also has an earlier release, 2.5.1a, and surprisingly, this version not only has the required switches, they function according to the man page.

Seeing these distinct differences in behavior, I duly submitted a bug report. The information provided must be clear enough to indicate something was wrong and the status had been changed and assigned to someone for further investigation. It could mean my usage of the switches combination is wrong too. In that case, 2.5.1a-1 needs to be correct. If my usage is right, and I am pretty sure I am right, the 2.5.1a-1 needs to be compared to 2.5.1a and rectified accordingly.

Problem in Borland's Delphi library.

I must admit that I am not a long time user of Borland's Delphi programming language but in the last 2 years, I have begun reading code and writing code in Delphi - both the Win32 and Delphi.Net, of course including the VCL.Net.

I am also fortunate that I am surrounded by Delphi experts who can either correct my usage or advises me of this 'Delphi specialties'. They have also fed me many interesting stuff and many I have taken on as a challenge to get them fixed as they are clearly illogical.

Many are definitely bugs while others are of their own creation. A samples can be found in this blog for example, Delphi handles COM Server, or trying to change ECMA CLI standard. There are others that are not reported in my blog.

It is particularly interesting to note that the COM problem was first discovered in Delphi 3 and numerous users tried in vain to have this rectified. The support would preferred to argue at length of the classification of this problem than to discuss the technical issues. Perhaps they were not skillful enough to understand.

My discovery actually unearths a possibility that someone in Borland has accidentally deleted the line testing the start up mode and the code still compiled correctly giving Borland an impression of correctness.

They do not see anything wrong when they are not developing in LUA. Thanks Borland for giving me a real life illustration of the danger for not developing in LUA. This may be the same reason why their TRegistry fails too.

Even putting this aside, my code analysis of their code clearly indicates that Borland's developers do not know what is the exception handler and thus have thrown the wrong exception or vice versa. This allows the application to crash catastrophically during start up.

With only 9 lines in dispute and a well described test scenario, they steadfastly maintained that there was no bug. If that is the case, why would an automation server, that is running as a stand alone, thus not requiring to be registered, fail to run in LUA? When cornered, Borland offered all sorts of stupid 'fixes', like using special registration to HKCU hive.

Stubbornness does not fix code. It only gives developers more reason to look down on this product. It gives me indisputable examples showing the poor quality of the product and more convincing argument in product evaluation or elimination. Developers that following my suggestion to address this issue came out smiling without the need to use unorthodox hacks and even considering the laughable work around from Borland.

My blog has documented many other reports of these kind of encounters. Some of the 'incorrect behaviour' are trying to be justified as features. While others the company could not bother to comment.

To be fair, their developers had acknowledged some of my finding, not documented here, to be more than spot on, exploring areas that they had not considered. But that was as far as it went. Hardly called satisfactorily resolved.

A far cry from other more mature and definitely users friendly companies or communities.

I will definitely be monitoring the Delphi 2007 to see if these issues are addressed. If they cannot even admit fault, the first step in fixing any bug, my bet is that the fault will still be there. So if you were considering upgrade, check these issues out first or else you may be getting the same old thing with a new wrapper!

Friday, June 8, 2007

Class libraries not aware of LUA

[This is a reproduction of a message in my old blog. With the imminent release of Delphi 2007, I am really interested to see if they have addressed this and how.]

Many frameworks have provided class wrapper to deal with Windows Registry access and many, including the ATL's CRegKey and Borland's Delphi TRegistry have a problem in dealling with registry access in LUA (Least-Privilege Users Account).

Not that they do not have facility to deal with this but the default security access rights is set too high making unwary developers that do not develop in LUA falling into the pitfall.

For ATL::CRegKey::Open() it is default to KEY_READ | KEY_WRITE and TRegistry.OpenKey uses as its TRegistry.Access which has a default value of the same.

What makes Borland's class not capable in dealing with LUA failure than that in ATL is the return value of these respective member functions. CRegKey::Open() returns the value from ::RegOpenKeyEx(), the raw API, which Borland returns a Boolean which is true if the API returns ERROR_SUCCESS. In so doing Borland has discarded the vital information that can allow the caller to take appropriate action.

For example, when using default security access value, the CRegKey::Open() returns 0x5 for an existing key in HKLM and this means "Access is denied". If the key does not exist and that you have access rights, the return value is 0x2. In the case of Borland's TRegistry.OpenKey(), all it can return is false to indicate if something is wrong. What's wrong is that the caller is unable to determine through the lack of information. Hence in many of Borland's usage, when it fails to gain access to the HKLM with the default access rights, it will simply assumes that the key does not exist and hence performing the wrong recovery routine!

Given that Borland has already published this interface, it cannot change it. At best, it can add a read-only property to allow caller to query it after the operation but this requires code to change. Oops!!! One such Borland's library function that causing me grieve with LUA is the BDE.DbiInit().

I am just wondering if Borland's developer developing their products in LUA or in Admin accounts. From what I have seen, obviously the former.

VCL Form Vs Win Form performance (old posting)

[This a reproduction of a message in my old blog. It is reproduced here to remind me to retest this when the BDS 2007 is released]


There are plenty of noises on the Internet regarding the performance of Windows Forms (WinForm) Versus Delphi Forms but not too many of them are willing to publish any profiler results to back the claims.
While the results published here are not exhaustive, they are repetitive and are here to compare two aspects:
  • How are Vcl.net, Vcl32 and Win form compared to each other on loading a form?
  • Are they sensitive to the number of controls on a form?

The tool I used to profile them is AQTime4 from AutomatedQA, which can be used to profile, Delphi.Win32, Delphi.Net, C#, C++ (managed or unmanaged).

Two applications are developed. One, the Large App, has 1 tab control with 5 tab pages and they contains buttons (47), combo boxes (20), edit boxes (40), rich edit controls (12), and treeview (16). All together 241 controls.

The other is a lot smaller with 1 tab control 5 tab pages and all up 40 controls of similar types.

These applications were developed in Delphi.Win32 and re-compiled to Delphi.Net and then re-coded to WinForm using either Delphi or C# all using Delphi Studio 2006. All these applications did not contain any code other than what was generated by the Form Designer.

Here are the results for the large app:

Large application with 241 controls
Build Type Method and Children Time (sec)
VCL 32 TForm2::Create() 0.16606
VCL.Net TForm2::ctor() 1.20292
WinForm/C# WinForm::ctor() 0.2645

For smaller applications here are the results:

Small Applications with 40 controls
Build Type Method and Children Time (sec)
VCL.32 [No Debug Symbols]
VCL.Net TForm1::ctor 0.53663
WinForm/Delphi TWinForm12::ctor 0.21640

Profile results on two real applications one using Delphi3 and the other Delphi2006 produced data in agreement with the trend but with a much severe ratio because they contain actual code that increases the load time further.

From these data we can conclude:

  1. VCL.Net is slower than VCL.Win32 and analysis of the call stacks in these two technologies revealed a lot more methods calls were involved in VCL.Net as compared to the lean VCL.32, even for reading form resources.
  2. VCL.Net is also slower than WinForm, regardless of the language used to program it.
  3. VCL.Net is more sensitive to the number of controls on a form as compared with WinForm, which is essentially flat. The ratio of VCL.Net : WinForm is 2.5 : 1 for a form with 40 controls as compared with 4.5 : 1 for one with 241 controls. Hence this means the load time of VCL.Net is sensitive to the number of controls on the form.
  4. The WinForm is relatively insensitive to the number of controls on the form.

So here you are.

Monday, June 4, 2007

Configuring SVN

When I started using SVN 1.4.3 (r23084) with the help of TortoiseSVN, I have struck a number of quirky things and a number of must do things:

Ignoring bin, debug, etc. files in .Net development

You can use TSVN to edit the config, which is located in
"\Documents and Settings\<user>\Application Data\Subversion" and the file name is Config with no extension.

To get it to ignore these directories or files, you need to uncomment or edit the global-ignores line. In mine, I have it set to:

global-ignores = bin obj *.suo *.obj *.pdb *.exe *.dll *.csproj.user

This will causes it to ignore the bin and obj sub-directories of a typical Visual Studio projects.

Supporting keyword substitution

Subversion supports keyword substitution even though it treats each check in as a file set. It is not as complete as that in VSS and these are the followings:
$Revision$, $Id$, $Date$, $HeadURL$ and $Author$

$HeadURL$ is particularly useful as it is expanded into the path in the repository and hence allowing one to quickly work out which branch does this file belong to. $Id$ is not as useful if you have already included $Revision$ + $Date$ + $Author$. BTW, it also supports the $Revision::$ convention to allow one to line up all the values, much like VSS.

To enable keyword substitution, you need to do:
  1. Add the above keywords into the source file.
  2. Edit the config file to include "enable-auto-props = yes"
  3. Then in the "[auto-props]" section define the file types that support the keywords.
There is a quirky thing with SVN and when you define the supported keywords for a file type, say C#, you need to include a leading dummy keyword and a trailing dummy keywords. They are kind of sacrificial keywords to be thrown away keeping the wanted ones. Otherwise the substitution logic will ignore the leading and trailing keywords. You can specify which keywords are supported for which file type. For example to specify expansion for C# file, this is what you have to do:
[auto-props]
*.cs = svn:eol-style=native;svn:keywords="xx Revision Id Date HeadURL Author yy"
The ones in italics are the dummy ones. You can use any names.

Don't put all projects into one repository

For information, see the Pragmatic Version Control for advice.

Use symbolic link to set up repository

If you are using RANU (Run as a normal users>, you should consider putting all the repositories in "\Documents and Settings\All Users\Documents\Svn-repos".

But doing so can make it hard to use command line program like svn, which uses a modified file URL convention requiring spaces to be replaced with %20. Rather than putting it on the root, you can set up a link to this directory in the root using the LinkD.exe, a tool which is part of the Windows Resource Kit. This does not damage the security settings of target directory or the link. You can use this command to do it:
Linkd c:\svn-repos "C:\Documents and Settings\All Users\Documents\Svn-Repos

Blog Archive