A site devoted to discussing techniques that promote quality and ethical practices in software development.

Sunday, April 29, 2007

Passing array of items in Delphi.Net - revisit

I have blogged about the worrying result of passing array of items in Delphi.Net using the "array of" type.

Apparently, there is another way to specify array of items to a function, that generates the same parameter type in IL code but causing the compiler to generate slightly different code inside the function, all without warning the user.

The technique is akin to C/C++'s use of typedef, except that in C/C++, typedef does not cause the compiler to generate different code. It is merely a form of short hand notation.

Not so in Delphi.Net.

The different form of specifying the array of items in a function's argument is by means of a type alias like this:

type
TArraryOfString = array of String;

procedure TSample.DoSomething(ar: TArrayOfString);
begin
// ....
end;

This form of declaration instructs the compiler not to overwrite the ar argument with its cloned copy. If one examine the IL code for the method's signature, it is identical to that of using "array of String" syntax. This generates identical code to that using any non-Delphi.Net languages.

This is a dangerous practice because both Delphi.Net Pascal syntax generate the same method signature in IL but with very different code in the function body and the developer is not told of the difference. The developer cannot see and feel any difference. The only time a developer can feel anything is different is when one tried to pass a nil array to the function. It is even worse if that assembly is to be used with other .Net assembly which can result in difference in behaviour, see my previous blog.

The morale of the story: Do not trust any assembly that is developed in Delphi.Net with array of parameter. Always check the code, with the help of Lutz Reflector or ILDasm, to determine if the argument has been overwritten with its clone. No non-Delphi.Net language will generate code like that without explicit action from the developer.

You can test this by passing a nil in the argument where you see array of in a method. If you get an NullReferenceException on entry to the method before your source code, you can bet your dollar that if is a Delphi.Net assembly using "array of" parameter.

Borland is more interested in Delphi.Win32 compatibility than to produce a compiler that generate safe, well behave and ECMA conforming assemblies.

Wednesday, April 25, 2007

An unscientific way to resuscitate a Netcomm NB5 Modem/Router

Recently, I was helping a friend to sort out their internet problem when they were changing ISP and they wanted to see if they could reuse their modem/router, a Netcomm NB5. They actually suspected that it was slightly faulty.

Of course I needed to get into the modem to alter settings to work with different ISP. Unfortunately they had forgotten the password, which had been changed from the factory default.

To cut the story short, they also wanted to get a WiFi router if they could reuse this modem. So I decided to take it home to replace my NB1300 Plus4 with this one in my home network infrastructure.

After the factory reset I managed to fire up the modem/router connecting to my ISP. For some reason when I switched it over to operate as a bridge providing modem service to my NetGear router, I accidentally caused the NB5 to go into a state that I could no longer acquire an IP address from it. I could not ping it and obviously could not hook the web browser onto it. It was dead for all intents and purposes.

I had met this kind of situation - comatose modem/router - before but with a different brand of router. In that situation, since it was still in warranty, I sent it back.

Since I could not use this comatose modem anymore, I left this powered down for a day or two and tried again with the same result.

Suddenly I vaguely recalled people had tried to hack into or by-pass those PIN protected car stereo unit by putting the device into a freezer. Out of desperation and seeing it couldn't do any more harm, I gave this a try.

I stuck the NB5 into the freezer for about 10 to 15 minutes (I nearly forgotten about it) and then I took it out (nice and cold like a block of ice) to thaw, making sure that I wiped away the moisture as it thawed.

When it reached temperature still lower than room temperature (I did say not very scientific, didn't I?), I powered it up, did another factory reset, plugged in my PC and a minute or so, the PC managed to acquired an IP address and I managed to browse into the settings page.

That boosted my confidence allowing me to change the NB5 to a bridge and providing the connectivity to my NetGear router. This would be my modem for the next few days to determine if it is faulty.

How Security Companies Sucker Us With Lemons

Recently, Bruce Schneier, wrote an article of this title in which he attempts to explain why so many bad security devices are available on the market.

He attempted to rationalise the situation by using a theory proposed by George Akerlof in his paper "The Market for Lemons".

While Bruce directs his attention to security devices like secure USB drive and firewalls, much of his arguments apply equally well to most software package on the market.

According to George's theory, the producer of the goods (in this case software) knows their product a lot better than the buyers and hence the buyers are always placed in a disadvantage position. Most buyers of the software are suckers for buzz words used by software producers.

I know intimately of one case in which the producer of a piece of software proclaiming their software uses relational database. But of course, the vendor never tells the buyers how well their exploit this technology. If the buyer knows about this, they would definitely be fuming.

Another case when multi-tier architecture is more fashionable then the old client-server one, so it changes its tune to proclaim that it is using multi-tier design but in reality it is still a two tier client-server architecture.

George went on to suggest to break this vicious circle by providing signal from source such as knowledgeable mechanics that can check on the used cars for the buyers. Or for someone to provide honest assessment of the goods for the buyers.

I have long been an advocate for Software Advocacy on behalf of software consumer. At the moment, the playing field is so severely tilted in favour of the software producer. No wonder we have so many badly written software peddling on the market.

Wednesday, April 18, 2007

A simple way to build multiple STA COM server

In .Net, particularly when building Web Service provider, the prefer apartment type is MTA. Under this condition if you have to use functionality provided by a legacy COM in-proc server that has ThreadingModel=Apartment, the performance of the solution involving mixed model is severely impaired.

The problem is fully documented in MSDN under the Mixed Model Development section and interested reader should consult that article.

The way to overcome this is to construct a multi-STA server with each STA hosting just one STA com object. In this way each client living in a MTA can call the STA com object independently from other invocation.

This kind of construction requires some administrative work and hence not as simple as one may think.

However, for most COM in-proc server the way around this problem is a lot easier and the technique is to employ COM aggregation to host the STA COM component that you need to use by MTA client.

A step by steps guide to aggregate another COM component using ATL can be found here.

The construction recipe to use COM aggregation to build a multiple STA server is given below:
1) Create a standard COM local (executable) server that contains a COM component with an interface of your choosing. This interface needs not contain any useful method and is not used at all. If you are developing this with Delphi, beware of the problem.

I recommend the use of ATL as it is a nice and skinny framework.

2) Then use the COM development framework to aggregate the legacy STA COM in-proc server. It is often extremely difficult to tell if a COM component supports aggregation or not. Just build it and test it is a good way to determine that. Aggregation is a construction technique and hence does not contain any detectable COM registration information. Incidentally all managed COM component automatically support COM-style aggregation and this can be used to construct .Net COM local server.

3) Make sure when the COM local server contains only STA by calling CoInitialize() or CoInitializeEx( 0, COINIT_APARTMENTTHREADED ).

4) When COM server registers class object, make sure it uses the REGCLS_SINGLEUSE flag forcing this server support only one COM object, thus one STA in-proc server by virtue of aggregation.

This technique has one great advantage and that is you can leave the source code unchanged except in the server instantiation code. Instead of creating the in-proc STA server, you are creating the com object belonging to the aggregater server.

Because you are interested only in the interface belonging to the aggregated component, you can immediately cast the result into the target interface.

This technique is not without cost and they are:
* It trades off serialising calls to methods in STA components from MTA with context switching and data marshalling costs. Typically the STA component exposes a fine-grained object model and any navigation will incur these costs. Hence one should minimise the use of dotted expression.

* It uses more processes than otherwise required.

* Involvement of COM local server could result in a situation where the server does not terminate.

* Memory consumption can remain relatively high if the client is a .Net solution. This is due to the involvement of two object life-cycle management schemes - the .Net RCW relying on GC to reclaim resources and the COM which relies on reference counting. In this situation, particularly if the dotted expression is used to navigate down the object model, this kind of expression generates a lot internal RCW's and each level of the dotted expression corresponds to a COM object.

These internal RCW's will only be disposed when the GC collects them. If they are not disposed, the corresponding COM object is not removed. This situation increases memory demand above that if both client and server are unmanaged code. The way around this is to avoid using dotted expression.

Tuesday, March 27, 2007

More on Delphi.Net's handling of "array of" parameter.

I have raised a concern and claimed that Delphi.Net generated CLI incompatible assembly to handle method that contained "array of String" in my previous blog.

In this message, I would like to make one correction that I made in that blog, though it does not invalidate the claim, to set the record straight and to report the complete analysis of the issue in handling "array of" in Delphi.Net.

The correction I would like to make is in relation to the Array.Clone() in which I allegedly and mistakenly claimed that it would invoke the ICloneable.Clone() and would perform deep copy.

System.Array.Clone() does not use deep copy. In fact according to the standard documentation, System.Array.Clone() performs only shallow copy. This operation will be proven to be Delphi's Achilles heel in its naive attempt to prevent code from changing elements of an array when ECMA clearly permits.

It will be shown that Delphi.Net's naive implementation restricts the design of a method of allowing null array of anything to be passed.

Passing an array of immutable objects

This is actually a generalised case of "array of String" because System.String is an immutable class. The situation is no different from that reported in my previous blog.

In dealing with an array of anything, Delphi compiler immediately generates a call to the Array.Clone() to overwrite the incoming array with its cloned version in vein attempt to prevent changes propagating back to the caller upon returning. The following IL fragment confirms this:

.method public newslot virtual final instance
void PlayWithData(string[] ar) cil managed
{
.param [1]
.custom instance void [mscorlib]System.ParamArrayAttribute::.ctor()
.override [CloneDemoSupport]LMar.CloneDemoSupport.IDataChanger2::PlayWithData
.maxstack 6
.locals init (
[0] int32 n,
[1] int32 num,
[2] int32 len)
L_0000: ldarg.1
L_0001: callvirt instance object [mscorlib]System.Array::Clone()
L_0006: castclass string[]
L_000b: starg.s ar
So any user code will be accessing this cloned version. The minute the clone is made, each element is referring to the same address since shallow copy simply duplicates the object reference (ie. the address).

However, because each element is an immutable object, its state cannot be changed unless a new object, with therefore a new object reference, is created. But this object reference is now stored in an array that is inside the function, a la an auto-variable of array type. Its value will not be passed back to the caller since only the address or reference to the array is pushed onto the stack in passing to the function.

In this scenario, Delphi.Net is successful in preventing changes made in the function from propagating back to the caller. In so doing, it has damaged the ECMA-CLI compatibility because such an assembly produces a different behaviour that a conforming one.

Passing an array of mutable objects

Previously we have considered an array of immutable objects and have found that Delphi.Net is successful in creating ECMA-CLI incompatible methods. How successful is it in dealing with an array of mutable objects?

Once again, we have discovered that Delphi.Net does not distinguish this and blindly invokes the Array.Clone() as shown here:
.method public newslot virtual final instance
void PlayWithData(class [CloneDemoSupport]CloneDemoSupport.Foo[] ar) cil managed
{
.param [1]
.custom instance void [mscorlib]System.ParamArrayAttribute::.ctor()
.override [CloneDemoSupport]CloneDemoSupport.IDataChanger::PlayWithData
.maxstack 5
.locals init (
[0] int32 n,
[1] int32 num,
[2] int32 len)
L_0000: ldarg.1
L_0001: callvirt instance object [mscorlib]System.Array::Clone()
L_0006: castclass [CloneDemoSupport]CloneDemoSupport.Foo[]
L_000b: starg.s ar

If the function does not attempt to replace each element with a brand new element, one can use the class' methods to change the state because by definition of mutable class, the class supports methods to change the state.

Since the object's reference is not changed in the cloned version, it is now addressing the same object as that held by the caller. In so doing changing the state of that object inside the function is the same as if the caller is changing the state of the same object.

In this usage, Delphi.Net's attempt to prevent method from changing the array fails and the Array.Clone() operation is superfluous wasting CPU and memory.

It can only be successful when one replaces an element with a brand new one.

Hence depending on implementation technique, Delphi.Net can produce assembly that sometimes conforms to ECMA-CLI specification while other time it becomes incompatible. This kind of inconsistency is not acceptable.

Array of any parameter is not allow to receive nil in Delphi.Net

If you examine the above listed IL code fragment, it becomes obvious that any Delphi.Net method having a "array of" anything parameter is not allowed to pass in a nil (or null in C#) array. The designer has no choice!

The reason is that if an array is nil, calling its Clone() method will cause NullReferenceException to be thrown.

In fact, we encountered this situation in practice but at that time we naively trusting Delphi.Net compiler to be doing the right thing. Now we realise that it is due to its naive implementation. The bug was made harder to track down because the exception pointed to none of the user code but to the method's entry.

For all these problems, Borland/CodeGear should reconsider its folly in trying to make Delphi.Net, which is governed by an ECMA standard, to be compatible, without consistent success I may add, with something proprietary! It is sheer madness.

Given Borland/CodeGear's pass history in addressing their bug, I doubt Delphi.Net will be corrected. Hence anyone contemplating of developing .Net solution should take these bugs into consideration.

What do you call an MP3 player not supporting MP3?

Well, I honestly don't know what do you call it but I saw them today.

I wandered into this electrical shop in the middle of the CBD and was browsing items in the cabinet for MP3 players. I noticed in front of each player, there was a cardboard listing the make, model etc. Prominently included were features like size, with FM Tuner etc. These features were contained in a grid together with a check or cross indicating if that feature was supported or not. Of course I based that assumption on my experience with Windows GUI programming.

But for a number of Sony MP3 Walkman, I noticed something amiss. Every model had a cross against features of MP3 and WMA format. So what do you call them?

Since the salesman asked me if he could be of any assistance to me, I asked them why were those MP3 players not supporting MP3 or WMA format. He looked stunned and then quickly told me that the person who set these card up made a mistake and that I should not believed in them.

This is not good enough and this highlights a very poor work ethics.

If someone discovers that is wrong, shouldn't he/she take on the initiative to change it or bombard the management with request to correct them?

How hard is to take a marker and change a cross to a tick! It is not exactly a rocket science. The most difficult ingredient is attitude!

These days, I have met numerous developers with similar attitude. Wanting to complete a task without spending the bare minimum amount to learn the technique or principle behind a technology. All one cares is to press a few key strokes or mouse clicks and something coming out the other side. In numerous cases, they grumbled having to read another book when I recommended them to as a way of improving their skill set.

It did not crash and that is their yardstick for job well done!

If problem occurs, more of this mouse clicks and key strokes or even Google it for something remotely like the solution (usually did not bother to study it carefully) until it does not crash. If they have discovered a chant that would do the job, they would have do the chant and dance too I am sure.

This is a sad but very true situation in many development shops.

Wednesday, March 21, 2007

Good work around

This seems like a good work around the annoying activation & re-activation headache.

As one blogger says:
When it comes down to it, Microsoft’s activation system is proving more problematic for legitimate users than it is for the pirates. It doesn’t make sense for Microsoft to impose all these rules and regulations on the consumer when they themselves have built-in a way to bypass their activation system in Vista. C’est la vie.
I do not give any moral support to anti-customer device such as this kind of fragile activation scheme. Work around like this is a fair game.

Blog Archive