Dissection of an amateurish COM business solution

There is a company which should remain nameless specialized in enterprise business application managing inventories, purchasing, work order, etc. the typical business operations. This blog post is to dissect this solution to demonstrate the:

• superficiality of this company in the COM technology
• wrong choice of architecture.

The dissection does not require access to source code because they are using COM, a industry wide interop standard that requires publication of the interfaces specification. This post will show you the free tools you need to see the laughable implementation. I will also offer integrator advice on controlling the rogue COM programs in future posting.

Program structure

The actual names of the programs are also altered to save the embarrassment. This program uses executables gratuitously and was first described in my previous post. It has a controller program, much like Windows Explorer, that can launch their business applications. Each business application, fictionally called M100, M200, M135, M500, etc are COM local servers allowing the controller to communicate with it and to allow inter-application communication.

The program uses Delphi3 and this is an important fact as it will become obvious. Naturally, they all have the Delphi's COM Server bug which can be observed using Process Monitor.

Their technical lead/designer knows this much that each application must implement a common interface, called IXyzApplication in order to meet the above stated objectives. But it is shown below that their modus operandi is not only naive and amateurish showing complete lack of understanding of COM but also their developers lack proper training and mentoring. In reality they have failed to meet the objectives to implement a common interface.

To dissect this program you will need to use the free tool called OleView, which is part of Visual Studio of any version or can be download from Microsoft.

The analysis

With this enterprise business application installed, it is time to use the OleView to examine the implementation defects. If you expand the interfaces tab and scroll down until you see the IXyzApplication, you will not be encountering just one entry of IXyzApplication but you will see at least 130+ copies of this interface. If you click on each IXyzApplication, the right hand pane will tell you the server supporting this interface, its type library and the COM identity.

In fact, the reason there are so many IXyzApplication is because each one of them:

• has unique IID, thus making them unique; the name is only for human consumption and essentially irrelevant.
• each one of them is supported in their individual business application.

In effect, they should rename the IXyzApplication to IM100Application, IM120Application, IM500Application etc to be precise.

This laughable situation is not caused by the the developer's fault but a management's fault; management has failed to provide technical leadership of sufficient caliber to guide their developers and to review their work. Their technical leaders also fail to provide structural infrastructure to allow their developers to implement this interface in their business applications; their current implementation also publishes unique IXyzApplication interface as well as implementation in each business application.

Obviously, to the management, it does not require much training to use Delphi3 IDE that so easily can produce a COM application with several keystrokes. Wow, could this be the silver bullet that has eluded Frederick Brooks [BROOK]? Don't think so and will be unveiled.

Furthermore, to their untrained technical leaders with only superficial understanding of COM, the Controller can talk to their business applications and that they are all named as IXyzApplication in each application. So isn't that fine? Not quite.

Using the OleView, we can disassemble the registered type library into IDL, Interface Description Language, to see a much clearer definition of the IXyzApplication. Here are the points to be noted:

• IXyzApplication is derived from IDispatch interface
  
• IXyzApplication is declared as dual interface
• IXyzApplication has distinct guid for the interface.

For the moment we will forget some of the more ambitious ones, which violate more COM practices. Drilling into their COM characteristic further and comparing several IXyzApplication you will also noticed that:

• The same method name can have different method id's in different business applications.
• Some IXyzApplication's have different set of methods.
• Some IXyzApplication has more methods.
• Some has a different method in a method id that is used by others.
  

In other words, these developers only believe using the same interface name is all that is required. They obviously has no basic idea of COM and they are poorly served by their incompetent technical lead/architect in not providing a single definition of IXyzApplication with which they can implement it. They have no clue on the concept of implementing an interface and publishing an interface.

Digging around the installation area and using OleView reveals that they appear to have interfaces publishing DLL that they could and should include IXyzApplication there. For some unknown reason they did not. Perhaps that is out of the ordinary task supported by the IDE and that they cannot handle this.

Standing back from this problem, one wonders how the controller can communicate to all these business applications each with a unique IXyzApplication interface? The answer further illuminates their total ignorance of COM.

In order for the controller to work and for business applications to talk to another one via the 'IXyzApplication', they are not using IXyzApplication but using the following technique:

1. They are actually using dispinterface via the IDispatch interface, the base interface of IXyzApplication.
2. They are using a language/framework that does not cache the method id.

Reason 1) is supported by the fact that there is only one definition of IDispatch belonging to Microsoft but there are at least 130+ 'IXyzApplication'. In order words their declaration that their interfaces supporting dual is incorrect or at least superfluous.

Even allowing the client/server to be using IDispatch calls, the haphazardly assigned method ids and missing ones would have play havoc to client that caches the method id.

The only reason that their Delphi solution works is because Delphi does not cache the method id at compile time, unlike those using Microsoft Foundation Class/ATL or Visual Basic 6. Using cache id is perfectly acceptable because COM spec demands the consistency in method id assignment and there is only supposed to be one and only one IXyzApplication. Basically Delphi will call IDispatch::GetIDsOfNames() followed by IDispatch::Invoke() to call each IXyzApplication methods.

Using this inefficient dynamic technique ignoring the COM interface specification is the only way to deal with a haphazardly assigned method ids.

While the solution 'works' albeit in a very tightly coupled to a particular tool that happens to use an inefficient implementation, it means that it has completely destroyed the benefits of an opened industry-wide interop standard, such as COM.

It fails to use a more efficient call mechanism, such as v-table call supported by dual interface that the IXyzApplication has adored with. It cannot possibly use IXyzApplication because which one is to be used? There are 130+ IXyzApplication. The management of this company does not understand the fundamental of COM; it is not the name that matters in COM; it is the guid. If the guid is the same, then it must be the same interface regardless what you call it. Their mistaken belief produces such a laughable solution.

Secondly, it fails to support interop in a wider sense supported by COM. Their action prevents their customers from writing custom controller or to interop directly to those business applications in other tools than Delphi; COM is a language and tool neutral interop technology. However, there is a way to do this without using Delphi as a client and that is the subject of a future post.

As a result, the customers are the poor losers in this sad affair. If this software is a car, some form of medicine or financial instrument, I am sure the company responsible for this will be prosecuted for misrepresentation of the facts or producing unsafe product. But in software, this is masqueraded as innovation!

The company has not found a silver bullet; they have only addressed incompetently the accidental part [BROOK] of software building as their product has demonstrated unequivocally.

---

Frederick Brooks, "No Silver Bullet - Essence and Accidents of Software Engineering", IEEE Computer, April 1987.

Delphi in-proc server registration/unregistration code has incomplete coverage

I have just discovered the DllRegisterServer() and DllUnregisterServer() code located in Delphi's ComServ.pas file for the ComServ unit lacks complete coverage of COM usage. It is not entirely a bug in a sense. It only means that it does not cater for all situations permitted by their language framework and supported by their IDE and COM.

However, if you are in that situation, you will not be shown any visible sign other than to discover the interfaces you are publishing are not registered. OleView.exe can show you the lack of result.

Description of the problem

When you create an ActiveX project in D2006, the IDE basically generates a plain old DLL and in Delphi's parlance, a library. What it does is to export the 4 required COM In-Proc server functions, DllRegisterServer(), DllUnreqisterServer(), DllCanUnloadNow and DllGetClassObject(). The implementations of these functions are found in ComServ.pas file.

Now if you then include a type library, you can begin to define interfaces in this library. This DLL, while devoid of any implementation, is of great significance to a COM-base solution as other in-proc or local servers can implement interfaces published in this registered type library. There is no common tool, definitely not from Microsoft, to register type library (tlb) and hence it is customarily to embed this interface only type library in an in-proc server that can be registered with DllRegisterServer() and unregistered with DllUnregisterServer().

When you do this, the D2006 produced interface only COM in-proc server will not register the type library and its interfaces as well performing the unregistration process.

RegSvr32, the Microsoft standard in-proc COM registration program, dutifully reports the information reported by DllRegisterServer() and DllUnregisterServer() supplied by CodeGear's code.

Where is the problem

It has been identified that this is caused by a crack in the design and implementation code in ComServ.pas. The implementation is based on a very narrow usage scenario, perhaps in quest of efficiency.

CodeGear assumes an in-proc server always has implementation code, known as coclass, that implements interfaces described in the type library. However, this scenario is not enforced in the IDE. You can describe as many interfaces as you like in the type library without one single coclass and the IDE nor compiler complaints.

In the CodeGear narrow usage scenario, the code in ComServ.pas expects the IClassFactory implementation in the coclass, found in the unit's initialization section generally in the form of the TAutoObjectFactory.Create(), responsible for loading the type library. This then has the flow-on effect of setting ComServer.FTypeLib in ComServ.pas.

Since the unit initialization sections are executed prior to any user code, by the time TComServer.UpdateRegistry() is called, TComServer.FTypeLib is not nil and the type library registration (unregistration) function will then be called.

However, in an ActiveX, whose sole existence is to publish interfaces, the above scenario is not realized and hence by the time TComServer.UpdateRegistry() called from DllRegisterServer() or DllUnregisterServer(), the TComServer.FTypeLib remains nil.

This situation is not considered as a bug in the UpdateRegistry() and dutifully returns S_OK resulting in fooling the user.

Incidentally, code review of Delphi 3's source code shows the same incompleteness and thus expecting the same malfunction.

Work arounds

The work arounds are listed from the most preferred method to the least.

Correct the code and embedded ComServ.pas in your project

The best way is to take a copy of ComServ.pas from CodeGear's source directory and include that into your project. It is worth removing the declaration of using ComServ in your uses statement in the DPK prior to adding the customised ComServ.pas. Failure to include this file will not bring in the fixed code.

You only need to fix the DllRegisterServer() and DllUnregisterServer() as follows:

function DllRegisterServer: HResult;
begin
 Result := S_OK;
 try
   ComServer.GetTypeLib;  // **** Added
   ComServer.UpdateRegistry(True);
 except
   Result := E_FAIL;
 end;
end;

function DllUnregisterServer: HResult;
begin
 Result := S_OK;
 try
   ComServer.GetTypeLib;  // **** Added
   ComServer.UpdateRegistry(False);
 except
   Result := E_FAIL;
 end;
end;

You only have to ensure that the type library is loaded prior to the calling of TComServer.UpdateRegistry() and hence simple addition as marked above is sufficient to rectify this problem. It only introduces slight inefficient if the CodeGear anticipated scenario is realized. As a word of optimisation, one could move the call of TComServer.GetTypeLib into the TComServer.UpdateRegistry(). But these functions are hardly frequently called functions, such operation is not really warranted.

Add a dummy coclass into project

The next best solution for those not wanting to tamper with CodeGear's code is to create a dummy coclass in a unit. This unit will then include the TAutoObjectFactory.Create call in the initialization to support the scenario expected by CodeGear. At this moment, I have not explore whether or not this coclass can be made as ole non-createable to prevent code from outside this DLL from creating it via COM API, such as CoCreateInstance().

The presence of this coclass can confuse users as that coclass will show up in tools like OleView and you then need to document its reason for existence.

This represents a compromise to a clean design.

Only good for development - use the Component Install facility

This is not really a solution as such but rather a desperate move to get them registered so that you can begin to develop with those interfaces.

This technique requires one to use the "Component | Install Component ... |Import a type library" facility available in the IDE to register the type library. Since this technique does not call DllRegisterServer() and hence it can register the type library.

However, in a deployment situation, installer relies on the invocation of DllRegisterServer() and hence this technique offers no solution in deployment scenario. Furthermore, if DllUnregisterServer() fails to unregister the type library and the interfaces, this technique does not have its complementary operation.

A positive outcome for Delphi Win32 COM developers

With the release of the much heralded "Delphi 2007 for Win32" by a company with a new name, finally it appears pure mathematical logic has won out over ego and pigheadedness that have prevailed for the past 8 years or so.

The issue with Delphi COM local server problem that I've blogged so passionately has finally be fixed in "Delphi 2007 for Win32". This problem exists in Delphi 3 and Borland over the years has steadfastly refused to fix it or to even acknowledge that it is a bug. May be it is Vista in particular with the UAC forcing them to fix it. I dare not claim credit for this turn around despite my submission through the normal channel and the numerous brush off.

The fix is almost identical to what I've described here. The important things to note is that they should now obey their documentation in relation to the local server command line switches and that it will not bother to re-register it after a successful COM activation. The latter is the most stupid of all. Programming is pure logic.

There are implications for many who are not fluent with COM local server and may have used this 'bug' to register COM servers without the switch. Or others that are aware of this problem but have cooked up their special work around that may now be rendered ineffective or may even fail.

So they have to check with their usage to determine if they now conform with the general COM local server registration usage.

Class libraries not aware of LUA

[This is a reproduction of a message in my old blog. With the imminent release of Delphi 2007, I am really interested to see if they have addressed this and how.]

Many frameworks have provided class wrapper to deal with Windows Registry access and many, including the ATL's CRegKey and Borland's Delphi TRegistry have a problem in dealling with registry access in LUA (Least-Privilege Users Account).

Not that they do not have facility to deal with this but the default security access rights is set too high making unwary developers that do not develop in LUA falling into the pitfall.

For ATL::CRegKey::Open() it is default to KEY_READ | KEY_WRITE and TRegistry.OpenKey uses as its TRegistry.Access which has a default value of the same.

What makes Borland's class not capable in dealing with LUA failure than that in ATL is the return value of these respective member functions. CRegKey::Open() returns the value from ::RegOpenKeyEx(), the raw API, which Borland returns a Boolean which is true if the API returns ERROR_SUCCESS. In so doing Borland has discarded the vital information that can allow the caller to take appropriate action.

For example, when using default security access value, the CRegKey::Open() returns 0x5 for an existing key in HKLM and this means "Access is denied". If the key does not exist and that you have access rights, the return value is 0x2. In the case of Borland's TRegistry.OpenKey(), all it can return is false to indicate if something is wrong. What's wrong is that the caller is unable to determine through the lack of information. Hence in many of Borland's usage, when it fails to gain access to the HKLM with the default access rights, it will simply assumes that the key does not exist and hence performing the wrong recovery routine!

Given that Borland has already published this interface, it cannot change it. At best, it can add a read-only property to allow caller to query it after the operation but this requires code to change. Oops!!! One such Borland's library function that causing me grieve with LUA is the BDE.DbiInit().

I am just wondering if Borland's developer developing their products in LUA or in Admin accounts. From what I have seen, obviously the former.