A site devoted to discussing techniques that promote quality and ethical practices in software development.

Showing posts with label CodeGear. Show all posts
Showing posts with label CodeGear. Show all posts

Tuesday, March 18, 2008

CodeGear Delphi 2006.Net's TRegistry fails in Framework 2 SP1

An error has been detected when TRegistry.ReadString in Delphi 2006.Net is promoted to run in .Net Framework 2.0 SP1.

The error is the result of coding error in Borland's VCL.Net library code that is manifested into data corruption caused by Microsoft's tightening the compliance rule to conform to Unicode 5 in Framework 2 SP1.

This article will pin-point the exact cause in Borland's code. It is an very common coding error that has not been picked up in code review and the .Net Framework of the past has chosen to ignore that mistake. Thus masking out the coding error.

The VCL bug causes the use of TRegistry.ReadString() to return a string that has an additional Unicode character of value 0xFFFD appended to the end. This is the Unicode's standard replacement character whenever the encoder detects an invalid Unicode Character. The use of this character is the default action in the .Net Framework.

It is worth noting that Microsoft.Win32.RegistryKey.GetValue() for REG_SZ data does not produce this error and is not affected by the installation of Framework 2 SP1.

Let's begin the code review from TRegistry.ReadString(), which can be found in Borland.Vcl.Registry.Pas line 546.
function TRegistry.ReadString(const Name: string): string;
var
Len: Integer;
RegData: TRegDataType;
Buffer: TBytes;
begin
Len := GetDataSize(Name);
if Len > 0 then
begin
SetLength(Buffer, Len);
GetData(Name, Buffer, Len, RegData);
if (RegData = rdString) or (RegData = rdExpandString) then
begin
SetLength(Buffer, Len - 1); // <<--- Line(A) - The mistake. // .... end;
The coding error is located in the SetLength() as indicated. To understand why this is a mistake, we need to refer to the PInvoke declaration for the registry access function RegQueryValueEx(), which is the corner stone for GetDataSize() and GetData().

The declaration can be found in Borland.Vcl.Windows.Pas, line 21,265 and is reproduced in part here:
[SuppressUnmanagedCodeSecurity, DllImport(advapi32, CharSet = CharSet.Auto, SetLastError = True, EntryPoint = 'RegQueryValueEx')]
function RegQueryValueEx(hKey: HKEY; lpValueName: string;
lpReserved: IntPtr; ..... ): Longint; external;
According to MSDN documentation for CharSet.Auto, this declaration causes all strings to be marshaled as 2-byte Unicode strings and that it will be calling RegQueryValueExW variant of the RegQueryValueEx function.

According to the documentation for RegQueryValueEx(), the data returned from calling RegQueryValueExW() for type REG_SZ is a 2-byte Unicode string and the 6th parameter should contain the length of the string
If the data has the REG_SZ, REG_MULTI_SZ or REG_EXPAND_SZ type, this size includes any terminating null character or characters unless the data was stored without them.
Also worth noting that the unit of this parameter is in bytes and not in characters. Therefore for a 2-byte Unicode string, this value is always even.

Now returning to Line(A) above. Since Buffer is of type TBytes, which is an array of bytes, if one subtracts 1 from the length of Buffer that is even, this will produce an odd number of bytes. The end result is in producing a nonsensical UTF-16 Unicode string, which is expected to compose of even number of bytes. Now, instead of ending with 2-bytes of zeros, the UTF-16 null terminator, the string now contains an odd byte of zero, which is clearly not a valid UTF-16 character.

According to the knowledge base article:
the trailing NULL byte was removed. However, now the NULL byte is converted to the Unicode replacement character.
As a result, a string returned from TRegistry.ReadString() was like this, for example, "C:\Program Files" now becomes "C:\Program Files\xFFFD" or in appearance like this "C:\Program Files�"

In conclusion, the extra character tagged onto the end is the result of Framework 2 SP1 highlighting the programming error in VCL library. As mentioned, Microsoft.Win32.RegistryKey class does not have this kind of mishandling in all versions of framework. It is not a bug in Framework 2 SP1.

If you have Delphi 2006.Net program, it is therefore recommended that you include an application configuration file containing the <supportedRuntime> element that constrains your application to run only in Framework 1 as a safety measure. Apparently this bug has been rectified in Delphi 2007.Net.

Friday, December 14, 2007

Watch out when using Form Derivation in VCL.Net

This is another case where it's important to examine the implementation in greater detail than accepting the fact on the face value.

Both VCL.Net and Win Form, including Delphi Win Form, support form inheritance. But their implementation and deployment support models are poles apart.

In Delphi, you must ship the source code of your base forms to your customer in order for them to derive from it and to author their derived form in the IDE. In order to view each derive form in the IDE (D2007 included), you must first open its ancestor form in the IDE. Otherwise you'll receive am error dialog box.

This is only the requirement of the IDE. The framework works fine at runtime if the controls on the derived form are created and rearranged dynamically at runtime.

Contrasting this to Win Form, even Delphi Win Form in D2006, there is not such requirement to ship source code. To derive from a form packaged in another assembly, all you need is to reference that assembly and that's all to it.

So it's important when comparing features to look deeper than the glossy brochure.

Tuesday, October 23, 2007

Delphi in-proc server registration/unregistration code has incomplete coverage

I have just discovered the DllRegisterServer() and DllUnregisterServer() code located in Delphi's ComServ.pas file for the ComServ unit lacks complete coverage of COM usage. It is not entirely a bug in a sense. It only means that it does not cater for all situations permitted by their language framework and supported by their IDE and COM.

However, if you are in that situation, you will not be shown any visible sign other than to discover the interfaces you are publishing are not registered. OleView.exe can show you the lack of result.

Description of the problem

When you create an ActiveX project in D2006, the IDE basically generates a plain old DLL and in Delphi's parlance, a library. What it does is to export the 4 required COM In-Proc server functions, DllRegisterServer(), DllUnreqisterServer(), DllCanUnloadNow and DllGetClassObject(). The implementations of these functions are found in ComServ.pas file.

Now if you then include a type library, you can begin to define interfaces in this library. This DLL, while devoid of any implementation, is of great significance to a COM-base solution as other in-proc or local servers can implement interfaces published in this registered type library. There is no common tool, definitely not from Microsoft, to register type library (tlb) and hence it is customarily to embed this interface only type library in an in-proc server that can be registered with DllRegisterServer() and unregistered with DllUnregisterServer().

When you do this, the D2006 produced interface only COM in-proc server will not register the type library and its interfaces as well performing the unregistration process.

RegSvr32, the Microsoft standard in-proc COM registration program, dutifully reports the information reported by DllRegisterServer() and DllUnregisterServer() supplied by CodeGear's code.

Where is the problem

It has been identified that this is caused by a crack in the design and implementation code in ComServ.pas. The implementation is based on a very narrow usage scenario, perhaps in quest of efficiency.

CodeGear assumes an in-proc server always has implementation code, known as coclass, that implements interfaces described in the type library. However, this scenario is not enforced in the IDE. You can describe as many interfaces as you like in the type library without one single coclass and the IDE nor compiler complaints.

In the CodeGear narrow usage scenario, the code in ComServ.pas expects the IClassFactory implementation in the coclass, found in the unit's initialization section generally in the form of the TAutoObjectFactory.Create(), responsible for loading the type library. This then has the flow-on effect of setting ComServer.FTypeLib in ComServ.pas.

Since the unit initialization sections are executed prior to any user code, by the time TComServer.UpdateRegistry() is called, TComServer.FTypeLib is not nil and the type library registration (unregistration) function will then be called.

However, in an ActiveX, whose sole existence is to publish interfaces, the above scenario is not realized and hence by the time TComServer.UpdateRegistry() called from DllRegisterServer() or DllUnregisterServer(), the TComServer.FTypeLib remains nil.

This situation is not considered as a bug in the UpdateRegistry() and dutifully returns S_OK resulting in fooling the user.

Incidentally, code review of Delphi 3's source code shows the same incompleteness and thus expecting the same malfunction.

Work arounds

The work arounds are listed from the most preferred method to the least.
Correct the code and embedded ComServ.pas in your project
The best way is to take a copy of ComServ.pas from CodeGear's source directory and include that into your project. It is worth removing the declaration of using ComServ in your uses statement in the DPK prior to adding the customised ComServ.pas. Failure to include this file will not bring in the fixed code.

You only need to fix the DllRegisterServer() and DllUnregisterServer() as follows:
function DllRegisterServer: HResult;
begin
Result := S_OK;
try
ComServer.GetTypeLib; // **** Added
ComServer.UpdateRegistry(True);
except
Result := E_FAIL;
end;
end;

function DllUnregisterServer: HResult;
begin
Result := S_OK;
try
ComServer.GetTypeLib; // **** Added
ComServer.UpdateRegistry(False);
except
Result := E_FAIL;
end;
end;
You only have to ensure that the type library is loaded prior to the calling of TComServer.UpdateRegistry() and hence simple addition as marked above is sufficient to rectify this problem. It only introduces slight inefficient if the CodeGear anticipated scenario is realized. As a word of optimisation, one could move the call of TComServer.GetTypeLib into the TComServer.UpdateRegistry(). But these functions are hardly frequently called functions, such operation is not really warranted.
Add a dummy coclass into project
The next best solution for those not wanting to tamper with CodeGear's code is to create a dummy coclass in a unit. This unit will then include the TAutoObjectFactory.Create call in the initialization to support the scenario expected by CodeGear. At this moment, I have not explore whether or not this coclass can be made as ole non-createable to prevent code from outside this DLL from creating it via COM API, such as CoCreateInstance().

The presence of this coclass can confuse users as that coclass will show up in tools like OleView and you then need to document its reason for existence.

This represents a compromise to a clean design.
Only good for development - use the Component Install facility
This is not really a solution as such but rather a desperate move to get them registered so that you can begin to develop with those interfaces.

This technique requires one to use the "Component | Install Component ... |Import a type library" facility available in the IDE to register the type library. Since this technique does not call DllRegisterServer() and hence it can register the type library.

However, in a deployment situation, installer relies on the invocation of DllRegisterServer() and hence this technique offers no solution in deployment scenario. Furthermore, if DllUnregisterServer() fails to unregister the type library and the interfaces, this technique does not have its complementary operation.

Tuesday, July 3, 2007

A positive outcome for Delphi Win32 COM developers

With the release of the much heralded "Delphi 2007 for Win32" by a company with a new name, finally it appears pure mathematical logic has won out over ego and pigheadedness that have prevailed for the past 8 years or so.

The issue with Delphi COM local server problem that I've blogged so passionately has finally be fixed in "Delphi 2007 for Win32". This problem exists in Delphi 3 and Borland over the years has steadfastly refused to fix it or to even acknowledge that it is a bug. May be it is Vista in particular with the UAC forcing them to fix it. I dare not claim credit for this turn around despite my submission through the normal channel and the numerous brush off.

The fix is almost identical to what I've described here. The important things to note is that they should now obey their documentation in relation to the local server command line switches and that it will not bother to re-register it after a successful COM activation. The latter is the most stupid of all. Programming is pure logic.

There are implications for many who are not fluent with COM local server and may have used this 'bug' to register COM servers without the switch. Or others that are aware of this problem but have cooked up their special work around that may now be rendered ineffective or may even fail.

So they have to check with their usage to determine if they now conform with the general COM local server registration usage.

Blog Archive