Some worrying experience with TrueCrypt 5.1

With a recommendation from Bruce Schneier, how can I not to give this Open-Source on-the-fly encryption tool a try.

In summary, my confident in this tool is severely dented and tested. It is not its encryption capability but in its operations:

1. When it goes bad, it appears to lock up the machine and the machine seems to misbehave becoming uncooperative. Ever since I had my XP Tablet PC (512M running XP Pro with minimal AV running on demand and is set up to run LUA) for over 2 years, I cannot remember I have to force reboot it so frequently until I am testing TrueCrypt. It almost wears out my reset button on my tablet.
2. Often, it is not a lock up but for some unknown reason it tkes a long time (~4 minutes) to allow the machine to come out of a comatose state. There is no sign that it will come out of this comatose state. All one can do is to wait and pray.
   
3. When the file container is on a Network Share, copying files out of it produces some very worrying result. The only way to get the PC back to normal is by rebooting it.
4. Unreliable in dealing with waking up from hibernation and running TrueCrypt.exe. It leads to cases where it cannot reconnect to the device driver in Traveler mode. As said, it is not happening all the time but the pattern leading to this has not yet been identified.
   

Since it is a tool that is designed to protect the most precious information, it should function as reliable as NTFS file system driver or even the Network driver. It should be rock solid and cannot misbehave like the scenarios I am elaborating below.

The facts of not knowing when it will fail, when I need to restart my machine as it cannot deal with waking up from hibernation, or when it locks up my machine have dented my confidence in this tool. The unknown is killing the confidence.

Mounting File Container from Network Share.

In this usage scenario, two PCs were involved. The remote machine, an XP Pro, contained the file container and a client machine, another XP Pro machine, was used to mount this container. The was never a chance that multiple client was mounting the same container. The exercise was to copy the entire contents, a directory tree full of PDF and chm files, from this container to a directory on the remote machine.

Each file in the container was not large (~24M) but there were many. XCopy command was used rather than the Windows Explorer.

After copying almost 80% from this container (2G), the client machine suddenly reported Delay Write Failed, Event ID 26 & 50. After that the Network Share was inaccessible though I could connect to it by IP address. The remote machine had not gone down at all as I could use RDP (Remote Desktop) to access it.

What was strange was that in Windows Explorer on the client machine it was showing the directory containing the container on the remote as being compressed and the file container was missing,


But the RDP session confirmed otherwise. After rebooting the client PC, the directory containing the file container on the remote machine and the container file were shown as not compressed.

What had gone wrong? Why was the Delay Write Failure causing the client machine to misbehave so badly.

Using it in Traveller Mode.

On my tablet PC, situations had been encountered when I tried to restart TrueCrypt.exe after the machine came out of hibernation only to encounter the following dialog boxes,
The report from System Information confirmed that TrueCrypt.sys had been stopped and hanging around.
This forced me to restart the machine defeating the purpose of hibernation.

I had been diligent to ensure all mounted volumes were dismounted and TrueCrypt.exe was terminated properly prior to put my machine into hibernation. What was causing this problem? Was it because the machine was in LUA? But then again, why prior situations of coming out of hibernation did not cause this problem?

The disturbing fact was that the machine has gone into hibernation and out of it for a number of times before encountering this problem. So far there did not appear to be any pattern to cause this problem and this unpredictable behavior was far worse than outright failure.

Lock up but technically not a lock up, if you wait long enough.

There were operations with TrueCrypt.exe or interactions with the mount drives that caused the Tablet PC (the Desktop XP did not seem to be affected by this) to appear to lock up. The behavior or parts of Tablet XP that seemed to be affected were:

• the Windows Explorer, even the File Open Dialog box was affected when invoked in another program, such as MSPaint.
  
• Process Explorer appeared to be unable to delete process,
  
• inability to use Windows Explorer. Command line operations unhindered unless it was querying a mounted drive's directory information.

Initially, I thought my machine froze as I could not even delete the TrueCrypt process and had to cold boot the machine. Later, with patience waiting for a few minutes (~3-5 minutes), the lock state was finally released.

During that time, the CPU was just ticking over with plenty of reserve around and memory was not in demand. So what was happening? I suspected the TrueCrypt driver was the culprit.

Why was the problem so severe that I could not do anything but wait and for how long? Everything seemed to be held back until TrueCrypt came out of comatose state. The was totally no feedback to the user. Often I did not start Windows Explorer relying solely in command prompt/PowerShell until all volumes were mounted. Still the lock up affected the desktop and start button. Terrible.

At other time, when I either mounted a volume or dismount one, it took an inordinately long time (~3-4 minutes) before TrueCrypt came out of the operation. So long that the XP ghost window came into effect. This meant the TrueCrypt was blocked in a method call unable to reach the GetMessage() call to clear the Windows Messages. Sometime it was quick to mount but terribly long to make that drive accessible. Initially I thought TrueCrypt and/or my machine was frozen but patience was the key ingredients to get it working. This was not good enough! I could mount a USB drive in seconds and reliably every time. All my file containers were on my hard drive and so it should have better response than USB drive.

At the moment, I am only using TrueCrypt for testing until my confidence is restored and that its behavior is more predictable.

Get WSJ articles for Free

The other day, I was shown of a technique to get WSJ articles for nothing. Apparently WSJ releases some articles to allow Google to present them in the News section.

There are several issues with this technique:

1. There are some links on www.wsj.com that are only accessible to subscribers and if you can't activate those pages, you cannot see the titles of those articles nor can you bring up portion of those pages. For example, the link for 'US Business' is only accessible to subscriber.
2. The technique seems to be back to front under utilizing the power of Google's search engine.

Surely, there is a better way and indeed there is and below is a better way:

1. Open your browser and go to www.google.com
2. Then click on the 'News' link.
   
3. On the News' page, click on 'Advanced News Search'
4. On the Advanced News search option page, on the edit box for 'Return only articles from the news source named', type 'Wall Street Journal'
5. Then press the Google Search button and you'll be presented with a list of WSJ free articles.

This is using the power of Google Search! If you are having trouble following those instructions, just click here for the list of articles.

As a paying subscriber to WSJ I can compare the completeness of the articles returned by Google and what are available to subscriber only. I can tell you that what's released for free is a small area and that paid subscription entitles access to many facility and data.

Still the availability of some articles which are free makes it easier to share information with others without violating copy right or subscription conditions.

CodeGear Delphi 2006.Net's TRegistry fails in Framework 2 SP1

An error has been detected when TRegistry.ReadString in Delphi 2006.Net is promoted to run in .Net Framework 2.0 SP1.

The error is the result of coding error in Borland's VCL.Net library code that is manifested into data corruption caused by Microsoft's tightening the compliance rule to conform to Unicode 5 in Framework 2 SP1.

This article will pin-point the exact cause in Borland's code. It is an very common coding error that has not been picked up in code review and the .Net Framework of the past has chosen to ignore that mistake. Thus masking out the coding error.

The VCL bug causes the use of TRegistry.ReadString() to return a string that has an additional Unicode character of value 0xFFFD appended to the end. This is the Unicode's standard replacement character whenever the encoder detects an invalid Unicode Character. The use of this character is the default action in the .Net Framework.

It is worth noting that Microsoft.Win32.RegistryKey.GetValue() for REG_SZ data does not produce this error and is not affected by the installation of Framework 2 SP1.

Let's begin the code review from TRegistry.ReadString(), which can be found in Borland.Vcl.Registry.Pas line 546.

function TRegistry.ReadString(const Name: string): string;
var
Len: Integer;
RegData: TRegDataType;
Buffer: TBytes;
begin
Len := GetDataSize(Name);
if Len > 0 then
begin
SetLength(Buffer, Len);
GetData(Name, Buffer, Len, RegData);
if (RegData = rdString) or (RegData = rdExpandString) then
begin
  SetLength(Buffer, Len - 1);  // <<--- Line(A) - The mistake. // .... end; 

The coding error is located in the SetLength() as indicated. To understand why this is a mistake, we need to refer to the PInvoke declaration for the registry access function RegQueryValueEx(), which is the corner stone for GetDataSize() and GetData().

The declaration can be found in Borland.Vcl.Windows.Pas, line 21,265 and is reproduced in part here:

[SuppressUnmanagedCodeSecurity, DllImport(advapi32, CharSet = CharSet.Auto, SetLastError = True, EntryPoint = 'RegQueryValueEx')]
function RegQueryValueEx(hKey: HKEY; lpValueName: string;
lpReserved: IntPtr; ..... ): Longint; external;

According to MSDN documentation for CharSet.Auto, this declaration causes all strings to be marshaled as 2-byte Unicode strings and that it will be calling RegQueryValueExW variant of the RegQueryValueEx function.

According to the documentation for RegQueryValueEx(), the data returned from calling RegQueryValueExW() for type REG_SZ is a 2-byte Unicode string and the 6th parameter should contain the length of the string

If the data has the REG_SZ, REG_MULTI_SZ or REG_EXPAND_SZ type, this size includes any terminating null character or characters unless the data was stored without them.

Also worth noting that the unit of this parameter is in bytes and not in characters. Therefore for a 2-byte Unicode string, this value is always even.

Now returning to Line(A) above. Since Buffer is of type TBytes, which is an array of bytes, if one subtracts 1 from the length of Buffer that is even, this will produce an odd number of bytes. The end result is in producing a nonsensical UTF-16 Unicode string, which is expected to compose of even number of bytes. Now, instead of ending with 2-bytes of zeros, the UTF-16 null terminator, the string now contains an odd byte of zero, which is clearly not a valid UTF-16 character.

According to the knowledge base article:

the trailing NULL byte was removed. However, now the NULL byte is converted to the Unicode replacement character.

As a result, a string returned from TRegistry.ReadString() was like this, for example, "C:\Program Files" now becomes "C:\Program Files\xFFFD" or in appearance like this "C:\Program Files�"

In conclusion, the extra character tagged onto the end is the result of Framework 2 SP1 highlighting the programming error in VCL library. As mentioned, Microsoft.Win32.RegistryKey class does not have this kind of mishandling in all versions of framework. It is not a bug in Framework 2 SP1.

If you have Delphi 2006.Net program, it is therefore recommended that you include an application configuration file containing the <supportedRuntime> element that constrains your application to run only in Framework 1 as a safety measure. Apparently this bug has been rectified in Delphi 2007.Net.