Designing API - Bad API can really hurts

An article titled "API Design Matters" is a must read article for any one in charge of designing API and has this to say:

Good APIs are a joy to use. They work without friction and almost disappear from sight: the right call for a particular job is available at just the right time, can be found and memorized easily, is well documented, has an interface that is intuitive to use, and deals correctly with boundary conditions.

So, why are there so many bad APIs around? The prime reason is that, for every way to design an API correctly, there are usually dozens of ways to design it incorrectly. Simply put, it is very easy to create a bad API and rather difficult to create a good one. Even minor and quite innocent design flaws have a tendency to get magnified out of all proportion because APIs are provided once, but are called many times. If a design flaw results in awkward or inefficient code, the resulting problems show up at every point the API is called.

The article went on to provide a list of guidelines. It also offers the following top advice:

APIs should be designed from the perspective of the caller. When a programmer is given the job of creating an API, he or she is usually immediately in problem-solving mode: What data structures and algorithms do I need for the job, and what input and output parameters are necessary to get it done? It's all downhill from there: the implementer is focused on solving the problem, and the concerns of the caller are quickly forgotten.
[...]
This alternative definition requires the implementer to think about the problem in terms of the caller.

This recommendation is not dissimilar to the advice "Your users is not you" or another saying is "there are more users than developers".

The best way to deal with this as alluded in the article is to use TDD (Test Driven Design) to combat this. Let the customer, as suggested by the paper, to write the Unit Test code.

A good API reads correctly and sensibly in the user's code and not in the implementer code. Far too many 'designer' starts the API design from an implementation point of view concerning too earlier with the implementation details, which can lead to a break down in abstraction. APIs are the visible interfaces to these abstractions.

In component technologies that heavily relying on interfaces, it is important the consultants or designers begin the process on the interface.

Using UML diagram is a good starting point as you can quickly play out the scenario to see the effect. At the same time it can document the requirements (Activity/Use case diagrams), how it is to be used (sequence diagrams), or error states (State Diagrams). Then at a later stage, have executable documentation in the form of Unit Test code.

The second most import point in my mind is to strive for 'miminalist' approach:

The fewer types, functions, and parameters an API uses, the easier it is to learn, remember, and use correctly. This minimalism is important. Many APIs end up as a kitchen sink of convenience functions that can be composed of other, more fundamental functions.

This topic was given a detail treatment by Scott Meyers, "When it comes to encapsulation, sometimes less is more.". Or the YAGNI principle.

I once worked for a company that prides itself in designing kitchen-sink API, regardless if it is COM or pure C-style interface. Methods are thrown into any place it can stash regardless if they make sense. Management only interested in getting things out quickly and applying bums-on-seats resource allocation algorithm and the result speaks voluminously of the foolishness of this myopic style, for a long time.

The article touches on the issue confronting the immutability, a term not used by alluded to, which can cause "crud that, eventually, does more damage than the good it ever did by remaining backward compatible.". When that happens, it is better to start again!

The paper correctly lays blame on education, particularly many developers are not interested or have the patience and time to learn the art and are only interested in getting something 'working' in the IDE. Experience is also overlooked when selecting someone to the task, as identified by the paper.

The example the author used to illustrate a bad API design touches on a topic that .Net/Java should have considered. The author complains about the destruction of the input parameter when using the Select() method and the API method was not dutifully documented.

In C++, you have const member function and const parameters to allow compiler to catch any misuse and to tell user of the consequence of calling the method. But this is sadly not available in .Net/Java. While you have out and ref qualifier but they serve a different purpose.

In .Net/Java, one cannot be sure what would happen to a parameter when you make a call; the method could change the state of the object, as illustrated in the example, it could take a copy of the data (cloning it), or it could hold onto the reference of the object. The latter allows the holding object to change the state of an object it has a reference to and that changes can then be reflected in some other part which has also holds a reference to this object.

All these unknowns require explicit documentation that not too many developers read and can make the API harder to use. My advice is if in doubt, use the "Learning Test" test pattern, which also has an effect of detecting changes in API's behaviour over time.

CLR does not have any support to make sure a method does not alter the state of an object; VB has a language-level construct but it is just a smoke and mirror.

On the topic of cloneable, this can be a double-edged sword. In defense of .Net, the general recommendation is not to implement ICloneable. One such problem is what if an inner class does not support ICloneable and the other is that the caller has no idea if a deep or shallow copy is used. In many way this is the flip side of C++ in which copying an object is a natural behavior.

---

"Test-Driven Development - by Example" by Kent Beck, Addison Wesley, 2004, Chapter 26, "Learning Test"

Allowing error message to corrupt download - www.ingdirect.com.au - Not Nice

For the last few months, whenever I downloaded banking data from ING Direct in Quicken format, the download data failed to be imported into my accounting package complaining about import data corruption.

Not easily deterred by this kind of things particularly the downloaded file was a text file and that I was very knowledgeable in QIF file format, I opened the file in a text edit and the reason was so obvious.

What happened was that at the end of the download of the correctly structured QIF data, their .Net web server application must have thrown an exception and their application then formatted the message into HTML format ramming it into the QIF file. Here is the error message:

Server Error in '/Client' Application.

---

Runtime Error

Description: An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.

Details: To enable the details of this specific error message to be viewable on remote machines, please create a <customerrors> tag within a "web.config" configuration file located in the root directory of the current web application. This tag should then have its "mode" attribute set to "Off".

<!-- Web.Config Configuration File -->

<configuration>
   <system.web>
       <customerrors mode="Off">
   </system.web>
</configuration>

Notes: The current error page you are seeing can be replaced by a custom error page by modifying the "defaultRedirect" attribute of the application's <customerrors> configuration tag to point to a custom error page URL.

<!-- Web.Config Configuration File -->

<configuration>
   <system.web>
       <customerrors mode="RemoteOnly" defaultredirect="mycustompage.htm">
   </system.web>
</configuration>

Not very nice. I doubt the download feature was used much otherwise people would have complained or people just believing mistakenly that the Internet corrupted the download.

Once I deleted the error message from the QIF file, I could import it successfully. Have they actually tested their web application at all? I wonder!

Fixing up Borland Delphi's COM Server Registration Problem

The story where Borland does not even understand their requirement for COM Server registration and then translating that to code has been described in the most kiddish format in my other blog message.

I doubt Borland will have the courage to admit mistake and fix their problem as this problem exists in their product unchanged since Delphi 3 to the latest, including those not yet released product. Hence it is a dead loss trying to get Borland to do something. I am going to show you how to fix this problem. It is as easy as learning "Mary has a little lamb" rhyme.

Before I'll show you the fix, let me reproduce their documentation partially here:

Start mode	Switch	Meaning
smAutomation	embedding	The application was started by Windows in response to a request from an automation controller.
smRegServer	regserver	The application was started only to add the server to the system registry.
smStandalone		The user started the application as a stand-alone, interactive application.
smUnregServer	unregserver	The application was started only to remove the server from the system registry.

Table 1 - Permissible COM Server switches

For running the automation server as a stand alone application, you do not include any switch.

From the above requirement, it seems pretty obvious that you only have to run through the COM registration manipulation code when you encounter the switches /regserver or /unregserver. Incidentally this is the standard stuff if you build the COM local server using VB6, MFC and ATL.

Now it is time to reveal how Borland handles this situation badly. To see this you can locate this code fragment in ComServ.pas at around line 373, if you have access to Borland's Delphi tool or simply search for TComServer.Initialize in Unit ComServ. It is reproduced here in Listing 1,

// Listing 1
procedure TComServer.Initialize;
begin
try
  UpdateRegistry(FStartMode <> smUnregServer);
except
  on E: EOleRegistrationError do
    // User may not have write access to the registry.
    // Squelch the exception unless we were explicitly told to register.
    if FStartMode = smRegServer then raise;
end;
if FStartMode in [smRegServer, smUnregServer] then Halt;

ComClassManager.ForEachFactory(Self, FactoryRegisterClassObject);
end;

In merely 9 lines (ignoring comments), the two bold red lines are erroneous and I am going to show you why.

Let's take the bug number 1 and that is the COM Server registration code (the first line in red).

If you assign the values listed in column 1 in table 1 to the variable FStartMode, one value at a time, and then ask yourself under what value does the function UpdateRegistry() is not called.

If your answer is that UpdateRegistry() is called regardless of the value of FStartMode, then you are correct and you begin to see how silly that piece of code is.

UpdateRegistry() is a procedure that executes COM registration steps when the parameter is true. Otherwise it performs COM unregistration steps.

If you have a basic understanding of COM, you will realise that by the time COM successfully launches your local server via a client's call to COM method like CoCreateInstance(), your server must have been registered properly. So in that situation a Delphi automation server will have a FStartMode = smAutomation.

If you have followed me so far, UpdateRegistry() is called regardless of the switch and this also include the situation when you, the COM server, are launched by COM. So why do you then have to update the registry again? Doing so is like you have just flew into an airport and you still insist on buying an inbound ticket. It is silly.

To fix that all you have to do is to check if the FStartMode is smRegServer or smUnregServer. If so then called UpdateRegistry().

Now the second bug requires one to perform a function calls tracing code review. But in short, no one in the possible call chain throws EOleRegistrationError the one that Borland has programmed to catch (see the second line in red in Listing 1). Instead the code throws EOleSysError when there is COM registration error.

If you look up Delphi documentation you can see that these are two distinct exceptions both derived from EOleError and that they are siblings to EOleError.

So the corrected code is as follows, Listing 2:

// Listing 2
procedure TComServer.Initialize;
begin
try
  if FStartMode in [smRegServer, smUnregServer] then
    UpdateRegistry(FStartMode <> smUnregServer);
except
//    on E: EOleRegistrationError do
  on E: EOleError do
      // User may not have write access to the registry.
    // Squelch the exception unless we were explicitly told to register.
    if FStartMode = smRegServer then raise;
end;
if FStartMode in [smRegServer, smUnregServer] then Halt;

ComClassManager.ForEachFactory(Self, FactoryRegisterClassObject);
end;

I have left Borland's incorrect catch statement there for comparison. Comparing this to that shown in Listing 1, you can see that it is pretty obvious that someone must have accidentally deleted the test for /Regserver or /Unregserver (the line in greed) or someone must be high on illegal substance and thinking what is left in Borland's code is a pretty cool optimisation.

As mentioned, many people have reported this bug before and Borland showed a complete lack of consumer concern to fix this problem. Incidentally, once you have registered, the subsequent calls to UpdateRegistry( true ) does not cause problem in LUA because COM's RegisterTypeLib() performs a read check to see if the COM information is the same. If so it does not update it. Otherwise it does. When that happen, you need power users or higher privilege.

If you want to replace Borland's buggy code, all you have to do is:

• Take a copy of ComServ.pas to somewhere safe.
• Edit the TComServer.Initialize to that shown in Listing 2.
• In the Delphi's COM Server project, include a copy of this file.
• Delete all the dcu file and close the entire project/group. Delphi IDE has this crazy frequently happening moment of memory lapse that it fails to pick up that your project now has a new file. Pretty dumb stuff.
• Reopen this and rebuild it.

This automation server should now work properly. The best way to test this is to unregister the COM Local server (just the executable) in an Admin account. Then start the automation server as a stand alone in LUA. In this situation, the program should start up properly. If this is built using Borland's buggy code, it will throw an exception and that is not the one Borland programmed to catch.

It is this simple to fix Borland's bug!