A site devoted to discussing techniques that promote quality and ethical practices in software development.

Showing posts with label XSD. Show all posts
Showing posts with label XSD. Show all posts

Friday, October 5, 2012

Microsoft's System.Xml.Serialization.XmlSerializer generates uncompilable code for some xs:double default values

Consider the following valid schema:
<xs:schema id="MyDemoSchema"
    targetNamespace="http://tempuri.org/MyDemoSchema.xsd"
    elementFormDefault="qualified"
    xmlns="http://tempuri.org/MyDemoSchema.xsd"
    xmlns:mstns="http://tempuri.org/MyDemoSchema.xsd"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
>
  <xs:complexType name="Foo">
    <xs:sequence>
      <xs:element name="Name" type="xs:string" minOccurs="0" maxOccurs="1" />
      <xs:element name="SomeNumber" type="xs:double" 
                  minOccurs="0" maxOccurs="1" 
                  default="NaN" />
    </xs:sequence>
  </xs:complexType>

  <xs:element name="Foo" type="Foo" />
</xs:schema>

The minute you execute the following piece of code:
Foo f = new Foo();
XmlSerializer ser = new XmlSerializer( typeof(Foo) ); // ---- Error CS0103 ---

will generate a CS0103 compilation error. This is caused by the CLR runtime in which it will at that moment generates a temporary serialization assembly that actually performs the serialization and deserialization process.

It is the code generation process that fails to handle W3C acceptable xs:double values such as NaN, INF and -INF producing uncompilable code.

In brief, but will provide a more detail expose below, is that it generates code like this:
   if( x != NaN ) { < --- causes CS0103
     // do something 
   }

Which naturally does not compile and it generates the same error message as when the XmlSerializer is instantiated. You can't even use the logical operator to test if the value of System.Double is Double.NaN.

This is the exact code fragment of the generated code causing the CS0103:
if (((global::System.Double)o.@SomeNumber) != NaN) {
    WriteElementStringRaw(@"SomeNumber", @"http://tempuri.org/MyDemoSchema.xsd", 
    System.Xml.XmlConvert.ToString((global::System.Double)((global::System.Double)o.@SomeNumber)));
}

There is no way this line can compile. It has to be corrected like this:
if ( !System.Double.IsNaN ((global::System.Double)o.@SomeNumber)  ) {
    WriteElementStringRaw(@"SomeNumber", @"http://tempuri.org/MyDemoSchema.xsd", 
    System.Xml.XmlConvert.ToString((global::System.Double)((global::System.Double)o.@SomeNumber)));
}

Detail Discussion and work around

To see how this manifested into a runtime error, you can use sgen.exe together with the /keep switch to look at what happen. The catch-22 is that if your schema's default value is NaN, sgen.exe will fail to generate the serialization dll because the generation process generates the above mentioned uncompilable code and you can't see the generated code.

So in order to see what is happening, you replace the default value with a double that you can recognised. In my experiment, I used -5.55.

Once your schema is modified, xsd.exe is used to generate C# classes that are bundled into an assembly, you can then use sgen.exe with the /keep switch to generate the serialization dll together with the source file. It also generates a file with the extension .cmdline which is essentially the response file for CSC.

Using our default value as search string, you can quickly locate the potentially offending piece of code. Below are the steps to prove that code generated by the serialization process is faulty and how to rectify it.

Now correct the line where it checks for -5.55 to using System.Double.IsNaN() as shown above.

The next step is to rebuild the serialization dll and to do that you need to modify the schema to change the default value back to NaN.

Then you use XSD to generate the class and rebuild your assembly.

Next you modify the cmdline file as follows:
  • remove the setting where it references System.Xml.dll to avoid duplicate reference compilation error. 
  • remove the /D: _DYNAMIC_XMLSERIALIZER_COMPILATION 
  • to aid experimentation change the /debug- to /debug.
Check the response file for the location of the CS file. You should execute this response file in that directory. For example if the source file is like this "OutTestDir\5fm3rmet.0.cs", you should run your CSC from the parent directory of "OutTestDir".

Next is to compile and generate the serialization dll which by now should have the offending line corrected using Double.IsNaN().

Next you modify the project for the dll that once called XmlSerializer constructor by
  • adding a reference to the serialization dll. 
  • then replace following piece of code where you instantiate an instance of XmlSerializer:
Foo f = new Foo();
StringBuilder buffer = new StringBuilder();
using( XmlWriter writer = XmlWriter.Create( buffer, null ) )
{
   XmlSerializer ser = new XmlSerializer( typeof(Foo) ); // <-- to be replaced
   ser.Serialize( writer, f );
}

With this:
Foo f = new Foo();
StringBuilder buffer = new StringBuilder();
using( XmlWriter writer = XmlWriter.Create( buffer, null ) )
{
   Microsoft.Xml.Serialization.GeneratedAssembly.FooSerializer ser = 
       new Microsoft.Xml.Serialization.GeneratedAssembly.FooSerializer();
   // XmlSerializer ser = new XmlSerializer( typeof(Foo) ); // <-- to be replaced
   ser.Serialize( writer, f );
}

If you have the PDB, you can easily step through the code to convince yourself that your fix indeed does work.

Let's hope Microsoft will fix this bug which is in .Net 4. May be .Net 5 has fixed this?

Sunday, December 5, 2010

Using WCF to produce Web Service Contract documents that must use a supplied schema

This post is to describe a very common scenario in SOA/Web Service world. To avoid chaos in SOA world, practitioners are encouraged to use "Standardization of Service Data Representation" so that services and clients are communicating using standardized or common vocabulary. They may be standards data representations specified within an enterprise or by trade groups like MIMOSA and are not necessary standard endorsed by W3C.

The problem at hand is to produce service contract documents that a system needs to interact with abstractly and the data interchanges must use a standardized or common representation. For ease of discussion, let's assumed the standardized data is from ACME Enterprise and supplied in schema file called ACMEEnterprise.xsd.

One way to do this is to use a process called Contract-First or Schema-First. This gives the designer the maximum control in what to put into the Service Contract. Service Contract, include WSDL and XSD, is not just for machine to execute but also containing information that are useful to service producers and consumers. However, it is not for the faint-hearted; it is only for the most determined soul.

Instead of using Contract-First, in this post, I am describing the necessary steps and settings of using WCF to generate Service Contract documents that use the prescribed data representation. In the frequently described scenarios, the developer is responsible for specifying the data and service contract. But in the problem at hand, much of the data representation or data contract, are predetermined.


Designing a Service Contract using WCF conforming to supplied data representation

As stated previously, the data representation is supplied in ACMEEnterprise.xsd and the targetNamespace is "http://ACMEEnterprise.org/2010/12/ACMEEnterprise.xsd". All data or message exchanges must use types specified in this schema file. The Service Contract many specify other data contracts it sees fit but if it is to describe data for ACME Enterprise, it must use the types specified in ACMEEnterprise.xsd.

Step 1 - Produce the .Net classes

The first step is to convert the types specified in the ACMEEnterprise.xsd into .Net classes that we can use in WCF constructs. For illustration purpose, I use C# but you can use any other .Net languages.

This can be achieved by using XSD.exe or SvcUtil.exe. Most WCF materials will recommend one to use SvcUtil.exe but if the schema is specified using the full set of W3C XSD Schema syntax, the chance that SvcUtil can process your schema file is slim. The reason is that SvcUtil is designed to work with DataContractSerializer which maps the CLR data types to XSD's or vice versa and that the Data Contract model only supports a limited subset of the W3C Schema specifications.

You have a better chance of successful conversion by using XSD.exe provided that you follow the caveat. What CLR namespace you choose to use is immaterial and is not something that will affect the wire format. They are local artifacts affecting only your .Net solutions.

Step 2 - Incorporate the generated file into your project

The next step is to incorporate the generated file into your project and begin using the types in that files to design your Service Contract as if you are dealing with normal WCF DataContract types.

The main thing to note is that each class generated is adored with the XmlTypeAttribute declaring the Namespace corresponding to the targetNamespace in ACME, like this:
[System.Xml.Serialization.XmlTypeAttribute
        (Namespace="http://ACMEEnterprise.org/2010/12/ACMEEnterprise.xsd")]
    [System.Xml.Serialization.XmlRootAttribute("employee",
          Namespace="http://ACMEEnterprise.org/2010/12/ACMEEnterprise.xsd", 
          IsNullable=false)]
    public partial class Employee : Person {
        // . . . 
    }

It is vitally important that this namespace is maintained when we generate the Service Contract documents for this type.

Step 3 - Mark the Service Contract with XmlSerializerFormatAttribute

This is a very important point. You may apply this attribute to only those Contract Operations that require to use XmlSerializer. In my case since every operation is using this serializer, I apply this attribute to the entire service contract. If you do not apply this attribute, types that came from ACMEEnterprise.xsd will be placed under the targetNamespace of "http://schemas.datacontract.org/2004/07/ACME.Enterprise" like this:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:tns="http://schemas.datacontract.org/2004/07/ACME.Enterprise" 
 elementFormDefault="qualified" 
 targetNamespace="http://schemas.datacontract.org/2004/07/ACME.Enterprise" 
 xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:complexType name="Person">
    <xs:sequence>
      <xs:element name="ageField" type="xs:int" />
      <xs:element name="firstNameField" nillable="true" type="xs:string" />
      <xs:element name="genderField" type="tns:Gender" />
      <xs:element name="hobbyField" nillable="true" type="xs:string" />
      <xs:element name="lastNameField" nillable="true" type="xs:string" />
      <xs:element name="secretNumberField" nillable="true" type="xs:long" />
    </xs:sequence>
  </xs:complexType>
This effective produces a different type on the wire. The use of XmlSerializerFormatAttribute retains the original targetNamespace.

Step 4 - Build the WCF Service Library and Generate the Contract documents

After you have finished building the WCF Service Library you can use SvcUtil.exe to produce the Service Contract documents. The process will generate the WSDL as well as the companion XSD files. While this process with the aid of XmlSerializerFormatAttribute preserves the targetNamespace, as shown below, for the types that are used in this service library and placing them in a XSD file resembling the original XSD file, the process is at best of low fidelity. That is the process loses information that are in the original documents deem not needed in WCF. Information such as <annotation>, <documentation>, and <restriction> elements are lost. The schema file only contains a subset of the types described in the original schema; it includes only types used in the service.
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:tns="http://ACMEEnterprise.org/2010/12/ACMEEnterprise.xsd" 
 elementFormDefault="qualified" 
 targetNamespace="http://ACMEEnterprise.org/2010/12/ACMEEnterprise.xsd" 
 xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:complexType name="Person">
    <xs:sequence>
      <xs:element minOccurs="0" maxOccurs="1" name="FirstName" type="xs:string" />
      <xs:element minOccurs="0" maxOccurs="1" name="LastName" type="xs:string" />
      <xs:element minOccurs="1" maxOccurs="1" name="Age" type="xs:int" />
      <xs:element minOccurs="1" maxOccurs="1" name="Gender" type="tns:Gender" />
      <xs:element minOccurs="1" maxOccurs="1" name="SecretNumber" nillable="true" type="xs:long" />
      <xs:element minOccurs="1" maxOccurs="1" name="Hobby" nillable="true" type="xs:string" />
    </xs:sequence>
  </xs:complexType>

You may replace the regenerated schema file for types for ACME Enterprise with the original one without problem so that it contains all the valuable information.

The process described here is a low fidelity process. It does not allow designer to include annotation in the WSDL file. The only way to create a full fidelity Service Contract is to use Contract-First process by authoring the messages and then the WSDL. This will be reported in full post.


"SOA Principle of Service Design" Section 6.3 "Types of Service Contract Standardization"

Wednesday, December 1, 2010

Caveat in using xsd.exe

Here is a trap that many can fall into when using xsd.exe:
1) If your xsd schema file uses <xsd:import> to add multiple schemas from different target namespace into the document, xsd ignores the schemaLocation attribute value.

In this case, you need to specify those imported xsd files on the command line.

2) If the document uses <xsd:include>, xsd uses the schemaLocation attribute value.

While it is great to use the latest and greatest but in many situation, particularly when you are given an XSD authored in other platform, SvcUtil.exe /dcOnly will frequently unable to handle XSD Schema syntax that XSD.exe can handle.

In that situation, alternately you can use the /importXmlTypes (/ixt) and ended up with types that implements IExtensibleDataObject, something that only existing in Microsoft world and may even impact on interoperability, certainly a J2EE Web Service does not have this.

Blog Archive