Recently I needed to specify exactly how I would like a specific class serialized. Suppose we have the following simple schema:

<xs:schema targetNamespace=" http://my.favourite.ns/person.xsd "
   elementFormDefault="qualified"
   xmlns:xs=" http://www.w3.org/2001/XMLSchema "
   xmlns=" http://my.favourite.ns/person.xsd" >
  <xs:complexType name="Person">
    <xs:sequence>
      <xs:element name="Name" type="xs:string" />
      <xs:element name="Surname" type="xs:string" />
      <xs:element name="Age" type="xs:int" minOccurs="0" />
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Let's call this schema President.xsd.

A valid XML instance against this schema would be, for example:

<Person xmlns="http://my.favourite.ns/person.xsd">
   <Name>Barrack</Name>
   <Surname>Obama</Surname>
   <Age>47</Age>
</Person>

Since we are serializing against a specific XML schema (XSD), we have an option of schema compilation:

xsd /c President.xsd

This, obviously, yields a programmatic type system result in a form of a C# class. All well and done.

Now.

If we serialize the filled up class instance back to XML, we get a valid XML instance. It's valid against President.xsd.

There is a case where your schema changes ever so slightly - read, the namespaces change, and you don't want to recompile the entire solution to support this, but you still want to use XML serialization. Who doesn't - what do you do?

Suppose we want to get the following back, when serializing:

<PresidentPerson xmlns=" http://schemas.gama-system.com/
      president.xsd" >
   <Name>Barrack</Name>
   <Surname>Obama</Surname>
   <Age>47</Age>
</PresidentPerson>

There is an option to override the default serialization technique of XmlSerializer. Enter  the world of XmlAttributes and XmlAttributeOverrides:

private XmlSerializer GetOverridedSerializer()
{
   // set overrides for person element
   XmlAttributes attrsPerson = new XmlAttributes();
   XmlRootAttribute rootPerson =
      new XmlRootAttribute("PresidentPerson");
   rootPerson.Namespace = " http://schemas.gama-system.com/
      president.xsd ";
   attrsPerson.XmlRoot = rootPerson;

   // create overrider
   XmlAttributeOverrides xOver = new XmlAttributeOverrides();
   xOver.Add(typeof(Person), attrsPerson);

   XmlSerializer xSer = new XmlSerializer(typeof(Person), xOver);
   return xSer;
}

Now serialize normally:

Stream ms = new MemoryStream();
XmlTextWriter tw = new XmlTextWriter(ms, null);
xSer.Serialize(tw, person);

This will work even if you only have a compiled version of your object graph, and you don't have any sources. System.Xml.Serialization.XmlAttributeOverrides class allows you to adorn any XML serializable class with your own XML syntax - element names, attribute names, namespaces and types.

Remember - you can override them all and still serialize your angle brackets.

I'm happy to announce our organization work is coming to fruition.

Bleeding Edge 2008 is taking the stage for the autumn season.

Portorož, Slovenia, October 1^st, 9:00

Official site, Registration, Sponsors

Go for the early bird registration (till September 12^th). The time is now.

Potential sponsor? Here's the offering.

One of our core products, Gama System eArchive was accredited last week.

This is the first accreditation of a domestic product and the first one covering long term electronic document storage in a SOA based system.

Every document stored inside the Gama System eArchive product is now legally legal. No questions asked.

Accreditation is done by a national body and represents the last step in a formal acknowledgement to holiness.

That means a lot to me, even more to our company.

The following blog entries were (in)directly inspired by the development of this product:

• Laws and Digital Signatures
• Reliable Messaging and Retry Timeouts
• Approaches to Document Style Parameter Models
• XmlSerializer, Ambient XML Namespaces and Digital Signatures
• Security Sessions and Service Throttling
• Reliable Message Delivery
• Reliable Message Delivery Continued
• Durable Reliable Messaging

We've made a lot of effort to get this thing developed and accredited. The certificate is here.

This, this, this, this, this, this, this, this, this and those are direct approvals of our correct decisions.

This is brilliant.

Sysinternals tools are now (actually were already when I left for vacation) available live via a web (http and WebDAV) based resource on http://live.sysinternals.comand \\live.sysinternals.com.

This means I can do the following:

[c:\]dir \\live.sysinternals.com\tools

 Directory of  \\live.sysinternals.com\tools\ *

 2.06.2008   1:16         <DIR>    .
 2.06.2008   1:16         <DIR>    ..
 2.06.2008   1:16         <DIR>    WindowsInternals
30.05.2008  17:55             668  About_This_Site.txt
13.05.2008  19:00         225.320  accesschk.exe
 1.11.2006  15:06         174.968  AccessEnum.exe
 1.11.2006  23:05         121.712  accvio.EXE
12.07.2007   7:26          50.379  AdExplorer.chm
26.11.2007  14:21         422.952  ADExplorer.exe
 7.11.2007  11:13         401.616  ADInsight.chm
20.11.2007  14:25       1.049.640  ADInsight.exe
 1.11.2006  15:05         150.328  adrestore.exe
 1.11.2006  15:06         154.424  Autologon.exe
 8.05.2008  10:20          48.476  autoruns.chm
12.05.2008  17:31         622.632  autoruns.exe 1.11.2006  
...
 1.11.2006  15:06         207.672  Winobj.exe
30.12.1999  12:26           7.653  WINOBJ.HLP
27.05.2008  16:21         142.376  ZoomIt.exe
     24.185.901 bytes in 103 files and 3 dirs
109.442.727.936 bytes free

Or, I can fire up a Windows Explorer window (or press the start key, then type) and just type: \\live.sysinternals.com\tools.

Or:

[c:\]copy \\live.sysinternals.com\tools\Procmon.exe
        C:\Windows\System32
\\live.sysinternals.com\tools\Procmon.exe =>
        C:\Windows\System32\Procmon.exe
     1 file copied

Brilliant and useful.

As promised, here are the sources from my NTK 2008 sessions [1].

Talk: Document Style Service Interfaces

Read the following blog entry, I tried to describe the concept in detail. Also, this blog post discusses issues when using large document parameters with reliable transport  (WS-RM) channels.

Demo: Document Style Service Interfaces [Download]

This demo defines a service interface with the document parameter model, ie. Guid CreatePerson(XmlDocument person). It shows three different approaches to creation of the passed document:

1. Raw XML creation
2. XML Serialization of the (attribute annotated) object graph
3. XML Serialization using the client object model

Also, versioned schemas for the Person document are shown, including the support for document validation and version independence.

Talk: Windows Server 2008 and Transactional NTFS

This blog entry describes the concept.

Demo 1: Logging using CLFS (Common Log File System) [Download]
Demo 2: NTFS Transactions using the File System, SQL, WCF [Download]
Demo 3: NTFS Transactions using the WCF, MTOM Transport [Download] [2]

[1] All sources are in VS 2008 solution file format.
[2] This transaction spans from the client, through the service boundary, to the server.

Suppose we have a document like this:

<?xml version="1.0"?>
<root xmlns="urn-foo-bar">
  <subroot>
    <value1>value1</value1>
    <value2>value2</value2>
  </subroot>
  <Signature xmlns="h ttp://www.w3.org/2000/09/xmldsig #">
    <SignedInfo>
      <CanonicalizationMethod
        Algorithm=" http://www.w3.org/TR/2001/REC-xml-c14n-20010315 " />
      <SignatureMethod
        Algorithm=" http://www.w3.org/2000/09/xmldsig#rsa-sha1 " />
      <Reference URI="">
        <Transforms>
          <Transform 
            Algorithm=" http://www.w3.org/2000/09/
              xmldsig#enveloped-signature "/>
        </Transforms>
        <DigestMethod
          Algorithm=" http://www.w3.org/2000/09/xmldsig#sha1 " />
        <DigestValue>1Xp...EOko=</DigestValue>
      </Reference>
    </SignedInfo>
    <SignatureValue>nls...cH0k=</SignatureValue>
    <KeyInfo>
      <KeyValue>
        <RSAKeyValue>
          <Modulus>9f3W...fxG0E=</Modulus>
          <Exponent>AQAB</Exponent>
        </RSAKeyValue>
      </KeyValue>
      <X509Data>
        <X509Certificate>MIIEi...ktYgN</X509Certificate>
      </X509Data>
    </KeyInfo>
  </Signature>
</root>

This document represents data and an enveloped digital signature over the complete XML document. The digital signature completeness is defined in the Reference element, which has URI attribute set to empty string (Reference Uri="").

Checking the Signature

The following should always be applied during signature validation:

1. Validating the digital signature
2. Validating the certificate(s) used to create the signature
3. Validating the certificate(s) chain(s)

Note: In most situations this is the optimal validation sequence. Why? Signatures are broken far more frequently then certificates are revoked/expired. And certificates are revoked/expired far more frequently then their chains.

1. Validating the digital signature

First, get it out of there:

XmlNamespaceManager xmlns = new XmlNamespaceManager(xdkDocument.NameTable); [1]
xmlns.AddNamespace("ds", " http://www.w3.org/2000/09/xmldsig #");
XmlNodeList nodeList = xdkDocument.SelectNodes("//ds:Signature", xmlns);
 
[1] xdkDocument should be an XmlDocument instance representing your document.

Second, construct a SignedXml instance:

foreach (XmlNode xmlNode in nodeList)
{
  // create signed xml object
  SignedXml signedXml = new SignedXml(xdkDocument); [2]

  // verify signature
  signedXml.LoadXml((XmlElement)xmlNode);
}

[2] Note that we are constructing the SignedXml instance from a complete document, not only the signature. Read this.

Third, validate:

bool booSigValid = signedXml.CheckSignature();

If booSigValid is true, proceed.

2. Validating the certificate(s) used to create the signature

First, get it out of there:

XmlNode xndCert = xmlNode.SelectSingleNode(".//ds:X509Certificate", xmlns); [3]

[3] There can be multiple X509Certificate elements qualified with h ttp://www.w3.org/2000/09/xmldsig # namespace in there. Xml Digital Signature specification is allowing the serialization of a complete certificate chain of the certificate used to sign the document. Normally, the signing certificate should be the first to be serialized.

Second, get the X509Certificate2 instance:

byte[] bytCert = Convert.FromBase64String(xndCert.InnerText);
X509Certificate2 x509cert = new X509Certificate2(bytCert);

Third, validate:

bool booCertValid = x509cert.Verify();

If booCertValid is true, proceed.

3. Validating the certificate(s) chain(s)

Building and validating the chain:

X509Chain certChain = new X509Chain();
bool booChainValid = certChain.Build(x509cert);
int intChainLength = certChain.ChainElements.Count; [4]

If booChainValid is true, your signature is valid.

Some Rules and Some Laws

We have three booleans:

• booSigValid - signature validity
• booCertValid - certificate validity
• booChainValid - certificate's chain validity

If booSigValid evaluates to false, there is no discussion. Someone changed the document.

What happens if one of the following two expressions evaluates to true:

1. ((booSigValid) && (!booCertValid) && (!booChainValid))
2. ((booSigValid) && (booCertValid) && (!booChainValid))

This normally means that either the certificate is not valid (CRLed or expired) [4], or one of the chain's certificate is not valid/expired.

[4] The premise is that one checked the signature according to 1, 2, 3 schema described above.

The Question

Is digital signature valid even if CA revoked the certificate after the signature has already been done? Is it valid even after the certificate expires? If signature is valid and certificate has been revoked, what is the legal validity of the signature?

In legal terms, the signature would be invalid on both upper assertions, 1 and 2.

This means, that once the generator of the signature is dead, or one of his predecessors is dead, all his children die too.

Timestamps to the Rescue

According to most country's digital signature laws the signature is valid only during the validity of the signing certificate and validity of the signing certificate's chain, both being checked for revocation and expiry date ... if you don't timestamp it.

If the source document has another signature from a trusted authority, and that authority is a timestamp authority, it would look like this:

<?xml version="1.0"?>
<root xmlns="urn-foo-bar">
  <subroot>
    <value1>value1</value1>
    <value2>value2</value2>
  </subroot>
  <Signature xmlns=" http://www.w3.org/2000/09/xmldsig #">
    ...
  </Signature>
  <dsig:Signature Id="TimeStampToken"
    xmlns:dsig=" http://www.w3.org/2000/09/xmldsig #">
    <dsig:SignedInfo>
      <dsig:CanonicalizationMethod
        Algorithm=" http://www.w3.org/TR/2001/REC-xml-c14n-20010315 " />
      <dsig:SignatureMethod
        Algorithm=" http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
      <dsig:Reference
        URI="#TimeStampInfo-113D2EEB158BBB2D7CC000000000004DF65">
        <dsig:DigestMethod
          Algorithm=" http://www.w3.org/2000/09/xmldsig#sha1" />
          <dsig:DigestValue>y+xw...scKg=</dsig:DigestValue>
      </dsig:Reference>
      <dsig:Reference URI="#TimeStampAuthority">
        <dsig:DigestMethod
          Algorithm=" http://www.w3.org/2000/09/xmldsig#sha1" />
        <dsig:DigestValue>KhFIr...Sv4=</dsig:DigestValue>
      <dsig:/Reference>
    </dsig:SignedInfo>
    <dsig:SignatureValue>R4m...k3aQ==</dsig:SignatureValue>
    <dsig:KeyInfo Id="TimeStampAuthority">
      <dsig:X509Data>
        <dsig:X509Certificate>MII...Osmg==</dsig:X509Certificate>
      </dsig:X509Data>
    </dsig:KeyInfo>
    <dsig:Object
      Id="TimeStampInfo-113D2EEB158BBB2D7CC000000000004DF65">
      <ts:TimeStampInfo
         xmlns:ts=" http://www.provider.com/schemas
           /timestamp-protocol-20020207 "
         xmlns:ds=" http://www.w3.org/2000/09/xmldsig #">
        <ts:Policy id=" http://provider.tsa.com/documents" />
          <ts:Digest>
            <ds:DigestMethod Algorithm=" http://www.w3.org/2000/
              09/xmldsig#sha1" />
            <ds:DigestValue>V7+bH...Kmsec=</ds:DigestValue>
          </ts:Digest>
          <ts:SerialNumber>938...045</ts:SerialNumber>
          <ts:CreationTime>2008-04-13T11:31:42.004Z</ts:CreationTime>
          <ts:Nonce>121...780</ts:Nonce>
      </ts:TimeStampInfo>
    </dsig:Object>
  </dsig:Signature>
</root>

The second signature would be performed by an out-of-band authority, normally a TSA authority. It would only sign a hash value (in this case SHA1 hash) which was constructed by hashing the original document and the included digital signature.

This (second) signature should be checked using the same 1, 2, 3 steps. For the purpose of this mind experiment, let's say it would generate a booTimestampValid boolean.

Now, let's reexamine the booleans:

1. ((booSigValid) && (!booCertValid) && (!booChainValid) && (booTimestampValid))
2. ((booSigValid) && (booCertValid) && (!booChainValid) && (booTimestampValid))

In this case, even though the signature's certificate (or its chain) is invalid, the signature would pass legal validity if the timesamp's signature is valid, together with its certificate and certificate chain. Note that the TSA signature is generated with a different set of keys than the original digital signature.

Actually booTimestampValid is defined as ((booSigValid) && (booCertValid) && (booChainValid)) for the timestamp signature/certificate/certificate chain [5].

[5] Legal validity is guaranteed only in cases where 1 or 2 are true.

There is a serious limitation present in the RTM version of WCF 3.0/3.5 regarding control of WS-RM retry messages during a reliable session saga.

Let me try to explain the concept.

We have a sender (communication initiator) and a receiver (service). When a reliable session is constructed between the two, every message needs to come to the other side. In a request-reply world, the sender would be a client during the request phase. Then roles would switch during the response phase.

The problem arises when one of the sides does not get the message acknowledgement in time. WCF reliable messaging implementation retries the sending process and hopes for the acknowledgement. All is well.

The problem is that there is no way for the sending application to specify how long the retry timeout should be. There is a way to specify channel opening and closing timeouts, acknowledgement interval and more, but nothing will define how long should the initiator wait for message acks.

Let's talk about how WCF acknowledges messages.

During a request-reply exchange every request message is acknowledged in a response message. WS-RM SOAP headers regarding sequence definition (request) and acknowledgements (response) look like this:

a1 <r:Sequence s:mustUnderstand="1" u:Id="_2">
a2    <r:Identifier>urn:uuid:6c9d...ca90</r:Identifier>
a3    <r:MessageNumber>1</r:MessageNumber>
a4 </r:Sequence>

b1 <r:SequenceAcknowledgement u:Id="_3">
b2    <r:Identifier>urn:uuid:6c99...ca290</r:Identifier>
b3    <r:AcknowledgementRange Lower="1" Upper="1"/>
b4    <netrm:BufferRemaining
b5       xmlns:netrm=" http://schemas.microsoft.com/ws/2006/05/rm" >
b6    </netrm:BufferRemaining>
b7 </r:SequenceAcknowledgement>

Request phase defines a sequence and sends the first message (a3). In response, there is the appropriate acknowledgement present, which acks the first message (b3) with Lower and Upper attributes. Lines b4-b6 define a benign and super useful WCF implementation of flow control, which allows the sender to limit the rate of sent messages if service side becomes congested.

When the session is setup, WCF will have a really small time waiting window for acks. Therefore, if ack is not received during this period, the infrastructure will retry the message.

Duplex contracts work slightly differently. There, the acknowledgement interval can be set. This configuration option (config attribute is called acknowledgementInterval) is named inappropriately, since it controls the service and not the client side.

It does not define the time limit on received acknowledgements, but the necessary time to send the acknowledgments back. It allows grouping of sent acks, so that multiple incoming messages can be acked together. Also, the infrastructure will not necessarily honor the specified value.

Now consider the following scenario:

1. The client is on a reliable network
2. Network bandwidth is so thin that the sending message takes 20s to come through [1]
3. Service instancing is set to Multiple
4. The solution uses a request-reply semantics

[1] It does not matter whether the initiator is on a dial up, or the message is huge.

What happens?

Service initiator sets up a reliable session, then:

1. First message is being sent
2. Since the retry interval is really small [2], the message will not get to the other end and the acknowledgement will not bounce back in time
3. First message is retried, now two messages are being transported
4. No acks received yet
5. First message is retried again
6. Network bandwidth is even thinner
7. First message is acknowledged
8. First message retry is discarded on the service side
9. Second message retry is discarded on the service side

[2] Under 3s.

The number of retry messages depends on the bandwidth and message size. It can happen that tens of messages will be sent before first acknowledgement will be received.

Adaptability algorithms

A good thing is that there are undocumented algorithms implemented for retry timeout. The implementation increases the reply timeout exponentially when the infrastructure detects that the network conditions demand more time (slow throughput) and allows reliable delivery (no losses). If loses are present the reply timeout decreases.

Retry timeout is actually calculated when establishing an RM session. It is based on the roundtrip time and is bigger if the roundtrip time is long.

So, when first messages in a session are exchanged, don't be too surprised to see a couple of message retries.

Wow, Stephen.

This is one of the best ideas I've heard of hedging against the dollar in terms if IT outsourcing cost. And I mean it.

I'm not in a position of valueing the description made, but I am willing to take the pill, no matter what.

What everybody needs is only to get to one million sterling project, taking half a year.

That's it, hedging done or not.

This week XML is ten years old. The core XML 1.0 specification was released in February 1998.

It's a nice anniversary to have.

The XML + Namespaces specification has a built in namespace declaration of http://www.w3.org/XML/1998/namespace. That's an implicit namespace declaration, a special one, governing all other. One namespace declaration to rule them all. Bound to xml: prefix.

XML was born and published as a W3C Recommendation on 10^th of February 1998.

So, well done XML. You did a lot for IT industry in the past decade.

If you're into Silverlight, and you should be, check out http://www.silverlightchallenge.eu, and especially sign up on http://slovenia.silverlightchallenge.eu and join one of our many developers who will participate in this competition.

More here.

Mr. Lawrence Lessig is a founder of Stanford Center for Internet and Society. He's also a chairman for the Creative Commons organization.

Lessig is one of the most effective speakers in the world, a professor at Stanford, who tries to make this world a better place from a standpoint of stupidity in terms of the copyright law.

The following is published on the CC's site:

We use private rights to create public goods: creative works set free for certain uses. ... We work to offer creators a best-of-both-worlds way to protect their works while encouraging certain uses of them — to declare “some rights reserved.”

Therefore Creative Commons stands for the mantra of some rights reserved and not all rights reserved in terms of meaningful usage of digital technology.

Being a libertarian myself, I cannot oppose these stands. Balance and compromise are good things for content and intellectual products, such as software.

Watch his masterpiece, delivered at TED.

We talked today, with Miha, a C# Yoda. Via IM, everything seemed pointless. Couldn't find a good case for the cause of the following mind experiment:

using System;
class Tubo
{
  public static void Test() {}
  private void Test() {} 
}

Note: I'm using Miha's syntax in this post.

We have a static method called Test and an instance method, also called Test. Parameter models of both methods are the same, empty.

Would this compile?

It does not.

The question is why and who/what causes this. There is actually no rationale behind not allowing this thing to compile since both, the compiler and the runtime know the method info upfront.

Since the runtime has all the information it needs, it is strange that this would not be allowed at compile time. However, a (C#) compiler has to determine whether you, the programmer meant to access a static or an instance method.

Here lie the dragons.

It is illegal in most virtual machine based languages to have the same method name and signature for a static/instance method.

The following is an excerpt from a Java specification:

8.4.2 Method Signature

Two methods have the same signature if they have the same name and argument types.

Two method or constructor declarations M and N have the same argument types if all of the following conditions hold:

• They have the same number of formal parameters (possibly zero)
• They have the same number of type parameters (possibly zero)
• Let <A₁,...,A_n> be the formal type parameters of M and let <B₁,...,B_n> be the formal type parameters of N. After renaming each occurrence of a B_i in N's type to A_i the bounds of corresponding type variables and the argument types of M and N are the same.

Java (and also C#) does not allow two methods with the same name and parameter model no matter what the access modifier is (public, private, internal, protected ...) and whether the method is static or instance based.

Why?

Simple. Programmer ambiguity. There is no technical reason not to allow it.

Consider the following:

using System;
class Tubo
{
  public static void Test();
  private void Test();
  public void AmbiguousCaller() { Test(); }
}

What would method AmbiguousCaller call? A static Test or instance Test method?

Can't decide?

That's why this is not allowed.

And yes, it would call the instance method if this would be allowed, since statics in C# should be called using a class name, as in Test.Test(). Note that the preceding example does not compile. Also note, that it is legal to have a AmbiguousCaller body as Test() or Tubo.Test().

There is another ambiguous reason. Local variables in C# cannot be named the same as the enclosing class. Therefore, this is illegal:

using System;
class Tubo
{
  private int Tubo;
}

It is. Do a csc /t:library on it.

Since you confirmed this fact, consider the following:

using System;
public class Tubo
{
  public static void StaticMethod() {}
  public void InstanceMethod() {}
}

public class RunMe
{
  public static void Main()
  {
    Tubo Tubo = new Tubo();
    Tubo.InstanceMethod();
    Tubo.StaticMethod();
  }
}

Focus on the Main method body. In Tubo.InstanceMethod() an instance method is called and a reference (namely Tubo) is used. In Tubo.StaticMethod() a static method is called and Tubo is not an instance reference, but class name.

It all comes down to programmer ambiguity. Given a chance, I would support this design decision too.

In WCF, collection data that is passed through the service boundary goes through a type filter - meaning you will not necessarily get the intrinsic service side type on the client, even if you're expecting it.

No matter if you throw back an int[] or List<int>, you will get the int[] by default on the client.

The main reason is that there is no representation for System.Collections.Generic.List or System.Collection.Generic.LinkedList in service metadata. The concept of System.Collection.Generic.List<int> for example, actually does not have a different semantic meaning from an integer array - it's still a list of ints - but will allow you to program against it with ease.

Though, if one asks nicely, it is possible to guarantee the preferred coll