Accreditus: Gama System eArchive 

One of our core products, Gama System eArchive was accredited last week.

This is the first accreditation of a domestic product and the first one covering long term electronic document storage in a SOA based system.

Every document stored inside the Gama System eArchive product is now legally legal. No questions asked.

Accreditation is done by a national body and represents the last step in a formal acknowledgement to holiness.

That means a lot to me, even more to our company.

The following blog entries were (in)directly inspired by the development of this product:

We've made a lot of effort to get this thing developed and accredited. The certificate is here.

This, this, this, this, this, this, this, this, this and those are direct approvals of our correct decisions.

Categories:  .NET 3.0 - General | .NET 3.0 - WCF | .NET 3.5 - WCF | Other | Personal | Work
Saturday, July 05, 2008 1:18:06 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Reliable Messaging and Retry Timeouts 

There is a serious limitation present in the RTM version of WCF 3.0/3.5 regarding control of WS-RM retry messages during a reliable session saga.

Let me try to explain the concept.

We have a sender (communication initiator) and a receiver (service). When a reliable session is constructed between the two, every message needs to come to the other side. In a request-reply world, the sender would be a client during the request phase. Then roles would switch during the response phase.

The problem arises when one of the sides does not get the message acknowledgement in time. WCF reliable messaging implementation retries the sending process and hopes for the acknowledgement. All is well.

The problem is that there is no way for the sending application to specify how long the retry timeout should be. There is a way to specify channel opening and closing timeouts, acknowledgement interval and more, but nothing will define how long should the initiator wait for message acks.

Let's talk about how WCF acknowledges messages.

During a request-reply exchange every request message is acknowledged in a response message. WS-RM SOAP headers regarding sequence definition (request) and acknowledgements (response) look like this:

a1 <r:Sequence s:mustUnderstand="1" u:Id="_2">
a2    <r:Identifier>urn:uuid:6c9d...ca90</r:Identifier>
a3    <r:MessageNumber>1</r:MessageNumber>
a4 </r:Sequence>

b1 <r:SequenceAcknowledgement u:Id="_3">
b2    <r:Identifier>urn:uuid:6c99...ca290</r:Identifier>
b3    <r:AcknowledgementRange Lower="1" Upper="1"/>
b4    <netrm:BufferRemaining
b5       xmlns:netrm="
http://schemas.microsoft.com/ws/2006/05/rm">
b6    </netrm:BufferRemaining>
b7 </r:SequenceAcknowledgement>

Request phase defines a sequence and sends the first message (a3). In response, there is the appropriate acknowledgement present, which acks the first message (b3) with Lower and Upper attributes. Lines b4-b6 define a benign and super useful WCF implementation of flow control, which allows the sender to limit the rate of sent messages if service side becomes congested.

When the session is setup, WCF will have a really small time waiting window for acks. Therefore, if ack is not received during this period, the infrastructure will retry the message.

Duplex contracts work slightly differently. There, the acknowledgement interval can be set. This configuration option (config attribute is called acknowledgementInterval) is named inappropriately, since it controls the service and not the client side.

It does not define the time limit on received acknowledgements, but the necessary time to send the acknowledgments back. It allows grouping of sent acks, so that multiple incoming messages can be acked together. Also, the infrastructure will not necessarily honor the specified value.

Now consider the following scenario:

  1. The client is on a reliable network
  2. Network bandwidth is so thin that the sending message takes 20s to come through [1]
  3. Service instancing is set to Multiple
  4. The solution uses a request-reply semantics

[1] It does not matter whether the initiator is on a dial up, or the message is huge.

What happens?

Service initiator sets up a reliable session, then:

  1. First message is being sent
  2. Since the retry interval is really small [2], the message will not get to the other end and the acknowledgement will not bounce back in time
  3. First message is retried, now two messages are being transported
  4. No acks received yet
  5. First message is retried again
  6. Network bandwidth is even thinner
  7. First message is acknowledged
  8. First message retry is discarded on the service side
  9. Second message retry is discarded on the service side

[2] Under 3s.

The number of retry messages depends on the bandwidth and message size. It can happen that tens of messages will be sent before first acknowledgement will be received.

Adaptability algorithms

A good thing is that there are undocumented algorithms implemented for retry timeout. The implementation increases the reply timeout exponentially when the infrastructure detects that the network conditions demand more time (slow throughput) and allows reliable delivery (no losses). If loses are present the reply timeout decreases.

Retry timeout is actually calculated when establishing an RM session. It is based on the roundtrip time and is bigger if the roundtrip time is long.

So, when first messages in a session are exchanged, don't be too surprised to see a couple of message retries.

Categories:  .NET 3.0 - WCF | .NET 3.5 - WCF
Tuesday, April 08, 2008 11:33:13 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Passing Collections Through Service Boundaries, Why and How 

In WCF, collection data that is passed through the service boundary goes through a type filter - meaning you will not necessarily get the intrinsic service side type on the client, even if you're expecting it.

No matter if you throw back an int[] or List<int>, you will get the int[] by default on the client.

The main reason is that there is no representation for System.Collections.Generic.List or System.Collection.Generic.LinkedList in service metadata. The concept of System.Collection.Generic.List<int> for example, actually does not have a different semantic meaning from an integer array - it's still a list of ints - but will allow you to program against it with ease.

Though, if one asks nicely, it is possible to guarantee the preferred collection type on the client proxy in certain scenarios.

Unidimensional collections, like List<T>, LinkedList<T> or SortedList<T> are always exposed as T arrays in the client proxy. Dictionary<K, V>, though, is regened on the client via an annotation hint in WSDL (XSD if we are precise). More on that later.

Let's look into it.

WCF infrastructure bends over backwards to simplify client development. If the service side contains a really serializable collection (marked with [Serializable], not [DataContract]) that is also concrete (not an interface), and has an Add method with the following signatures...

public void Add(object obj);
public void Add(T item);

... then WCF will serialize the data to an array of the collections type.

Too complicated? Consider the following:

[ServiceContract]
interface ICollect
{
   [OperationContract]
   public void AddCoin(Coin coin);

   [OperationContract]
   public List<Coin> GetCoins();
}

Since the List<T> supports a void Add<T> method and is marked with [Serializable], the following wire representation will be passed to the client:

[ServiceContract]
interface ICollect
{
  [OperationContract]
  void AddCoin(Coin coin);

  [OperationContract]
  Coin[] GetCoins();
}

Note: Coin class should be marked either with a [DataContract] or [Serializable] in this case.

So what happens if one wants the same contract on the client proxy and the service? There is an option in the WCF proxy generator, svcutil.exe to force generation of class definitions with a specific collection type.

Use the following for List<T>:

svcutil.exe http://service/metadata/address
  /collectionType:System.Collections.Generic.List`1

Note: List`1 uses back quote, not normal single quote character.

What the /collectionType (short /ct) does, is forces generation of strongly typed collection types. It will generate the holy grail on the client:

[ServiceContract]
interface ICollect
{
  [OperationContract]
  void AddCoin(Coin coin);

  [OperationContract]
  List<Coin> GetCoins();
}

In Visual Studio 2008, you will even have an option to specify which types you want to use as collection types and dictionary collection types, as in the following picture:

On the other hand, dictionary collections, as in System.Collections.Generic.Dictionary<K, V> collections, will go through to the client no matter what you specify as a /ct parameter (or don't at all).

If you define the following on the service side...

[OperationContract]
Dictionary<string, int> GetFoo();

... this will get generated on the client:

[OperationContract]
Dictionary<string, int> GetFoo();

Why?

Because using System.Collections.Generic.Dictionary probably means you know there is no guarantee that client side representation will be possible if you are using an alternative platform. There is no way to meaningfully convey the semantics of a .NET dictionary class using WSDL/XSD.

So, how does the client know?

In fact, the values are serialized as joined name value pair elements as the following schema says:

<xs:complexType name="ArrayOfKeyValueOfstringint">
  <xs:annotation>
    <xs:appinfo>
      <IsDictionary
        xmlns="
http://schemas.microsoft.com/2003/10/Serialization/">
        true
      </IsDictionary>
    </xs:appinfo>
  </xs:annotation>
  <xs:sequence>
    <xs:element minOccurs="0" maxOccurs="unbounded"
      name="KeyValueOfstringint">
      <xs:complexType>
        <xs:sequence>
          <xs:element name="Key" nillable="true" type="xs:string" />
          <xs:element name="Value" type="xs:int" />
        </xs:sequence>
      </xs:complexType>
    </xs:element>
  </xs:sequence>
</xs:complexType>
<xs:element name="ArrayOfKeyValueOfstringint"
  nillable="true" type="tns:ArrayOfKeyValueOfstringint" />

Note: You can find this schema under types definition of the metadata endpoint. Usually ?xsd=xsd2, instead of ?wsdl will suffice.

As in:

<GetFooResponse>
  <KeyValueOfstringint>
    <Key>one</Key>
    <Value>1</Value>

    <Key>two</Key>
    <Value>2</Value>

    <!-- ... -->

    <Key>N</Key>
    <Value>N</Value>
  </KeyValueOfstringint>
<GetFooResponse>

The meaningful part of type service-to-client-transportation resides in <xs:annotation> element, specifically in /xs:annotation/xs:appinfo/IsDictionary element, which defines that this complex type represents a System.Collections.Generic.Dictionary class. Annotation elements in XML Schema are parser specific and do not convey any structure/data type semantics, but are there for the receiver to interpret.

This must be one of the most excellent school cases of using XML Schema annotations. It allows the well-informed client (as in .NET client, VS 2008 or svcutil.exe) to utilize the semantic meaning if it understands it. If not, no harm is done since the best possible representation, in a form of joined name value pairs still goes through to the client.

Categories:  .NET 3.0 - WCF | .NET 3.5 - WCF | Web Services
Thursday, September 27, 2007 10:04:47 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Approaches to Document Style Parameter Models 

I'm a huge fan of document style parameter models when implementing a public, programmatic façade to a business functionality that often changes.

public interface IDocumentParameterModel
{
   [OperationContract]
   [FaultContract(typeof(XmlInvalidException))]
   XmlDocument Process(XmlDocument doc);
}

This contract defines a simple method, called Process, which processes the input document. The idea is to define the document schema and validate inbound XML documents, while throwing exceptions on validation errors. The processing semantics is arbitrary and can support any kind of action, depending on the defined invoke document schema.

A simple instance document which validates against a version 1.0 processing schema could look like this:

<?xml version="1.0?>
<Process xmlns="http://www.gama-system.com/process10.xsd" version="1.0">
   <Instruction>Add</Instruction>
   <Parameter1>10</Parameter1>
   <Parameter2>21</Parameter2>
</Process>

Another processing instruction, supported in version 1.1 of the processing schema, with different semantics could be:

<?xml version="1.0?>
<Process xmlns="http://www.gama-system.com/process11.xsd" version="1.1">
   <Instruction>Store</Instruction>
   <Content>77u/PEFwcGxpY2F0aW9uIHhtbG5zPSJod...mdVcCI</Content>
</Process>

Note that the default XML namespace changed, but that is not a norm. It only allows you to automate schema retrieval using the schema repository (think System.Xml.Schema.XmlSchemaSet), load all supported schemas and validate automatically.

public class ProcessService : IDocumentParameterModel
{
   public XmlDocument Process(XmlDocument doc)
   {
      XmlReaderSettings sett = new XmlReaderSettings();

      sett.Schemas.Add(<document namespace 1>, <schema uri 1>);
      ...
      sett.Schemas.Add(<document namespace n>, <schema uri n>);

      sett.ValidationType = ValidationType.Schema;
      sett.ValidationEventHandler += new
         ValidationEventHandler(XmlInvalidHandler);
      XmlReader books = XmlReader.Create(doc.OuterXml, sett);
      while (books.Read()) { }

      // processing goes here
      ...
   }

   static void XmlInvalidHandler(object sender, ValidationEventArgs e)
   {
      if (e.Severity == XmlSeverityType.Error)
         throw new XmlInvalidException(e.Message);
   }
}

The main benefit of this approach is decoupling the parameter model and method processing version from the communication contract. A service maintainer has an option to change the terms of processing over time, while supporting older version-aware document instances.

This notion is of course most beneficial in situations where your processing syntax changes frequently and has complex validation schemas. A simple case presented here is informational only.

So, how do we validate?

  • We need to check the instance document version first. This is especially true in cases where the document is not qualified with a different namespace when the version changes.
  • We grab the appropriate schema or schema set
  • We validate the inbound XML document, throw a typed XmlInvalidException if invalid
  • We process the call

The service side is quite straightforward.

Let's look at the client and what are the options for painless generation of service calls using this mechanism.

Generally, one can always produce an instance invoke document by hand on the client. By hand meaning using System.Xml classes and DOM concepts. Since this is higly error prone and gets tedious with increasing complexity, there is a notion of a schema compiler, which automatically translates your XML Schema into the CLR type system. Xsd.exe and XmlSerializer are your friends.

If your schema requires parts of the instance document to be digitally signed or encrypted, you will need to adorn the serializer output with some manual DOM work. This might also be a reason to use the third option.

The third, and easiest option for the general developer, is to provide a local object model, which serializes the requests on the client. This is an example:

ProcessInstruction pi = new ProcessInstruction();
pi.Instruction = "Add";
pi.Parameter1 = 10;
pi.Parameter2 = 21;
pi.Sign(cert); // pi.Encrypt(cert);
pi.Serialize();
proxy.Process(pi.SerializedForm);

The main benefit of this approach comes down to having an option on the server and the client. Client developers have three different levels of complexity for generating service calls. The model allows them to be as close to the wire as they see fit. Or they can be abstracted completely from the wire representation if you provide a local object model to access your services.

Categories:  .NET 3.0 - WCF | .NET 3.5 - WCF | Architecture | Web Services | XML
Monday, September 24, 2007 11:19:10 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 INETA Talk: About WCF Messaging Sessions 

Just came back from today's talk on WCF session support.

Deliverables:

Remember: proxy.Close() is your friend.

Categories:  .NET 3.0 - WCF
Tuesday, March 27, 2007 7:39:27 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Security Sessions and Service Throttling 

WCF includes support for establishing a security session through a simple configuration attribute. The primary reason of a security session is a shared security context which enables clients and services to use a faster, symmetric cryptographic exchange.

WCF sessions should not be thought of in terms of HTTP based sessions, since the former are initiated by clients and the latter by the servers. In other terms, WCF sessions are there to support some kind of shared context between a particular client and a service. This context can be anything, and is not limited to security contexts.

The attribute that establishes a security session and shared context is called, well, establishSecurityContext and is present in binding configuration. An example of such a binding would be:

<bindings>
  <wsHttpBinding>
    <binding name="SecureBinding">
      <security mode ="Message">
        <message clientCredentialType="Certificate" establishSecurityContext="true"/>
      </security>
    </binding>
  </wsHttpBinding>
<bindings>

This binding allows HTTP based communication, demands message based security (think WS-Security) and uses certificates to sign/encrypt the message content. The attribute establishSecurityContext is set to true, which actually enforces a WS-SecureConversation session between the client and the service.

The following is a simplified version of what is going on under the covers:

  • Client instantiates the service proxy
    No message exchange is taking place yet.
  • Client requests a SCT (Secure Context Token)
    This is done by the infrastructure, when the first service method is called. SCT (again simplified) represents a secure context, which includes the symmetric key. The message which demands it is (per WS-SecureConversation spec) called RST (Request Secure Token).
  • Service responds with RSTR (Request Secure Token Response) message
    Session bootstraps and is ready to piggyback all further message exchanges.

What is not well known is that there is a very low limit on the number of sessions a service is willing to accept. The default is set to 10 sessions and this was changed (from 64) late in the WCF development cycle (summer 2006). So RTM ships with this default.

Service session count is greatly influenced by the instancing scheme the service is using. Since instancing is a completely different beast, let's leave this for another post (uhm, I already wrote something here). Let's just say that hitting the session problem is a non-issue when using singleton instancing (InstanceContextMode = InstanceContextMode.Single).

The main issue is, that most developers think of Indigo WCF services in terms of simple request-response semantics and forget that such sessions get queued up on the service side if you do not terminate them appropriately.

This is the default service throttling behavior in the shipping version of WCF:

<behaviors>
  <serviceBehaviors>
    <behavior name="DefaultThrottlingBehavior">
      <serviceThrottling maxConcurrentCalls="16"
                         maxConcurrentSessions="10"
                         maxConcurrentInstances="<Int32.MaxValue>" />
    </behavior>
  </serviceBehaviors>
</behaviors>

Every service is throttled, even if you don't specify it. You can, of course, override all three throttling parameters.

Sessions can only be initiated by the client. They can be explicitly terminated only by the client. There are three ways a session can get terminated:

  • Explicitly by the client
  • Implicitly by a timeout
  • Implicitly by errors

Timeout can pass on a client or a service and, if the timeout happens, the transport channel that ensured communication gets faulted (Channel.State = CommunicationState.Faulted). Session is also terminated if an error is detected, thus invalidating the possibility to continue.

Remember that every service proxy you use will demand a new session on the service side (when using
sessionful services). If you spawn 11 threads and use a non-singleton proxy you will cause 10 sessions on the service side. The 11th will not get setup and will block the client thread until the timeout expires. In this case, the WCF infrastructure on the service side will issue a warning in the trace log, but nothing will be returned to the client. It seems as if the service would stop working. Nada.

There is a method on every service proxy and it's there for a reason. The method is called Close(). It closes the transport channel graciously and terminates the security session, thus freeing up service resources. The same happens for reliable messaging session or a combination of both.

Note: Another message (pair) is exchanged on Close(). This message is saying "We are done." to the service. Thus, one should be cautious when calling Close() for any exceptions, like CommunicationObjectFaultedException.

The best practice is to catch any CommunicationException exceptions when closing the channel.

To illustrate this, consider the following. You call a single method on a sessionful service. Then hang on to the service proxy instance for an hour. The inactivity timeout (attribute inactivityTimeout on a RM binding) is set to 10 minutes. So, ten minutes after the first call the channel gets faulted. Then you call Close(). This call will fail and throw an exception.

The following is the expected way of communication with sessionful services:

ServiceClient proxy = new ServiceClient("<Configuration>");
try
{
  // call service methods
  int intSum = proxy.Add(1, 2);

  // ... call all other methods ...

  
// call service methods
  int intSum = proxy.Subtract(1, 2);

  // close session
  proxy.Close();
}
catch (TimeoutException ex)
{
  // handle timeout exceptions
  proxy.Abort();
}
catch (FaultException<T1> ex)
{
  // handle typed exception T1
  proxy.Close();
}
...
catch (FaultException<Tn> ex)
{
  // handle typed exception Tn
  proxy.Close();
}
catch (FaultException ex)
{
  // handle all other typed exceptions
  proxy.Close();
}
catch (CommunicationException ex)
{
  // handle communication exceptions
  proxy.Abort();
}

Note that using the using statement is not recommended. The problem of using statement in this case is that CLR automatically calls Dispose() method on the using variable when exiting the statement. Since Close() can fail (remember, another message is exchanged), you can miss this important exception.

Everything in this post is true for all kinds of WCF sessions. It is not limited to security or RM sessions only.

Categories:  .NET 3.0 - WCF | Architecture
Tuesday, February 27, 2007 5:04:52 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF, MSMQ et al: Durable Reliable Messaging 

I'm thinking about reliable messaging these days (isn't it evident?). There are a couple of specific reasons for that. First, I can imagine a wonderful world of seamless message delivery. Second, currently there are technology limitations that prohibit what most of the industry would like to see.

Let's take a look.

There's a sentence in the first post of the series that says:

There is no durable reliable messaging support in WCF.

This is conditionally true. There are built-in bindings that support MSMQ integration and use MSMQ transport as a delivery mechanism, thus making the message exchange durable. Limitations of this approach include platform dependence, durable store dependence and more importantly, one-way messaging. To implement a durable duplex session one would need to implement two one-way service contracts. Downside? Forget about clients sitting behing a NAT without poking holes in your firewalls.

There are multiple uses for having durable messaging, the most prominent being different client/service lifetimes. Why can't I dispatch a message to a service when the service is down or I have no network connectivity?

Reliable messaging support in WCF is designed to support reliable transport and does not make any assurances of what happens to messages when they hit both ends of the wire. The problem is not in the WCF implementation but rather the WS-RM spec, which does not imply on how and when a message should/must be persisted.

There are different implementations of durable transports or at least, some people are already thinking about it. Amazon's SQS Transport for WCF is a good idea, and so is SQL Server SSB usage... if you control both ends of the wire.

The industry (read WS-* stack) needs a specification which will cover interoperable + durable message delivery. BYODS ("bring your own durable store") concept would be efficient only when demanding a durable store on the other side of the wire, is governed by a messaging specification.

Categories:  .NET 3.0 - WCF
Wednesday, February 21, 2007 11:25:32 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: About Exposing Metadata 

Everything about WCF is about keeping your service boundary intact. By default this also applies to exposing/publishing metadata. In ASMX days, one would need to opt-out of exposing metadata, while in WCF, one has to opt-in.

Let's say, for example, you have the following service config declaration:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <services>
      <service name = "Exposing.Metadata">
        <host>
          <baseAddresses>
            <add baseAddress =
                 "http://localhost:123/MetadataService
">
          </baseAddresses>
        </host>
        <endpoint address = ""
                  binding = "wsHttpBinding" 
                  contract = "Exposing.IServiceContract"/>
      </service>
    </services>
  </system.serviceModel>
</configuration>

This would host Exposing.Metadata service and define a single wsHttpBinding based endpoint listening at the base address.

So, the service endpoint address is http://localhost:123/MetadataService. If you would hit the endpoint URL with a web browser, a nice service page would be returned telling you that you have created a service, but there is no metadata exposed. So, hitting the endpoint with svcutil.exe would not allow you to grab metadata and generate the proxy code.

WCF, by default, does not expose any metadata. You have to ask for it nicely.

There are a couple of options to expose metadata of WCF services. The most basic way of doing it would be to expose it via HTTP based Get requests and retrieve WSDL. The following code fragment would do it, if you hosted the service using a ServiceHost class:

[ 1]  using (ServiceHost sh = new ServiceHost(typeof(OpService)))
[ 2]  {
[ 3]   ServiceMetadataBehavior smb = new ServiceMetadataBehavior();
[ 4]   smb.HttpGetEnabled = true;
[ 5]   sh.Description.Behaviors.Add(smb);
[ 6]   sh.Open();
[ 7]   Console.WriteLine("Service running...");
[ 8]
[ 9]   Console.ReadLine();
[10]   sh.Close();
[11]  }
 
Lines 3-5 instantiate and add a service behavior that allows a simple HTTP Get request to succeed for metadata retrieval. Also, the mentioned service page would now render a link to the WSDL definition of the service.

Declarative configuration with the same effect is this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name = "MetadataBehavior">
         <serviceMetadata httpGetEnabled = "true"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <services>
      <service name = "Exposing.Metadata"
               behaviorConfiguration = "MetadataBehavior">
        <host>
          <baseAddresses>
            <add baseAddress =
                 "
http://localhost:123/MetadataService"/>
          </baseAddresses>
        </host>
        <endpoint address = "" 
                  binding = "wsHttpBinding"
                  contract = "Exposing.IServiceContract"/>
      </service>
    </services>
  </system.serviceModel>
</configuration>

We declared a behavior, called MetadataBehavior and assigned it to our Exposing.Metadata service.

There are a couple attributes in <serviceMetadata> element worth discussing:

  1. externalMetadataLocation allows you to specify the path to your WSDL metadata file, so you can provide it out of band and not rely on the infrastructure to generate it for you.
  2. httpGetEnabled exposes this metadata via HTTP transport.
  3. httpGetUrl enables you to define a different relative WSDL metadata endpoint address. If 'wsdl' was used, our service would expose WSDL metadata at http://localhost:123/MetadataService/wsdl.
  4. httpsGetEnabled allows you to expose metadata via HTTPS transport.
  5. httpsGetUrl is analog to httpGetUrl but for HTTPS.

This nails down all options one would want for exposing HTTP based metadata endpoints.

Now, exposing metadata using the WS-MetadataExchange (WS-MEX or MEX) protocol is also opt-in. First, you have an option to include only HTTP based metadata, as described. Second, there is a way to expose all metadata only using WS-MEX.

Here's how, first the config file:

<serviceBehaviors>
  <behavior name = "MetadataBehavior">
    <serviceMetadata />
  </behavior>
</serviceBehaviors>

Note: This is only the <serviceBehaviors> section, not the complete service config file.

You have to tell the hosting infrastructure to expose metadata by adding a service behavior with the <serviceMetadata> element. What this does, essentially, is add an interface called IMetadataExchange to your contract - but does not expose it yet. You have to add a metadata endpoint to your service and bind it to a transport of choice.

Continuing with previous service configuration, here's what we get:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name = "MetadataBehavior">
          <serviceMetadata/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <services>
      <service name = "Exposing.Metadata"
               behaviorConfiguration = "MetadataBehavior">
        <host>
          <baseAddresses>
            <add baseAddress =
                 "
http://localhost:123/MetadataService"/>
          </baseAddresses>
        </host>
        <endpoint address = ""
                  binding = "wsHttpBinding"
                  contract = "Exposing.IServiceContract"/>
        <endpoint address = "mex"
                  binding = "mexHttpBinding"
                  contract = "IMetadataExchange"/>
      </service>
    </services>
  </system.serviceModel>
</configuration>

When this service is hosted, MEX metadata is exposed at http://localhost:123/MetadataService/mex and you can reach it with svcutil.exe or 'Add Service Reference' in Visual Studio. HTTP based WSDL access is not allowed since <serviceMetadata> element does not include the httpGetEnabled property with the value of true. It's false by default, which resonates nicely in terms of WCF's opt-in scenarios.

Remember, service orientation is about defining boundaries. And reaching metadata about your service should be confined to your choosing, not the platforms.

Categories:  .NET 3.0 - WCF
Monday, February 19, 2007 2:05:46 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Reliable Message Delivery Continued 

In the previous post we discussed possible scenarios and methods for configuring reliable message delivery in WCF. Let's look further into this great WCF feature.

Delivery assurances can have a large impact on the way your code processes incoming messages. Especially, ordering can be of importance if your code relies on ordered message delivery. Since configuration can be changed by any WCF service administrator there is a knob in WCF which lets you demand the appropriate binding. The knob is in a form of a declarative attribute, called DeliveryRequirementsAttribute.

Here's how it's used:

[DeliveryRequirements(RequireOrderedDelivery = true,
   QueuedDeliveryRequirements =
      QueuedDeliveryRequirementsMode.Required)]
interface IServiceContract
{
   int Operation(int a, int b);
}

DeliveryRequirementsAttribute can be set on any service interface or service class implementation. It has three properties:

  1. QueuedDeliveryRequirements which has three possible values, Allowed, NotAllowed and Required. Setting NotAllowed or Required makes the binding either demand or prevent WS-RM usage. Setting Allowed does not make any requirements.
  2. RequireOrderedDelivery demands a binding that supports and has ordered delivery turned on.
  3. TargetContract property is applicable only when the attibute is applied to a class definition. Since WCF service interfaces can be implemented by multiple classes, one can specify a specific interface for which the queued delivery requirements are defined.

The specified contract interface would demand a WS-RM capable binding and thus prevent service administrators to turn reliable delivery off.

In case where an administrator would turn reliable delivery (or ordering, in this case) off, the service host would throw an exception while trying to host the service.

So, DeliveryRequirementsAttribute guards developers from wrongdoings of service administrators. It should set QueuedDeliveryRequirements = QueuedDeliveryRequirementsMode.Required or RequireOrderedDelivery = true, when there are objective reasons to demand guaranteed and/or ordered delivery of messages.

Categories:  .NET 3.0 - WCF | Architecture
Monday, February 19, 2007 12:50:01 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Reliable Message Delivery 

Windows Communication Foundation includes concepts which enable the developer to insist on reliably delivering messages in both directions. The mechanism is actually transport agnostic and allows messages to be flown reliably from client to service and replies from service to client.

Underneath, WS-ReliableMessaging (WS-RM) implementation is used. Click here for Microsoft's specification. Content is the same.

WS-RM is based on a concept of a message sequence. You can think of it in terms of a session, although it does not carry HTTP session semantics (remember, it's transport agnostic). A communication initiator (RM source) sends a <CreateSequence> element inside the message body to establish a RM sequence. Service side (RM destination) responds with either <CreateSequenceReponse> or <CreateSequenceRefused>, in cases where new sequence is not welcome.

After the sequence is initialized, there is an additional SOAP Header in messages. It identifies the message number being transferred. The following is a simple example of headers of the first two messages:

<S:Envelope>
   <S:Header>
      ...
      <wsrm:Sequence>
         <wsrm:Identifier>http://webservices.gama-system.com/RM/Service</wsrm:Identifier>
         <wsrm:MessageNumber>1</wsrm:MessageNumber>
      </wsrm:Sequence>
   </S:Header>
   ...
   <S:Body>
      ...
   </S:Body>
</S:Envelope>

<S:Envelope>
   <S:Header>
      ...
      <wsrm:Sequence>
         <wsrm:Identifier>http://webservices.gama-system.com/RM/Service</wsrm:Identifier>
         <wsrm:MessageNumber>2</wsrm:MessageNumber>
      </wsrm:Sequence>
   </S:Header>
   ...
   <S:Body>
      ...
   </S:Body>
</S:Envelope>

After all messages have been exchanged and acknowledged, the RM destination sends a <SequenceAcknowledgement> inside the body of the message. RM source (communication initiator) then tears down the sequence with a <TerminateSequence> message.

So, how can this specification be implemented in various technology stacks? Well, WCF implements reliable messaging using the in-memory message buffers. There is no durable reliable messaging support in WCF.

Here's how it works:

  • Communication initiator (client) sends a <CreateSequence> message.
  • Service side responds with a <CreateSequenceResponse>.
  • Client sends 1..n messages and buffers them, while waiting for all acknowledgements.
  • Service side can and will dispatch the received messages as soon as possible. Now, there are two options:
    • Communication is configured for ordered delivery
      First message will be dispatched to the service model as soon as it arrives, noting that it has been processed. Every other message is processed in order. If message number 5 has been processed and the next received message carries sequence number 7, it will not be dispatched until message 6 is received and processed by the service model.
    • Communication allows out-of-order delivery
      First message will be dispatched to the service model as soon as it arrives, so will all the following messages. Since we did not demand ordered delivery the processing pipeline does not care on when to process the messages. It will process them as soon as they are received. They are acknowledged as soon as possible, but not before acknowledgement interval.
  • Service side sends a <SequenceAcknowledgement> message only when all messages have been acknowledged.
  • Initiator then stops the sequence with a <TerminateSequence> message.

So, how do we go about enabling WS-RM and realizable delivery in WCF? Simple. Here's the config file:

<wsHttpBinding>
   <binding configurationName="myReliableBinding">
      <reliableSession enabled="true" ordered="true" />
   </binding>
</wsHttpBinding>

The <reliableSession> element has two attributes. The first one, enabled, enables reliable message delivery. The second one, ordered, enables in-order processing by the service model.

One has to acknowledge that the following is true for different WCF bindings:

basicHttpBinding - RM not supported
wsHttpBinding - RM supported, not default
wsDualHttpBinding - RM implicit
netTcpBinding - RM supported, not default

There are a couple of options available for anyone using the custom binding, in regard to reliable messaging behavior:

  1. acknowledgementInterval specifies the time lapse duration of message acknowledgement. The default is two seconds, while always acknowledging the first message. This gives you an efficient way to group the acknowledgement messages, thus conserving network traffic.
  2. flowControlEnabled enables message sender to acknowledge the receive buffer on the recipient side. It turns out that every acknowledgement message includes a service buffer status and if this attribute is set, sender will not send messages if the recipient buffer is full, thus not occupying network resources.
  3. inactivityTimeout defines the absolute duration of a no-message (infrastructure and application messages) session. After the timeout, the session is torn down.
  4. maxPendingChannels defines how many reliable channels the service can put in its waiting-to-open queue. If there's a significant load, the pending requests can queue up and the service refuses to open a reliable session (think <CreateSequenceRefused>).
  5. maxRetryCount defaults to 8. It defines how many times the infrastructure retries with message delivery. If the limit is achieved, the channel will become faulted.
  6. maxTransferWindowSize defines how many messages should be buffered. On service side all messages that have not been acknowledged increase the count. On the client side, all messages which have not received their acknowledgement pair are counted.
  7. ordered defines in-order delivery. It's values are true and false.

There is some confusion in the wsHttpBinding's <reliableSession> config element. There's an enabled property in the Visual Studio config schema, which should not have any influence on reliable session establishment (and does not have a matching object model method/property). It does, however. There is a difference if you setup a reliable session by using a customBinding or wsHttpBinding.

I.e., here's the custom binding config:

<customHttpBinding>
   <binding configurationName="customReliableBinding">
      <reliableSession ordered="true" />
   </binding>
</customBinding>

This will enable reliable and ordered session support in any custom binding.

So, in general - WCF implementation of WS-ReliableMessaging gives you automatic message retries, duplicate detection, and ordering. It should be turned on for any multi-hop message paths, but can be very valuable even in high latency/packet loss network scenarios.

Update 2007-02-19

William Tay, a friend and distributed systems expert, replies with a terrific message exchange pattern for groking WS-RM specification. Don't walk, run.

Categories:  .NET 3.0 - WCF | Architecture
Sunday, February 18, 2007 9:24:28 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 .NET 3.0 Middleware Technologies Day: Third Incarnation 

Third incarnation of the .NET 3.0 Middleware Technologies day went through yesterday.

Here are the deliverables:

If you did not get/notice the printed articles, here's what should put you to sleep over the weekend:

Also, the accompanying book is available here.

[1] Only available in Slovene.

Thursday, February 15, 2007 6:09:56 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 WCF: Serious Seriousness 

This must be one of the best Channel 9 interviews of all time.

Rory did make quite an effort to handle the guys, but at building 42, what can you expect.

It all seems as the Brady Bunch of Microsoft on tape, but they sure did ship the most important Microsoft technology stack of the decade.

Well done on both parts.

Categories:  .NET 3.0 - WCF
Tuesday, November 14, 2006 12:03:26 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 MSDN Event Presentation: Code and PPT 

I just finished presenting my talk on the October 2006 Slovenian MSDN event.

Here are the deliverables:

Code: Download
PPT: Download [Slovenian]

Solution file includes:

  1. WCF service using wsDualHttpBinding for duplex session communication scenario. WCF service hosts a workflow which computes stock tax based on the current (complex) Slovenian legislation.
  2. WF workflow named ComputeTaxWorkflow. Workflow calls out to our public ASP .NET WebServices: StockQuotes and ExchangeRates.
  3. WCF Client, which sends requests and receives responses when the workflow is done

Check it out. It's free.

Monday, October 23, 2006 11:04:56 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 .NET 3.0 Middleware Technologies Day 

We had a nice discussion throughout the day today, together with around 50 attendees. It's hard to even cover feature changes of .NET 3.0 in a day, but we managed to cover the important things of WCF and WF, spending a complete day inside the Visual Studio.

Here are the demos: Download
And the PPT slides: Download [Slovenian]

And the link to the Tom Archer's compatibility matrix for the .NET Framework 3.0 downloads.

Thanks to everyone who attended.

Categories:  .NET 3.0 - WCF | .NET 3.0 - WF | Architecture | Work
Thursday, October 05, 2006 6:03:06 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Article: Cooperation Between Workflows and Services 

Last article discusses service-workflow cooperation options in WinFX and dives into communication scenarios for Windows Workflow Foundation.

Language: Slovenian


Naslov:

Sodelovanje storitev in delovnih tokov

Sodelovanje storitev in delovnih tokov

Categories:  Articles | .NET 3.0 - WCF | .NET 3.0 - WF
Wednesday, June 07, 2006 10:29:02 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Article: Transactional Semantics in Loosely Coupled Distributed Systems 

Last article finished my XML series. This one focuses on transactional semantics in a service oriented universe.

Language: Slovenian


Naslov:

Transakcijska semantika v šibko sklopljenih, porazdeljenih sistemih

Transakcijska semantika v šibko sklopljenih, porazdeljenih sistemih

Tuesday, June 06, 2006 9:59:30 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Article: Concepts and Semantics of Service Contracts 

The second article is about concepts and semantics of service contracts. It deals with WCF (Windows Communication Foundation) contract definition and its behavioral aspects.

Language: Slovenian


Naslov:

Koncepti in semantike storitvenih pogodb

Koncepti in semantike storitvenih pogodb

Categories:  .NET 3.0 - WCF | Architecture | Articles | Web Services
Friday, June 02, 2006 7:19:45 PM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Windows Workflow Foundation: Calling WCF Services 

Having the ability to call services from inside your workflows is a good thing. Things get a little obfuscated when one wants to expose a WCF based service to Windows Workflow Foundation (WF).

What it comes down to is that currently (beta 2 build of WF) we do not have any built-in workflow activities which would allow you to communicate with the WCF advanced services (meaning WS-Security enabled, TCP based, WS-RM enabled, WS-Tx enabled, ...).

Assuming we have an OrderService exposed using the following config:

<configuration>
  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="Default">
          <security mode="None"/>
        </binding>
        <binding name="Secure">
          <security mode="Message">
            <message clientCredentialType="Windows"/>
          </security>
        </binding>
        <binding name="Reliable">
          <reliableSession enabled="true" ordered="true"/>
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service name="WCFCalledByWorkflow.OrderService" >
        <endpoint address=""
                  binding="wsHttpBinding"
                  behaviorConfiguration="Default"
                  contract="WCFCalledByWorkflow.IOrderService" />
        <endpoint address="/secure"
                  binding="wsHttpBinding"
                  behaviorConfiguration="Secure"
                  contract="WCFCalledByWorkflow.IOrderService" />
        <endpoint address="/reliable"
                  binding="wsHttpBinding"
                  behaviorConfiguration="Reliable"
                  contract="WCFCalledByWorkflow.IOrderService" />
      </service>
    </services>
  </system.serviceModel>
</configuration>

This, combined with the following hosting app:

using (ServiceHost sh = new ServiceHost(typeof(OrderService),
  
new Uri("http://localhost:666/OrderService")))
{
   sh.Open();
   Console.WriteLine("Order service running..\n");
   Console.WriteLine("Listening on:");
   foreach(ServiceEndpoint se in sh.Description.Endpoints)
   {
      Console.WriteLine(se.Address.ToString());
   }
   Console.WriteLine("\nPress [Enter] to stop the service.");
   Console.ReadLine();
   sh.Close();
}

Would produce the following output when run:

Order service running..

Listening on:
http://localhost:666/OrderService
http://localhost:666/OrderService/secure
http://localhost:666/OrderService/reliable

Press [Enter] to stop the service.

Now, this is not enough. Our service exposes three different endpoints, each one has different message requirements. And what is more important, all three are based on SOAP 1.2 + WS-Addressing.

There is a 'Default' endpoint which is plain vanilla SOAP 1.2 endpoint, without any security (have to mention it: WCF services are secure by default, one has to turn off security to achive this). Second endpoint uses Windows based message security and third turns on WS-RM. Security wise, second and third endpoints are the same (remember, defaults?).

As said, WF is currently not able to communicate with any of the above endpoints. What needs to be done is adding another endpoint to your service, which would expose it in ASMX compatible way.

The new config is this:

<configuration>
  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="Default">
          <security mode="None"/>
        </binding>
        <binding name="Secure">
          <security mode="Message">
            <message clientCredentialType="Windows"/>
          </security>
        </binding>
        <binding name="Reliable">
          <reliableSession enabled="true" ordered="true"/>
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service name="WCFCalledByWorkflow.OrderService" >
        <endpoint address=""
                  binding="wsHttpBinding"
                  behaviorConfiguration="Default"
                  contract="WCFCalledByWorkflow.IOrderService" />
        <endpoint address="/secure"
                  binding="wsHttpBinding"
                  behaviorConfiguration="Secure"
                  contract="WCFCalledByWorkflow.IOrderService" />
        <endpoint address="/reliable"
                  binding="wsHttpBinding"
                  behaviorConfiguration="Reliable"
                  contract="WCFCalledByWorkflow.IOrderService" />
        <endpoint address="/legacy"
                  binding="basicHttpBinding"
                  contract="WCFCalledByWorkflow.IOrderService"/>
      </service>
    </services>
  </system.serviceModel>
</configuration>

So our Windows Workflow Foundation compatible endpoint is <base address>/legacy.

What this means is that you can bind all your InvokeWebService activities inside workflows to published WCF services by just adding another ASMX compatible endpoint to the WCF service.

The difference in two WCF configs is here:

<endpoint address="/legacy"
   binding="basicHttpBinding"
   contract="WCFCalledByWorkflow.IOrderService"
/>

Code sample can be downloaded here. It includes two projects, a WF and a WCF project. ServiceHost is implemented.

Categories:  .NET 3.0 - WCF | .NET 3.0 - WF
Wednesday, May 31, 2006 11:54:58 AM (Central Europe Standard Time, UTC+01:00)  #    Comments

 

 Windows Communication Foundation: negotiateServiceCredential attribute 

In WCF there's a knob which you can turn to configure service credential propagation semantics.

It's called negotiateServiceCredential and is present in bindings/<bindingOfChoice>/binding/security/message/@negotiateServiceCredential in the WCF configuration