The other day I posed a question regarding the processing of data from a web service where the structure (i.e., the names and quantities of the fields) varies.  The fictitious example had us suppose that we wrote a component designed to plug into and consume the services of an application that may be installed across a wide customer base where each customer might customize (albeit indirectly) the results of the web service.  The web service, of course, would purely be an abstraction of the customer's customizations and business model.  For instance, a customer may decide to add a special/custom property to an 'Account' within the system that helps him track some aspect of the Account that isn't out-of-the-box.  This may, as a byproduct, affect the structure of the web service such that the new field is now present.

Part of our solution is to be accepting of such a change in structure in a manner that crosses any and all customer needs.  One customer may customize the heck out of an entity (such as Account) while another may make just a few changes.  Others will leave it alone entirely.  Our product must be able to operate on and affect any and all fields (including custom ones).

We discussed that there are some considerations to make in order to support this environment.

The question posed, then, was "How would you approach this problem?"  How would you design a system whereby you can consume all of the attributes of the entity (e.g., Account) across the board?

From within Visual Studio you can "Add Web Reference".  From the command-line (and my preferred mechanism) you use WSDL.exe.  Regardless of route you take (heck, you can even do it manually - I've done that in the past too), you'll end up with a proxy class with method calls to the web service and classes representing the various structured types that the web service exposes.

This isn't enough when you want to support a structure that will vary from implementation to implementation; the service won't have any inkling of any of the custom fields.

Alright, enough setup.  How would you solve this potential issue?

There are a couple of approaches worth investigating, each with their own varying degrees of complexity and control.  And probably many more that are eluding me and are simpler still.

First of all you can rewrite all of the traffic to/from the web service via a SoapExtension.  This approach may ultimately yield the greatest flexibility, but at the expense of a lot of work.  Essentially, with a SoapExtension you can inject your code into the web service pipeline and read, alter, or even rewrite completely all of the traffic to and from the web service before it is ultimately returned to you (or delegated on to another SoapExtension in the chain).  In fact, when I was first experimenting with this issue of dynamically structured data this was my first approach.  I have the history documented here if you're interested in reading more about it.

A second approach takes a different angle on this idea.  It involves defining a "bucket" into which all fields not formally defined within your proxy class are grouped.  To achieve this we must alter the generated proxy class by adding a custom field (note, don't re-gen the proxy class once you do this or you'll lose your changes).

Recall the original example:

[XmlType(Namespace="http://schemas.devstone.com/svc")]
public abstract class customer : BusinessObj {
   public string id;
   public string name;
}

Let's add our "bucket" and we'll call it "custom":

[XmlType(Namespace="http://schemas.devstone.com/svc")]
public abstract class customer : BusinessObj {
   public string id;
   public string name;
   [XmlAnyElement()] public XmlElement[] custom;
}

This very small change is what works the magic for us.  The XmlAnyElementAttribute identifies that the "custom" member will represent any XML elements within the results from the web service that don't have a corresponding member in the object.  That's pretty cool.

Ok, this is only part of the problem.  This gives us the ability reference potentially anything that the web service may throw our way, but how do we make use of this in our application?

I like to define a wrapper class for the object that encapsulates the business object:

public sealed class DynamicObj {
   public DynamicObj(BusinessObj busObj) {
      _busObj = busObj;
      _busType = busObj.GetType();  // cache the type for repetitive use
   }

   BusinessObj _busObj;
   Type _busType;

   // expose the business object directly so we can continue to interact with the web service with
   // the actual object (not the wrapped one)
   public BusinessObj Object {
      get { return _busObj; }
   }

   // provide a indexer mechanism to read/write field values
   public object this[string fieldName] {
      get { return getFieldValue(fieldName); }
      set { /* I'll leave this as an exercise for you, the reader - it's not difficult at all */ }
   }

   // returns the value associated with the specified field name
   public object getFieldValue(string fieldName) {
      FieldInfo field = _busType.GetField(fieldName, BindingFlags.Public | BindingFlags.Instance);
      if ( null != field )
         return field.GetValue(_busObj);
      else {
         XmlElement customField = getCustomField(fieldName);
         return ( null != customField ? customField.InnerText : null );
      }
   }

   // returns simply the number of fields defined by the business object
   public int FieldCount {
      get {
         FieldInfo[] fields = getDefinedFields();
         XmlElement[] elems = getCustomFields();
         return fields.Length + elems.Length - 1;  // subtract 1 to account for the presence of the 'custom' field
      }
   }

   // returns an array consisting of all of the fields within the business object
   // NOTE: depending on usage, we might want to cache the result of this method so
   // we're not constantly regenerating it.
   public string[] GetFieldNames() {
      string[] ret = new string[FieldCount];
      int i = 0;
      foreach ( FieldInfo field in getDefinedFields() )
         if ( "custom" != field.Name ) ret[i++] = field.Name;
      foreach ( XmlElement elem in getCustomFields() )
         ret[i++] = elem.Name;
      return ret;
   }

   // returns an array of all fields formally defined on the business object
   private FieldInfo[] getDefinedFields() {
      return _busType.GetFields(BindingFlags.Public | BindingFlags.Instance);
   }

   // returns a list of all fields found within the 'custom' field.
   private XmlElement[] getCustomFields() {
      FieldInfo customField = _busType.GetField("custom", BindingFlags.Public | BindingFlags.Instance);
      if ( null == customField )
         throw new NotImplementedException("The BusinessObj does not support the 'custom' field.");
      else
         return customField.GetValue(_busObj) as XmlElement[];
   }

   private XmlElement getCustomField(string fieldName) {
      foreach ( XmlElement elem in getCustomFields() ) {
         if ( fieldName == elem.Name )
            return elem;
      }
      return null;
   }
}

As you can see, it's pretty simple and removes a lot of the complexity of dealing with an object of whose structure you're not aware.  There is a whole lot more you can do with this object than I'm letting on, but it may be a starting point.  There are a few issues that this approach does not address (such as the notion of 'type').  For example, there's no way of knowing whether you should be dealing with an int, a string, a DateTime, etc - but I'll leave that implementation up to you.

What do you think? Plausable? Idiotic? Cool?

I'd like your feedback and input.