<grin>I've had a lot of free time lately.  So much, in fact, that I don't know what to do with myself.</grin>

With this copious free time I've dabbled a bit in Subversion.  I'd experimented with it before, but not on a grand scale.  In fact, I've used SourceSafe (which in my experience isn't as bad as people would have you believe), SourceGear's Vault (which I use presently and absolutely love), and I've experimented a bit with CVS, Perforce, et al.  I've been very impressed with Subversion and thought I'd give it a whirl for all of my personal software development.  If nothing else, I get a good, rock-solid SCM system and a more well-rounded understanding of this other system.

Thus, with that in mind, I set out to establish a new source code control system.

First of all, I downloaded the current version (version 1.4.3) and installed it on my server.  This gave me the %ProgramFiles%\Subversion folder and added the %ProgramFiles%\Subversion\bin folder to my path.

Next, I don't ever have to want to logon to that server interactively just to start the application 'listening'.  So, I set it up as a service with the following command line:

sc.exe create svnserve binpath= "C:\Program Files\Subversion\bin\svnserve.exe --service --root C:\SVN" displayname= "SubVersion" depend= tcpip start= auto obj= "NT AUTHORITY\NetworkService"

(Note that there is no space before the equal sign for each parameter but there is one afterwards.  Personally, I think this is a silly construct for the sc.exe command, but maybe it made the command line easier to deal with.  Oh well, I didn't write it)

That statement will register the svnserve.exe application as an NT Service so in the event that I have to restart the server the service will automatically start.

Next, I created the root folder for my repositories (which for purposes of this example is C:\SVN) and granted the NT AUTHORITY\NetworkService account the appropriate rights to it.

The very first thing that I didn't really care for with respect to Subversion was the way it does authentication.  I wanted to have a single source for all authentication and not maintain it for each repository.  While not a show stopper, I don't like the idea of having to change n passwd files (one for each repository) in the event my password changes.  Now that I mention it, I don't like the idea of having to put it in a file at all (especially in clear text!) - there are oh so many security issues with this.  I have seen some articles on how to set it up to use Windows Authentication (which would be ideal in my situation), but I'm not that far along with it yet.

Nonetheless, I set up a simple folder named “C:\SVN\(auth)” to hold the authz and passwd files (I like to use parens on folder names like this so they filter to the top of my directory lists and stand out).

I then edited the files accordingly:

authz

[/]
Aaron = rw

passwd

[users]
Aaron = xxxxxx

As is traditional with SCM systems (and in keeping with that tradition), each repository has three folders: \trunk, \tags, and \branches.  However, when you create a repository those don't get created automatically for you.  Rather than always remembering to create these for each repository, I created a template folder called “C:\SVN\(template_repo)” which contains the structure and helper files for new repositories.  In addition to the folders there's a simple svnserve.conf configuration file that is designed to replace the automatically created svnserve.conf file in each new repository and point back to the (auth) folder.  That way, each new repository uses the same catalog of credentials.  Then I have a batch file that simplifies the creation of a repository and automatically imports the folder structure, replaces the files, and deletes extraneous ones.  I put this batch file in the root C:\SVN folder:

CreateRepo.bat

@echo off
if "%1" == "" goto USAGE

echo Creating SVN repository "%1".
svnadmin create "%1"

echo Establishing default credentials.
copy "(template_repo)\svnserve.conf" "%1\conf\svnserve.conf" > NUL
del "%1\conf\authz." > NUL
del "%1\conf\passwd." > NUL

echo Building folder structure.
svn import "(template_repo)\folders" svn://%COMPUTERNAME%/%1 --message "Initialize folder structure for repository." --quiet

echo Done.  "%1" repository created.
goto DONE

:USAGE
echo.
echo Invalid usage.  Please specify a repository name.

:DONE

I guess the last thing I needed to do to set it all up was to configure my backups.  There's not much point in having an SCM system without a good backup system in place.  Well, everything done to this point was done with a purpose.  I intentionally put everything in the C:\SVN folder to have a single point of reference for all repositories and file, and to simplify backups.  I can simply walk the directories therein and back them up.  If I add a new repository it's automatically included in my backups and there's nothing more to set up.

In the Subversion documentation they warn against simply 'copying' the repository tree or your backup might be completely invalid and useless.  Instead, you're recommended to run svnadmin.exe with the hotcopy argument.

I wrote a very simple VBScript (appropriately named BackupSVN.vbs) to take care of my backups.  I put this file one level out of the repository root (e.g., C:\ for this example).

Essentially the script file enumerates all of the directories, determining whether they are Subversion repositories or not.  For those that are, it runs svnadmin hotcopy.  For those that aren't, it simply runs xcopy.  All backups are then made to the \SVNBackup\SVN folder.  The script then executes WinRar to archive the tree into a date-stamped .rar file which is placed in the \SVNBackup folder.

Though it doesn't do it yet, I'll soon have this archiving the .rar files to a remote store (e.g., a network share) because a backup on the same physical drive isn't much good if the drive crashes.

BackupSVN.vbs

' ************************************************************************************

Dim fso, shell, f, root, fileName
Set fso = CreateObject("Scripting.FileSystemObject")
Set shell = CreateObject("WScript.Shell")
Set root = fso.GetFolder(".\SVN")

'
' copy each folder to the svn backup directory
'

For Each f in root.SubFolders
   if ( IsSVNFolder(f) ) then
      cmd = "svnadmin hotcopy ""SVN\" + f.Name + """ ""SVNBackup\SVN\" + f.Name + """"
   else
      cmd = "xcopy ""SVN\" + f.Name + "\*.*"" ""SVNBackup\SVN\" + f.Name + """ /I /E /Y /H"
   end if
   Call Exec(cmd)
next

'
' copy each file in the \SVN directory to the backup directory and archive it
'

fileName = """SVNBackup\SVNBackup (" + Replace(FormatDateTime(Date, 2), "/", "-") + ").rar"""
Call Exec("xcopy SVN\*.* SVNBackup\SVN /Y")
Call Exec("""%ProgramFiles%\WinRar\rar.exe"" a " + fileName + " SVNBackup\SVN\*.* /ep1 /r /df")
fso.DeleteFolder(".\SVNBackup\SVN")

' ************************************************************************************

Function Exec(cmd)
   Dim result
   Set result = shell.Exec(cmd)
   Do While ( result.Status = 0 )
      WScript.Sleep 100
   Loop
End Function

Function IsSVNFolder(folder)
   IsSVNFolder = HasFolder(f, "conf")
End Function

Function HasFolder(folder, name)
   Dim f
   for each f in folder.SubFolders
      if ( f.Name = name ) then
         hasFolder = true
         exit function
      end if
   next
   hasFolder = false
End Function

I simply set this VBScript to run nightly and I'm off to the races.

Today was not a fun day.  Among the many tasks that I had to accomplish my primary development virtual machine decided to barf on me.  The system would boot but then the moment I attempted to run VS and load a project it would 'blink' off and reboot.  It did this twice and then decided to run a CHKDSK at the third boot.  This found several invalid file chains and ended up 'restoring' several files.

At this point, the OS wouldn't even boot, saying that the C:\WINDOWS\SYSTEM32\CONFIG directory was missing.  Great!  My next step was to run a repair from the Windows XP boot disk.  This ran fine (for a while) until it eventually crashed with an invalid memory address read.

The rest of my day was spent attempting to get my development files off of the VPC so I could move over (a.k.a., rebuild) another.  That, in and of itself, isn't a big deal - I have several base images to work with so I'm not worried.  Getting the files off of a VPC image turned out to be non-trivial.  There may be better ways to do this (and I'm open to thoughts), but here's what I did:

1. Create a secondary, blank VHD (virtual hard drive).  I had to do this because VPC wouldn't let me mount a non-CD/DVD/ISO drive within the image and I couldn't share a folder between the VM and the host machine.
2. Boot the VPC to the command prompt (I used an old Win98 disk cause it was laying around).
3. FDISK to create a partition on the VHD.
4. Boot to the XP Recovery Console (via the XP boot disk) - a step which we'll soon see was a waste of time.
5. FORMAT D:\ /FS:NTFS
6. At this point, I wanted to copy my ~\dev directory to the new VHD.  Well, as it turns out the XP Recovery Console doesn't allow XCOPY (even when referenced directly in the c:\windows\system32 folder) !! Even then, COPY doesn't allow wildcards nor directory copying !!  This completely, utterly baffles me.
7. I then remembered my best friend in boot-time travails: Windows PE (Pre-installation Environment) Server Edition: boot to WinPE.
8. Once I booted up to PE it was a cakewalk.  It's a good thing too, because manually copying 2490 files manually (one by one due to lack of wildcards in NTRC) would have been a nightmare.

Well, now I'm back up and running, but what a nightmare.

The other day I posed a question regarding the processing of data from a web service where the structure (i.e., the names and quantities of the fields) varies.  The fictitious example had us suppose that we wrote a component designed to plug into and consume the services of an application that may be installed across a wide customer base where each customer might customize (albeit indirectly) the results of the web service.  The web service, of course, would purely be an abstraction of the customer's customizations and business model.  For instance, a customer may decide to add a special/custom property to an 'Account' within the system that helps him track some aspect of the Account that isn't out-of-the-box.  This may, as a byproduct, affect the structure of the web service such that the new field is now present.

Part of our solution is to be accepting of such a change in structure in a manner that crosses any and all customer needs.  One customer may customize the heck out of an entity (such as Account) while another may make just a few changes.  Others will leave it alone entirely.  Our product must be able to operate on and affect any and all fields (including custom ones).

We discussed that there are some considerations to make in order to support this environment.

The question posed, then, was "How would you approach this problem?"  How would you design a system whereby you can consume all of the attributes of the entity (e.g., Account) across the board?

From within Visual Studio you can "Add Web Reference".  From the command-line (and my preferred mechanism) you use WSDL.exe.  Regardless of route you take (heck, you can even do it manually - I've done that in the past too), you'll end up with a proxy class with method calls to the web service and classes representing the various structured types that the web service exposes.

This isn't enough when you want to support a structure that will vary from implementation to implementation; the service won't have any inkling of any of the custom fields.

Alright, enough setup.  How would you solve this potential issue?

There are a couple of approaches worth investigating, each with their own varying degrees of complexity and control.  And probably many more that are eluding me and are simpler still.

First of all you can rewrite all of the traffic to/from the web service via a SoapExtension.  This approach may ultimately yield the greatest flexibility, but at the expense of a lot of work.  Essentially, with a SoapExtension you can inject your code into the web service pipeline and read, alter, or even rewrite completely all of the traffic to and from the web service before it is ultimately returned to you (or delegated on to another SoapExtension in the chain).  In fact, when I was first experimenting with this issue of dynamically structured data this was my first approach.  I have the history documented here if you're interested in reading more about it.

A second approach takes a different angle on this idea.  It involves defining a "bucket" into which all fields not formally defined within your proxy class are grouped.  To achieve this we must alter the generated proxy class by adding a custom field (note, don't re-gen the proxy class once you do this or you'll lose your changes).

Recall the original example:

[XmlType(Namespace="http://schemas.devstone.com/svc")]
public abstract class customer : BusinessObj {
   public string id;
   public string name;
}

Let's add our "bucket" and we'll call it "custom":

[XmlType(Namespace="http://schemas.devstone.com/svc")]
public abstract class customer : BusinessObj {
   public string id;
   public string name;
   [XmlAnyElement()] public XmlElement[] custom;
}

This very small change is what works the magic for us.  The XmlAnyElementAttribute identifies that the "custom" member will represent any XML elements within the results from the web service that don't have a corresponding member in the object.  That's pretty cool.

Ok, this is only part of the problem.  This gives us the ability reference potentially anything that the web service may throw our way, but how do we make use of this in our application?

I like to define a wrapper class for the object that encapsulates the business object:

public sealed class DynamicObj {
   public DynamicObj(BusinessObj busObj) {
      _busObj = busObj;
      _busType = busObj.GetType();  // cache the type for repetitive use
   }

   BusinessObj _busObj;
   Type _busType;

   // expose the business object directly so we can continue to interact with the web service with
   // the actual object (not the wrapped one)
   public BusinessObj Object {
      get { return _busObj; }
   }

   // provide a indexer mechanism to read/write field values
   public object this[string fieldName] {
      get { return getFieldValue(fieldName); }
      set { /* I'll leave this as an exercise for you, the reader - it's not difficult at all */ }
   }

   // returns the value associated with the specified field name
   public object getFieldValue(string fieldName) {
      FieldInfo field = _busType.GetField(fieldName, BindingFlags.Public | BindingFlags.Instance);
      if ( null != field )
         return field.GetValue(_busObj);
      else {
         XmlElement customField = getCustomField(fieldName);
         return ( null != customField ? customField.InnerText : null );
      }
   }

   // returns simply the number of fields defined by the business object
   public int FieldCount {
      get {
         FieldInfo[] fields = getDefinedFields();
         XmlElement[] elems = getCustomFields();
         return fields.Length + elems.Length - 1;  // subtract 1 to account for the presence of the 'custom' field
      }
   }

   // returns an array consisting of all of the fields within the business object
   // NOTE: depending on usage, we might want to cache the result of this method so
   // we're not constantly regenerating it.
   public string[] GetFieldNames() {
      string[] ret = new string[FieldCount];
      int i = 0;
      foreach ( FieldInfo field in getDefinedFields() )
         if ( "custom" != field.Name ) ret[i++] = field.Name;
      foreach ( XmlElement elem in getCustomFields() )
         ret[i++] = elem.Name;
      return ret;
   }

   // returns an array of all fields formally defined on the business object
   private FieldInfo[] getDefinedFields() {
      return _busType.GetFields(BindingFlags.Public | BindingFlags.Instance);
   }

   // returns a list of all fields found within the 'custom' field.
   private XmlElement[] getCustomFields() {
      FieldInfo customField = _busType.GetField("custom", BindingFlags.Public | BindingFlags.Instance);
      if ( null == customField )
         throw new NotImplementedException("The BusinessObj does not support the 'custom' field.");
      else
         return customField.GetValue(_busObj) as XmlElement[];
   }

   private XmlElement getCustomField(string fieldName) {
      foreach ( XmlElement elem in getCustomFields() ) {
         if ( fieldName == elem.Name )
            return elem;
      }
      return null;
   }
}

As you can see, it's pretty simple and removes a lot of the complexity of dealing with an object of whose structure you're not aware.  There is a whole lot more you can do with this object than I'm letting on, but it may be a starting point.  There are a few issues that this approach does not address (such as the notion of 'type').  For example, there's no way of knowing whether you should be dealing with an int, a string, a DateTime, etc - but I'll leave that implementation up to you.

What do you think? Plausable? Idiotic? Cool?

I'd like your feedback and input.

With .NET, consuming web services is a snap.  Heck, it's not all that difficult without .NET, but it sure makes it even easier.  In general, and for simple cases, all you need to do is add a web reference to the target .asmx, have the proxy generated (via VS.NET or wsdl.exe), and you're off to the races.

Some additional care and consideration should be taken, however, when the data returned from the web service can vary in format and structure.  Allow me illustrate this with an example:

Suppose you are developing an application that must consume some data from a web service.  This web service returns structured data that, via the auto-generated proxy, is represented in a class.  For argument sake, this class is Customer (which derives from BusinessObj) and it has a few properties as identified below:

[XmlInclude(typeof(customer))]
public abstract class BusinessObj { }

[XmlType(Namespace="http://schemas.devstone.com/svc")]
public abstract class customer : BusinessObj {
   public string id;
   public string name;
}

Furthermore, this web service is part of a larger application that a company will purchase and deploy internally.  In fact the customer may customize each type of data (e.g., Customer) by adding custom fields and attributes.  Your component is designed as a 3rd-party add-in for this application and may be deployed across this varied environment.

Some customers will leave the base application alone (not customizing it) where other will 'go-to-town' and change the schema entirely to fit their business needs.

The question is this: how can your application operate in such a varied environment, consume the web service, and process all of the customizations that the customer has implemented?

You cannot, for instance simply create your web service proxy against the default schema and expect that to work.  The reason being that once you deploy your component into a customized environment, your proxy will not know about the custom fields and cannot serialize/deserialize them.  You won't get an error, per sé, but any custom fields will be ignored and not made available to your application.

This could be disastrous in the event that you needed to populate or read the custom fields.

How would you approach this problem?

NOTE: I'll post my answer tomorrow (or perhaps tonight) of how I've implemented it in the past to great success.

Phew! What a crazy, fun weekend!  I'm finally getting caught up.

This past weekend (April 14th, 2007) we had the opportunity to host and participate in the second Code Camp in Utah officially entitled “Utah Spring Code Camp 2007”.  Our goal is to put these on every six months, the next one sometime between Sept and Nov of this year.

Nonetheless, this event was a lot of fun.  There were many in attendance, though not quite as many as we were hoping for.

I had the opportunity speak on .NET Serialization.  It was a variation on a presentation that I had prepared a few months back for the Utah .NET User Group and this time, after rewriting the presentation and code samples from scratch it went MUCH better.  I am quite happy with how it came out (I hope those in attendance gleaned some nuggets of information).

As it turns out one presenter was not able to make it out to the Code Camp (calling in around 10:30 or 11:00 in the morning) so we had to scramble a bit.  I had some ideas up my sleeve based on what I had trimmed from my presentation and we focused the second hour on XML Serialization.  That was a lot of fun!

The third block was a shortened (30 min) presentation in which I discussed an evolution in accessing application configuration files (e.g., web.config, app.exe.config, et al).  Beginning with the most basic access method via AppSettings, we progressively made the transition to ConfigurationSectionHandlers and XML Serialization.  We used this technique to deserialize a block of XML from the configuration file into an object in code that we can access.

All in all, I'm quite happy with the Code Camp, though I wish I could have attended more of the presentations.  I guess that's a drawback from having multiple tracks running simultaneously - you have to pick and choose what you're gonna see.

The code for my presentations can be downloaded here.  Enjoy!

Today, April 10th, 2007, marks the three year anniversary of my blog.  What an amazing three years!  This marks the 504th post on this here blog.

Looking back over the past year I'm pretty happy with the many things that have been accomplished and the progress that I've made, both as a developer but also as a person in general.  Among other things, I've strived (more of late than usual) to not focus solely on computers, as it were, and more on life in general.  You could say I've ramped up other aspects of my life because I recognize that there's more to life than just sitting behind this monitor and keyboard.

For instance, I've taken up construction and woodworking (long time loves of mine to which I've not dedicated as much time as I would have liked before).  As soon as I clean out my garage I can really go to town.

Also, looking back on my blog, I've made some changes periodically and sporadically.  I've recently added Google Ads and a Google Search.  I hope those aren't visual obtrusive and annoying.  It's been interesting to see how many ad clicks are made and how many searches are performed daily on the blog.

You may have also noticed my download and rating controls.  I get notified each time someone downloads a file and/or rates a blog post or comment.  That's been a lot of fun to see at least one download a day by someone (most of the time I don't know who they are).  I've received but a very small handful of emails regarding issues with the downloads.  Have they been working for everyone?  I would assume so.  I've found it quite interesting to see that the Image Rotator Control is, by far, the most popular download.  I would not have expected that.

Well, enough rambling.

Here's to another fruitful, fun-filled year!  Enjoy!

There's gonna be a fun MSDN event this coming Thursday, April 12th (same day as our Utah .NET User Group meeting - reminders going out shortly) from 1:00 PM - 5:00 PM at Jordan Landing in West Jordan on ASP.NET AJAX and CardSpace.  I'll be there - it should be a great time.

Here's the event link with all the details:  http://msevents.microsoft.com/cui/EventDetail.aspx?culture=en-US&EventID=1032326499

This past Easter weekend I had my first chance to dabble in an MMORPG (Massively-Multiplayer Online Role-Playing Game) called 'The Lord of the Rings Online - Shadows of Angmar”.  I've long resisted playing these games (though I flirted with the Star Wars Galaxies game for about 2 hrs) for a multitude of reasons, not the least of which is the potential time-sink it would become.

I was initially turned on to the game by a good friend, Eric Tolman, and the screenshots that he took while in-game.  I thought I'd bite the bullet and install it and give it a trial run during a free, open beta they're calling 'World Tour'.  I can honestly say - WOW (and no, not World of Warcraft).  The game is pretty amazing.  I played for a few hours, completing quests, gaining levels, interacting with other players, formed a fellowship, and helped save the town.  It was a lot of fun.  Like I say, though, I don't have much to compare it to since I've never really played the other games like it.

Though I'm not sure I'll ultimately subscribe to the service (when do I really have time to play games?), I can honestly say I'd have fun if I did.  I still have another 14-odd days left in the World Tour beta before the game goes live, and I intend to enjoy them.

As a reminder to everyone, we've having our second Utah Code Camp this next Saturday (April 14th, 2007) at Neumont University.  The last event was a great success and a ton of fun.  Please set aside the date and register.  Here's the official blurb:

What: Utah Spring Code Camp
When:  April 14th 2007 9:00 AM - 5:00 PM
Where: Neumont University
Registration: http://utahcodecamp.eventbrite.com

The local .NET Users Group and SQL Server Users Group is conducting a “Code Camp” for local software programmers next month at Neumont University.  The code camp is by the community for the community.  Always free and Always for the community.

The Saturday, April 14th event is scheduled from 9:00 AM to 5:00 PM. The conference is free please register at. http://utahcodecamp.eventbrite.com
Lots of Sponsors and Lots of software and Tech Gadgets to giveaway!
You can check out www.msutahevents.com  for a session schedule and speaker list for the Code Camp.