There are two primary reasons for venturing into the realms of clustering/load balancing - availability & scalability. In this post I’ll give a quick demo of how to setup Windows Network Load Balancing (NLB) on Server 2003 and how it affects the availability of a web application.

When we have several nodes doing the same thing, if one of them fails, the cluster as a whole continues - provided that the nodes are not so overburdened that a single node failing will kill the others due to the extra load. Most applications will have an upper limit on how much it can scale on a single box. You can get very far by vertically scaling your solution (buying faster & usally exponentially more expensive hardware), but true scalability always comes in the form of horizontal scaling. Add another box and receive more or less linear return of investment in regards of added computing power.

NLB enables us to easily add new nodes to a cluster and thus letting them share the workload. There are several issues to consider before setting up a cluster for a real application. Does the application share user file data? - in such a case you’ll have to make sure all nodes in the cluster have access to those files, and remember - the cluster is no stronger than the weakest link. Usually you’d probably go for a SAN to store all common files. State is also an important factor as most web applications rely on storing user specific data in statebags like the Session object. When the load is balanced out on several servers, the user could potentially visit several servers during his visit to the website, and unless the state storage is centralized, the user will have different session data on each server. Again, remember that you’ll have to provide a redundant storage location for the session data, or else you’ll compromise the availability of the cluster.

In this demonstration I’ll be using two virtual PCs running on my own host computer. All three computers are running on the subnet 192.168.0.X. Here are the virtual machines involved:

192.168.0.34 - VENETIAN
192.168.0.32 – MIRAGE

When you setup an NLB cluster, you create a new virtual IP address that gets mapped to each individual server, besides their own static IP address. In this demo I’ll setup the cluster on the virtual IP address 192.168.0.50. Before we get too far, I should mention that if you are going to setup an NLB cluster in a production environment, you should use machines with dual NICs so one NIC can connect to the public lan while the other NIC connects to a private switched lan where only the cluster nodes are connected. This ensures that the internal cluster communication is not polluting the general network traffic, aswell as making it a lot more efficient since we’ll then be able to utilize unicast communciation between the nodes instead of relying on multicast communication.

For demonstrating the effects of a node crash, I’ve created a very simple load testing tool, the main functionality is an infinite loop trying to make a request at a time while registering success/failures. It’ll also show the text result that is returned in a one-line textbox:

while (running)
{
	HttpWebRequest req = (HttpWebRequest)WebRequest.Create(txtUrl.Text);
	req.Timeout = 1000;
	req.KeepAlive = false;

	try
	{
		WebResponse res = req.GetResponse();
		using (StreamReader sr = new StreamReader(res.GetResponseStream()))
			lastResult = sr.ReadToEnd();

		successfulRequests++;
	}
	catch (WebException)
	{
		failedRequests++;
	}
}

As I’m running this test on two VPCs on my local machine, I won’t be getting any extra performance out of my cluster since the VPCs will just use 50% CPU each. This demo will concentrate on the availability of the cluster - but feel assured that performance will be better if you distribute the cluster over several separate computers.

Here’s the demo of how to setup & use NLB:

Other good resources

Rick Strahl on Web Farming with the Network Load Balancing Service in Windows Server 2003

Peter A. Bromberg on Network Load Balancing, Session State and IP Affinity

Downloads

SimpleLoadTester.zip - Sample code

Some of you may know that I used to play a lot of poker. Unfortunately that’s not the case any more. I really enjoy live poker when I’m in Vegas, I enjoy the major tournaments and I’ve definitely not participated in my last WSOP. But as for online poker and the daily grind, I’ve quit it. I just don’t find it exciting any more. While the mathematical aspect acquired my interest early on, I never enjoyed grinding as such, it was purely for monetary reasons.

Anyways, one of the ways I kept enjoying was by utilizing one of my skills, multi tasking. I remember back in the days when you could play all the Party skins at the same time, about 20 tables simultaneously. While multitabling I started developing a small helper application for my own use, Multi Table Helper (MTH). Before long, it got released publicly and it got quite popular.

Anyways, fast forward a couple of years, MTH was up to version 2.28, I had it running on Stars & Party and actually on Crypto and FTP as well, locally. Now, this was also about the time my company went haywire in regards to client jobs. I’d lost my interest in online poker. I worked round the clock, so MTH didn’t get the attention it needed.

Fast forward about a year more. I realized MTH needed an update, and I actually wanted to continue the development. I started the Core project. The Core project was meant to be semi-open source. I worked on creating a stable and extensible core that could be extended in any way possible. I never got to finish it off though.

I’ve now realized that I won’t have the time to finish the Core project off anytime soon, and without playing poker myself, it’s tough to find a real interest in it. So here we go, I’m releasing both the original latest release of MTH as well as the MTH Core source code under the GPL license.

Now, a couple of notices.

I’ve removed pretty much everything registration related, there may still be some leftovers though. You cannot compile neither project as it is now since it depends on Desaware SpyWorks. SpyWorks is not free and thus I cannot bundle it. If you wish to compile MTH in its current state, you will have to buy SpyWorks yourself.

MTH uses AutoIt3 as well, I have not bundled it, you will have to download and install it yourself.

MTH’s Speech recognition depends on the Windows Speech API, you will have to download and install that from Microsoft first.

MTH is released under the GNU General Public License. You are allowed to continue work on MTH, make releases and use parts in your own projects. You may not use parts of MTH for commercial products, any parts of MTH used in your own projects must be released as by the demands of the GPL license. Please see the license.txt file included in the downloads.

The classic MTH (2.28) is not particularly beautiful architecturally, but it is by far the most complete application of the two. The Core project is much further from actually working, but I hope someone will continue the work and complete the Core project as I originally imagined it. Both applications will need some work, but it should be a very good starting point.

Those of you that have bough MTH

Thank you. I will not be accepting any further registrations. I will keep the current registration service running, I do not have any plans of shutting it down, so you should be able to use MTH just like you do today.

Help

If you have any questions regarding the source code, or any other related matter, please let me know. I will not guarantee any help in regards of making the code run, but I might be able to give some pointers.

Downloads

https://github.com/improvedk/Multi-Table-Helper

I spend a lot of time architecting for scalability, availability and security during my daily work. Currently I’ve got a distributed system consisting of several windows services communicating across machines using WCF and authenticating through Active Directory.

In such a situation, if the Active Directory Domain Controller (let’s just call it DC from now on) dies, everything more or less dies as no clients/servers are able to authenticate incoming requests anymore. Security is paramount, so the services are not allowed to simply cache the domain logon, thus the logon has the occur at each service call - requiring a fully working DC.

In this post I’ll attempt to implement a secondary failover DC and investigate how it affects a downtime situation. I’ll be using a couple of simple WCF based applications to test the DC. I will be using three virtual machines. Luxor is the primary DC. MGM is part of the domain, and this is the machine hosting the WCF server. Later on I’ll add the third virtual machine, Excalibur, being the failover DC. The WCF client will be running from my own machine. Note that I will not show how to install the primary DC, there are plenty other great guides on how to setup the primary DC. I also won’t be going into how to install the primary DNS server.

This is the WCF server interface:

using System;
using System.Security.Principal;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;

namespace Contracts
{
	public class WCFHelper
	{
		/// <summary>
		/// Creates an instance of the specified interface type by channeling to the service host.
		/// </summary>
		/// <typeparam name="T">The interface type to create.</typeparam>
		/// <param name="binding">The binding protocol to use.</param>
		/// <param name="endpointAddress">The complete address for the endpoint.</param>
		/// <returns>An instance of type T.</returns>
		public static T CreateChannel<T>(Binding binding, string endpointAddress)
		{
			// Create an endpoint for the specified binding & address
			ServiceEndpoint endpoint = new ServiceEndpoint(ContractDescription.GetContract(typeof(T)), binding, new EndpointAddress(endpointAddress));
			binding.SendTimeout = new TimeSpan(0, 0, 1);
			binding.ReceiveTimeout = new TimeSpan(0, 0, 1);
			binding.OpenTimeout = new TimeSpan(0, 0, 1);
			binding.CloseTimeout = new TimeSpan(0, 0, 1);

			// Create a channel factory of type T
			ChannelFactory<T> factory = new ChannelFactory<T>(endpoint);
			factory.Credentials.Windows.ClientCredential.UserName = "RedundancyCheck";
			factory.Credentials.Windows.ClientCredential.Password = "RedundancyCheck";
			factory.Credentials.Windows.ClientCredential.Domain = "IPAPER";
			factory.Credentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;

			// Return the created channel
			return factory.CreateChannel();
		}

		/// <summary>
		/// Creates a ServiceHost hosting the specific implementation TImplementationType of interface TInterfaceType at the specified endpointAddress.
		/// </summary>
		/// <typeparam name="TInterfaceType">The interface type to host.</typeparam>
		/// <typeparam name="TImplentationType">The implementation type to host.</typeparam>
		/// <param name="binding">The binding protocol to use.</param>
		/// <param name="endpointAddress">The endpoint where the service should be hosted at.</param>
		/// <param name="mexEndpointAddress">The endpoint where the service metadata should be hosted at.</param>
		/// <returns>An instance of ServiceHost.</returns>
		public static ServiceHost CreateServiceHost<TInterfaceType, TImplentationType>(Binding binding, string endpointAddress, string mexEndpointAddress)
		{
			// Create new service host
			ServiceHost host = new ServiceHost(typeof(TImplentationType));

			// Create endpoints
			host.AddServiceEndpoint(typeof(TInterfaceType), binding, endpointAddress);

			// Create metadata endpoint if it doesn't exist
			ServiceMetadataBehavior smb = host.Description.Behaviors.Find<ServiceMetadataBehavior>();
			if (smb == null)
			{
				smb = new ServiceMetadataBehavior();
				smb.MetadataExporter.PolicyVersion = PolicyVersion.Policy15;
				host.Description.Behaviors.Add(smb);
			}
			if (binding is NetTcpBinding)
				host.AddServiceEndpoint(ServiceMetadataBehavior.MexContractName, MetadataExchangeBindings.CreateMexTcpBinding(), mexEndpointAddress);
			else if (binding is BasicHttpBinding)
				host.AddServiceEndpoint(ServiceMetadataBehavior.MexContractName, MetadataExchangeBindings.CreateMexHttpBinding(), mexEndpointAddress);
			else
				throw new ArgumentOutOfRangeException("Invalid binding: " + binding);

			return host;
		}
	}
}

Notice that I’ve hardcoded the user “IPAPERRedundancyCheck” with a password of “RedundancyCheck” - this is for test purposes only, don’t even bother commenting on password security :) Also notice that I’ve set a timeout of 1 second - when the DC fails, I don’t want to spend 30 seconds before knowing if it’s down, I want to know about it right away. And since all machines are running locally, 1 second is plenty.

This is the server that’ll be running on the MGM machine on port 8000:

using System;
using System.Security;
using System.Security.Permissions;
using System.ServiceModel;
using Contracts;

namespace WcfServer
{
	class Program
	{
		static void Main(string[] args)
		{
			// Create service host
			ServiceHost host = WCFHelper.CreateServiceHost<IServer, Server>(new NetTcpBinding(SecurityMode.Transport), "net.tcp://localhost:8000", "net.tcp://localhost:8000/mex");

			// Open host
			host.Open();

			Console.Read();
		}
	}

	[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
	public class Server : IServer
	{
		public string Ping()
		{
			// Security check
			try
			{
				new PrincipalPermission(null, "ADTest").Demand();
			}
			catch
			{
				throw new SecurityException();
			}

			// Let caller know that we're alive
			return "Pong!";
		}
	}
}

We’ve got an implementation of the IServer interface with the single method “Ping”. We test that the user is part of the “ADTest” role by demanding it on the current principal. If something fails we throw a SecurityException which will let the client now authentication failed. If the client is authenticated, we return a pong.

And finally we have the client that’ll be running on my own machine:

using System;
using System.ServiceModel;
using System.Threading;
using Contracts;

namespace WcfClient
{
	class Program
	{
		static void Main(string[] args)
		{
			while (true)
			{
				try
				{
					IServer server = WCFHelper.CreateChannel(new NetTcpBinding(SecurityMode.Transport), "net.tcp://192.168.0.35:8000");

					using (server as IDisposable)
					{
						try
						{
							Console.WriteLine(server.Ping());
						}
						catch (Exception ex)
						{
							Console.WriteLine(ex.Message);
						}
					}
				}
				catch (CommunicationObjectFaultedException)
				{ }

				Thread.Sleep(250);
			}
		}
	}
}

We create a channel to the MGM machine (static IP of 192.168.0.35). We’ll continue to call the IServer service every 250ms, writing either the result of the Ping function, or a message explaining any problems that have occurred. Note that in real life situations we’d not create a new channel each time, but in this case we have to, so we really do authenticate on each call (so we’re affected immediatly when the DC dies).

The following clip will show what happens when we run the server & client while the DC goes down (by pausing the virtual machine). Note that we’re receiving a timeout exception, not a SecurityException. This is because I’m using SecurityMode.Transport which requires us to authenticate before even reaching the service, thus the method is never invoked, and the PrincipPermission.Demand() call is NOT the one failing us, it’s the WCF security layer trying to open a TCP transport. As soon as the DC (running on LUXOR) fails, we lose connectivity with our service (running on MGM).

The goal obviously is to prevent this from happening, we cannot have all our services brought to a standstill if the DC fails. The first step in installing a failover AD DC is to get a DNS secondary server up and running (on the soon to be secondary DC machine) so we have redundant DNS functionality.

A quick recap of the servers:

LUXOR = Primary DC, primary DNS
EXCALIBUR = To be secondary DC, secondary DNS
MGM = Client server

Now that we’ve got the secondary DNS set up, we’re ready to install Active Directory on the secondary AD server (EXCALIBUR). The following video shows how easy it is to install a failover DC:

That’s it! After the server reboots, it now functions as a failover DC in case the primary one kicks the bucket. I’ll end this post post by running my WcfServer application on the MGM server whilst both LUXOR and EXCALIBUR are running. You’ll se a fluent stream of “Pong”s returning. After shutting down LUXOR, the WCF client will immediately start reporting connection problems, but after a short while it automatically starts returning Pongs again - it got a hold of the second DC! Now, if I shut down the second DC aswell, we’ll get errors in our client again. If I then restart the primary DC, after a short while, the client starts Ponging again - we got a hold of the primary DC. So we haven’t eliminated downtime completely, but we’ve reduced it to a 5-30 sec period before everything automatically switches over to the failover DC.

This is my first blog post utilizing videos - does it work? Do you prefer seeing live video like this, or a long series of screenshots? I know what I prefer :)

UPDATE

After testing my code through based on JP’s comments, I’ve realized my implementation was way too naïve and cannot be used for most datasets. For a correct weighted random implementation, see Dems’ answer on StackOverflow.

Original (flawed) implementation

There are no built-in functions for selecting weighted averages in SQL Server. Fortunately it’s a simple task to do so oneself.

We’ll use this table as an example:

CREATE TABLE #tmp
(
	Name varchar(64),
	Points int
)

INSERT INTO #tmp VALUES ('Mark', 25);
INSERT INTO #tmp VALUES ('Jakob', 12);
INSERT INTO #tmp VALUES ('Peter', 17);
INSERT INTO #tmp VALUES ('Anders', 0);
INSERT INTO #tmp VALUES ('Kirsten', 33);
INSERT INTO #tmp VALUES ('Mads', 4);

This table represents a list of players in an arbitrary game. The more points you have, the bigger the chance of winning. It has to be weighted, meaning that the person with just 4 points may win, but is unlikely to do so.

The RAND() function in SQL Server returns a floating point number between 0 and 1. Multiplying that with our points gives a random weight based on the amount of points. Unfortunately the RAND() function is seeded once for each query, not for each row - meaning that for each row RAND() will yield the same result, effectively multiplying the points with a constant all the way through. We need to provide a new seed for the RAND() function for each row. NEWID() returns a new unique identifier that may be used as a seed if cast to VARBINARY:

SELECT Name, Points, RAND(CAST(NEWID() AS VARBINARY)) * Points AS Weight FROM #tmp ORDER BY Weight DESC

Name     Points  Weight
Peter    17      15,9795741766356
Mark     25      14,9122204505153
Kirsten  33      9,67888480542761
Jakob    12      9,38697608441358
Mads     4       0,833340539027792
Anders   0       0

And here we have the result ordered by weight. As you can see, although Kirsten has the most points, Peter ended up winning the competition.

FIG1: Showing the statistical distribution of using RAND(NEWID())

DECLARE @SampleSize int = 100000;

WITH RND AS
(
	select top (@SampleSize) ROUND(RAND(CAST(NEWID() AS VARBINARY)), 1, 1) AS RandValue from sys.objects cross join sys.columns
)
SELECT
	COUNT(*),
	RandValue,
	COUNT(*) / CAST(@SampleSize AS float) * 100 AS [%]
FROM
	RND
GROUP BY
	RandValue
ORDER BY
	COUNT(*) DESC

Results:

COUNT	VAL	%
10117	0,8	10,117
10091	0,4	10,091
10073	0	10,073
10034	0,9	10,034
9996	0,5	9,996
9993	0,2	9,993
9956	0,7	9,956
9927	0,6	9,927
9923	0,3	9,923
9890	0,1	9,89

I often need to transfer data from my business layer to my presentation layers in a strongly typed way. In this example I’ll use the following very struct and corresponding DataTable to represent the data I need to transfer:

struct Test
{
	public string Name;
	public int Value;
}

using (DataTable dt = new DataTable())
{
	dt.Columns.Add("Name", typeof(string));
	dt.Columns.Add("Value", typeof(int));
}

Our objective is basically to transfer the DataRows in the DataTable into a List that can be transferred on to the next layer.

The fastest way possible would be doing it manually like so:

List list = new List();
foreach (DataRow dr in dt.Rows)
{
	Test t = new Test();
	t.Name = dr["Name"].ToString();
	t.Value = Convert.ToInt32(dr["Value"]);

	list.Add(t);
}

But this takes a lot of time to write, especially if your objects vary a lot and you have to create a lot of them. That’s where my MapList function comes into play:

private static List MapList(DataTable dt)
{
	List list = new List();

	FieldInfo[] fields = typeof(T).GetFields();
	T t = Activator.CreateInstance();

	foreach (DataRow dr in dt.Rows)
	{
		foreach (FieldInfo fi in fields)
			fi.SetValueDirect(__makeref(t), dr[fi.Name]);

		list.Add(t);
	}

	return list;
}

It takes a DataTable as the sole parameter (you could easily use a DataReader if you wanted to). It retrieves the fields of the generic type by reflection. It is important to note that this includes all fields of the type, so we’re expecting there to be a 1:1 map of the DataTable and the types’ fields. Another important remark is that the generic type must be a struct - for us to be able to move the type instantiation outside of the loop (for performance), it must be a struct (since adding it to the List will create a copy). If it were a class, we would overwrite the already existing objects each time we iterated a new row in the DataTable.

The SetValueDirect() method is somewhat faster than SetValue(). Caching the TypedReference for t by creating it outside the loop actually decreased performance, I’ll probably have to look into the IL code to identify why. I’ve also tried caching the DataRow ordinals, there is no performance gain to be seen unless we’re talking several millions of datarows, and in that case - this function is not the way to go.

I would really like to obtain a pointer to the struct fields and set the values directly using some unsafe pointer magic - anyone know how to obtain a pointer to the field hiding behind the FieldInfo type we get by reflection?

So what’s the catch as opposed to doing it manually? Performance. Here’s a graph that shows the performance hit in sets of 1 to 1.000.000 iterations. Note that at 1000 iterations the timing shifts from ticks to milliseconds and that the graph is using a base 10 logarithmic scale.

Obviously there’s a performance hit - rather consistently, a factor of 10 -, but depending on the situation it is to be used in, mapping the objects by reflection may easily be a viable solution.

ConsoleApplication1.rar - Sample solution

I do a lot of backend programming for Flash frontends. That basically means a lot of ASPX pages that simply return some XML and accept some incoming XML for parameters. Most of the UI logic ends up getting cluttered with manual XML stringbuilding, so I saw this as an obvious opportunity to play around with a fluent interfaces.

Now, here’s an example of a typical boolean yes/no result from a Flash query:

<?xml version="1.0" encoding="utf-8"?>
<root>
	<result type="boolean">true</result>
</root>

I’d usually create this bit of XML using a simple StringBuilder like so:

StringBuilder output = new StringBuilder();
output.Append("<?xml version="1.0" encoding="utf-8"?>");
output.Append("<root>");
output.Append("<result type="boolean">true</result>");
output.Append("</root>");

This has the advantage of being very fast to write, but readability suffers from the escaped quotes, lack of indentation and there’s a whole lot of text when the XML becomes just a bit more advanced.

A “prettier” way is to use the DOM through the XmlDocument like so:

XmlDocument xd = new XmlDocument();
xd.AppendChild(xd.CreateXmlDeclaration("1.0", "utf-8", ""));

XmlNode root = xd.CreateElement("root");
xd.AppendChild(root);

XmlNode result = xd.CreateElement("result");
result.InnerText = "true";

XmlAttribute type = xd.CreateAttribute("type");
type.Value = "boolean";

result.Attributes.Append(type);
root.AppendChild(result);

While this does produce exactly the same XML, it takes up twice as many lines of code, excluding the whitespace lines. Without whitespace it is even more unreadable.

Let me introduce you to my quick’n’simple fluent interface that uses XmlDocument internally, XmlOutput:

XmlOutput xo = new XmlOutput()
	.XmlDeclaration()
	.Node("root").Within()
		.Node("result").Attribute("type", "boolean").InnerText("true");

Using XmlOutput we’re down to four lines, the shortest example yet. While linecount is not, and should not be, a measurement of quality, it is preferred. I believe using XmlOutput is, by far, the most readable example.

Basically, using Node() creates a new node within the current node. If no node has been created previously, it will automatically be the root node. Any time a new node is created, it automatically becomes the “current node”. Calling Within() moves the context into the current node, thus any newly created nodes will be created within that node. Attribute() will add an attribute to the current node, likewise will InnerText() set the InnerText of the current node. EndWithin() moves the context to the parent node, it is not mandatory for “closing” the nodes, it is only required when you actually need to move the scope.

Let me present you with a couple of examples. Dynamic data:

XmlOutput xo = new XmlOutput()
	.XmlDeclaration()
	.Node("root").Within()
		.Node("numbers").Within();

for (int i = 1; i <= 10; i++)
	xo.Node("number").Attribute("value", i.ToString()).InnerText("This is the number: " + i);

<?xml version="1.0" encoding="utf-8"?>
<root>
	<numbers>
		<number value="1">This is the number: 1</number>
		<number value="2">This is the number: 2</number>
		<number value="3">This is the number: 3</number>
		<number value="4">This is the number: 4</number>
		<number value="5">This is the number: 5</number>
		<number value="6">This is the number: 6</number>
		<number value="7">This is the number: 7</number>
		<number value="8">This is the number: 8</number>
		<number value="9">This is the number: 9</number>
		<number value="10">This is the number: 10</number>
	</numbers>
</root>

And complex structures:

XmlOutput xo = new XmlOutput()
	.XmlDeclaration()
	.Node("root").Within()
		.Node("user").Within()
			.Node("username").InnerText("orca")
			.Node("realname").InnerText("Mark S. Rasmussen")
			.Node("description").InnerText("I'll handle any escaping (like < & > for example) needs automagically.")
			.Node("articles").Within()
				.Node("article").Attribute("id", "25").InnerText("Handling DBNulls")
				.Node("article").Attribute("id", "26").InnerText("Accessing my privates")
				.EndWithin()
			.Node("hobbies").Within()
				.Node("hobby").InnerText("Fishing")
				.Node("hobby").InnerText("Photography")
				.Node("hobby").InnerText("Work");

<?xml version="1.0" encoding="utf-8"?>
<root>
	<user>
		<username>orca</username>
		<realname>Mark S. Rasmussen</realname>
		<description>I'll handle any escaping (like < & > for example) needs automagically.</description>
		<articles>
			<article id="25">Handling DBNulls</article>
			<article id="26">Accessing my privates</article>
		</articles>
		<hobbies>
			<hobby>Fishing</hobby>
			<hobby>Photography</hobby>
			<hobby>Work</hobby>
		</hobbies>
	</user>
</root>

Finally, say hello to XmlOutput:

using System.Xml;
using System.Collections.Generic;

public class XmlOutput
{
	// The internal XmlDocument that holds the complete structure.
	XmlDocument xd = new XmlDocument();

	// A stack representing the hierarchy of nodes added. nodeStack.Peek() will always be the current node scope.
	Stack<XmlNode> nodeStack = new Stack<XmlNode>();

	// Whether the next node should be created in the scope of the current node.
	bool nextNodeWithin;

	// The current node. If null, the current node is the XmlDocument itself.
	XmlNode currentNode;

	/// <summary>
	/// Returns the string representation of the XmlDocument.
	/// </summary>
	/// <returns>A string representation of the XmlDocument.</returns>
	public string GetOuterXml()
	{
		return xd.OuterXml;
	}

	/// <summary>
	/// Returns the XmlDocument
	/// </summary>
	/// <returns></returns>
	public XmlDocument GetXmlDocument()
	{
		return xd;
	}

	/// <summary>
	/// Changes the scope to the current node.
	/// </summary>
	/// <returns>this</returns>
	public XmlOutput Within()
	{
		nextNodeWithin = true;

		return this;
	}

	/// <summary>
	/// Changes the scope to the parent node.
	/// </summary>
	/// <returns>this</returns>
	public XmlOutput EndWithin()
	{
		if (nextNodeWithin)
			nextNodeWithin = false;
		else
			nodeStack.Pop();

		return this;
	}

	/// <summary>
	/// Adds an XML declaration with the most common values.
	/// </summary>
	/// <returns>this</returns>
	public XmlOutput XmlDeclaration() { return XmlDeclaration("1.0", "utf-8", ""); }

	/// <summary>
	/// Adds an XML declaration to the document.
	/// </summary>
	/// <param name="version">The version of the XML document.</param>
	/// <param name="encoding">The encoding of the XML document.</param>
	/// <param name="standalone">Whether the document is standalone or not. Can be yes/no/(null || "").</param>
	/// <returns>this</returns>
	public XmlOutput XmlDeclaration(string version, string encoding, string standalone)
	{
		XmlDeclaration xdec = xd.CreateXmlDeclaration(version, encoding, standalone);
		xd.AppendChild(xdec);

		return this;
	}

	/// <summary>
	/// Creates a node. If no nodes have been added before, it'll be the root node, otherwise it'll be appended as a child of the current node.
	/// </summary>
	/// <param name="name">The name of the node to create.</param>
	/// <returns>this</returns>
	public XmlOutput Node(string name)
	{
		XmlNode xn = xd.CreateElement(name);

		// If nodeStack.Count == 0, no nodes have been added, thus the scope is the XmlDocument itself.
		if (nodeStack.Count == 0)
		{
			xd.AppendChild(xn);

			// Automatically change scope to the root DocumentElement.
			nodeStack.Push(xn);
		}
		else
		{
			// If this node should be created within the scope of the current node, change scope to the current node before adding the node to the scope element.
			if (nextNodeWithin)
			{
				nodeStack.Push(currentNode);

				nextNodeWithin = false;
			}

			nodeStack.Peek().AppendChild(xn);
		}

		currentNode = xn;

		return this;
	}

	/// <summary>
	/// Sets the InnerText of the current node without using CData.
	/// </summary>
	/// <param name="text"></param>
	/// <returns></returns>
	public XmlOutput InnerText(string text)
	{
		return InnerText(text, false);
	}

	/// <summary>
	/// Sets the InnerText of the current node.
	/// </summary>
	/// <param name="text">The text to set.</param>
	/// <returns>this</returns>
	public XmlOutput InnerText(string text, bool useCData)
	{
		if (useCData)
			currentNode.AppendChild(xd.CreateCDataSection(text));
		else
			currentNode.AppendChild(xd.CreateTextNode(text));

		return this;
	}

	/// <summary>
	/// Adds an attribute to the current node.
	/// </summary>
	/// <param name="name">The name of the attribute.</param>
	/// <param name="value">The value of the attribute.</param>
	/// <returns>this</returns>
	public XmlOutput Attribute(string name, string value)
	{
		XmlAttribute xa = xd.CreateAttribute(name);
		xa.Value = value;

		currentNode.Attributes.Append(xa);

		return this;
	}
}

Enjoy!

Recently I was developing a couple of simple ORM classes that had me confused.

[Serializable]
public class Domain
{
	// Read
	private int domainID;
	public int DomainID { get { return domainID; } }
	
	private string domainName;
	public string DomainName { get { return domainName; } }

	// Read/Write
	public int? CompanyID;
}

Take this simple object as an example. It represents a website domain, it has an ID from the database aswell as a read only domain name and a belonging CompanyID.

Now, I want to create a Load() function that given a domain ID will instantiate a new instance of a Domain object and populate its values from the database. Now, as the DomainID and DomainName variables are private, I’ll have to make a constructor method to pass in those values, right? It seems not, if my Load method is a static method of the Domain class itself:

public static Domain Load(int domainID)
{
	SqlCommand cmd = new SqlCommand("SELECT Domain, CompanyID FROM tblDomains WHERE DomainID = @DomainID");
	cmd.Parameters.Add("@DomainID", SqlDbType.Int).Value = domainID;

	DataRow dr = DB.GetDR(cmd);

	if (dr == null)
		return null;
	else
	{
		Domain d = new Domain();
		
		d.domainID = domainID;
		d.domainName = DBConvert.To<string>(dr["Domain"]);
		d.CompanyID = DBConvert.To<int?>(dr["CompanyID"]);

		return d;
	}
}

The interesting part is in the else block. I create a new Domain instance, and I’m able to set the private field values directly. This saves me a lot of work as I don’t have to create constructor methods - I’d rather not have to maintain those as well, and I think this was is a lot more readable. Anyways, I don’t understand why this is possible. Granted, my static method is part of the Domain class and as such could have access to private variables, but these private variables are not static and thus they belong to the actual Domain instance (d). Since they belong to the Domain instance and I’m setting them through the instance variable d, how am I able to access them? Ought they noe be private, even though I’m writing my code inside the Domain function?

Everything compiles and runs perfectly, I just don’t understand why this is possible.

Update:
Jakob Andersen provided me with the answer. It’s simply a matter of scope, whether the method is static or not does not matter. Also encapsulation is ignored as scope takes precedence.

Reading and writing values to the DB has always been a bit cumbersome when you had to take care of nullable types and DBNull values. Here’s a way to make it easy. Based on this post by Peter Johnson and this post by Adam Anderson I gathered a couple of ideas and combined them to make a completely generic class that will handle DBNulls for both reads and writes, as well as handling nullable types. Let me present the code, I’ll go over it afterwards:

public static class DBConvert
{
	/// <summary>
	/// Handles reading DBNull values from database in a generic fashion
	/// </summary>
	/// <typeparam name="T">The type of the value to read</typeparam>
	/// <param name="value">The input value to convert</param>
	/// <returns>A strongly typed result, null if the input value is DBNull</returns>
	public static T To<T>(object value)
	{
		if (value is DBNull)
			return default(T);
		else
			return (T)changeType(value, typeof(T));
	}

	/// <summary>
	/// Handles reading DBNull values from database in a generic fashion, simplifies frontend databinding
	/// </summary>
	/// <typeparam name="T">The type of the value to read</typeparam>
	/// <param name="ri">The Container item in a databinding operation</param>
	/// <param name="column">The dataitem to read</param>
	/// <returns>A strongly typed result, null if the input value is DBNull</returns>
	public static T To<T>(RepeaterItem ri, string column)
	{
		if (DataBinder.Eval(ri.DataItem, column) is DBNull)
			return default(T);
		else
			return (T)changeType(DataBinder.Eval(ri.DataItem, column), typeof(T));
	}

	/// <summary>
	/// Internal method that wraps Convert.ChangeType() so it handles Nullable<> types
	/// </summary>
	/// <param name="value">The value to convert</param>
	/// <param name="conversionType">The type to convert into</param>
	/// <returns>The input value converted to type conversionType</returns>
	private static object changeType(object value, Type conversionType)
	{
		if (conversionType.IsGenericType && conversionType.GetGenericTypeDefinition().Equals(typeof(Nullable<>)))
		{
			if (value == null)
				return null;

			conversionType = Nullable.GetUnderlyingType(conversionType);
		}

		return Convert.ChangeType(value, conversionType);
	}

	/// <summary>
	/// Simplifies setting SqlParameter values by handling null issues
	/// </summary>
	/// <param name="value">The value to return</param>
	/// <returns>DBNull if value == null, otherwise we pass through value</returns>
	public static object From(object value)
	{
		if (value == null)
			return DBNull.Value;
		else
			return value;
	}
}

The first To method significantly simplifies the process of setting database values when using SqlParameters (we all do, right?).

This is how I used to handle possible DBNulls when reading into a nullable integer:

if(dr["CountryID"] is DBNull)
	c.CountryID = null;
else
	c.CountryID = Convert.ToInt32(dr["CountryID"]);

And this is how it’s done using my DBConvert class:

c.CountryID = DBConvert.To<int?>(dr["CountryID"]);
c.Recommended = DBConvert.To<bool>(dr["Recommended"]);
d.companyMessageCreated = DBConvert.To<DateTime?>(dr["CompanyMessageCreated"]);
d.Message = DBConvert.To<string>(dr["Message"]);
d.OverallScore = DBConvert.To<int>(dr["OverallScore"]);

Notice how it works for both nullable ints, DateTimes and whatever other nullable types you wish. It also works for normal types like string, int and so forth. It’ll automatically typecast it into the type specified as a generic parameter. However, remember that the database value must match the value being converted to, you cannot use .To(“some string value”), it will fail.

The private changeType() method is a wrapper for the ChangeType() method that takes care of nullable types since the builtin Convert.ChangeType() method does not support casting into nullable types.

The second To simplifies databinding values in the frontend ASPX files. This is how I used to print a DateTime column in a ShortDateString format:

<%# Convert.ToDateTime(DataBinder.Eval(Container.DataItem, "Created")).ToShortDateString() %>

And this is how it can be done using the DBConvert class, generically:

<%# DBConvert.To<DateTime>(Container, "Created").ToShortDateString() %>

Nullable types, as well as null strings also have to be handled when assigning SqlParameter values. The usual way for both nullable types as well as strings might look like this:

if(CountryID == null)
	cmd.Parameters.Add("@CountryID", SqlDbType.Int).Value = DBNull.Value;
else
	cmd.Parameters.Add("@CountryID", SqlDbType.Int).Value = CountryID;

Using the DBConvert class this can be done a bit simpler:

cmd.Parameters.Add("@CountryID", SqlDbType.Int).Value = DBConvert.From(CountryID);
cmd.Parameters.Add("@CountryID", SqlDbType.NVarChar, 2048).Value = DBConvert.From(MyString);

This will automatically convert null strings as well as nulled nullable types to DBNull.

Enjoy :)

I often see my fellow comp. sci. students writing their (relatively) simple Java code in applications like Emacs, Nano or Eclipse. I’m not fond of either application. I much prefer Visual Studios text handling, solution overview, output windows and so forth. What most people don’t know is that you can actually extend Visual Studio to a great extent. One way to extend Visual Studio is to write plugins using .NET, but there’s a way that is much simpler (albeit also more limited). I will now show how you can make Visual Studio compile and run your Java applications all within Visual Studio itself.

First of all, you need to download and install the Java JDK. Basically, you need to be able to call “javac” and “java” from any command prompt - which means you have to setup the environment settings so you have your JDK bin in the PATH variable.

Create a new Visual Studio project. It really doesn’t matter much what type you choose as there is no native Java project types. Choosing J# will not give you any advantages over, say a C# project. In this example I’ll use a C# Class Library project.

Start out by deleting the automatically created Class1.cs file. Add a new text file instead, I’ll call it MyApplication.java. You can write any standard Java code in the Java files, just like you’d ordinarily do.

One of the really cool features of Visual Studio is that it actually includes Intellisense for a lot of the standard Java classes, so you’re not left totally on your own.

Now comes the compilation part. Add a new text file to the project and call it Compile.bat. This will be the bat file that manages the actual compilation and execution of the application afterwards. Leave the file empty for now, we’ll enter the code in a short while.

Go to Tools -> External Tools…

Add a new entry called “Javac”, set the command path to your Compile.bat file and make sure the directory is set to the ProjectDir macro path. Check the “Use Output window” checkbox, this ensures the output is output directly into the Visual Studio output window.

Now enter the following into the Compile.bat file:

del Output /S /Q
mkdir Output
javac *.java -d Output
cd Output
start java MyApplication

Modify the MyApplication.java file so it ends with a call to System.in.read(), this ensures the application will stay open after we start it.

import java.io.*;

class MyApplication
{
	public static void main(String args[]) throws IOException
	{
		System.out.println("Hello World!");
		System.in.read();
	}
}

Now simply go to Tools -> Javac and watch your Java application compile and run.

You can of course modify the build script in whatever way you wish to support larger applications. You could also use ANT build scripts, unit tests and so forth. To make compiling easier, you can create a key command (Tools -> Options -> Keyboard -> Tools.ExternalCommandX where X is the Javac commands index in the Tools menu) to the Javac command in the Visual Studio settings, I use Ctrl+Shift+J for Java compilation myself.

I will be attending JAOO this year, I’m looking forward to a packed calendar in excitement:

It’s a tough decision, choosing which sessions to fit into the available slots. There are several slots where I’d love to watch multiple sessions.

I look forward to meeting a lot of people, colleagues and friends. Conferences are superb for developing ones network.

I like the multitude of technologies and areas that are covered at JAOO, Java, JavaScript, .NET, virtualization, architecture and so forth. I’d love to see some more DBA related sessions though (especially MSSQL 2008).