I’ve previously written of how to automatically map a DataTable into a strongly typed collection of objects. There’s a problem though, it’s not fast… I wanted to improve on it, and this is what I ended up with.
The original method relied heavily on reflection to set the values directly. Reflection’s bad in regards of speed, mkay? But reflection is not necessarily evil, you can do great good with it. Now, the problem with the original method is that each field is set using reflection for each row, that’s [number of fields] * [number of rows] fields being set using reflection, resulting in pretty poor performance. If we compare it to the manual way, there’s obviously some kind of a gap. If we could just somehow, in a generic way, create mapper methods like we do manually…
We’ll have to create a DynamicMethod that takes in a DbDataReader, converts the current row to a generic type T. We’ll call it MapDR, although it doesn’t really matter what it’s called.
// Our method will take in a single parameter, a DbDataReader Type[] methodArgs = { typeof(DbDataReader) }; // The MapDR method will map a DbDataReader row to an instance of type T DynamicMethod dm = new DynamicMethod("MapDR", typeof(T), methodArgs, Assembly.GetExecutingAssembly().GetType().Module); ILGenerator il = dm.GetILGenerator();
In this method, we’ll create an instance of the generic type T and store it as a variable.
// We'll have a single local variable, the instance of T we're mapping il.DeclareLocal(typeof(T)); // Create a new instance of T and save it as variable 0 il.Emit(OpCodes.Newobj, typeof(T).GetConstructor(Type.EmptyTypes)); il.Emit(OpCodes.Stloc_0);
Then we’ll look each property of the type.
foreach (PropertyInfo pi in typeof(T).GetProperties())
Now we’ll read the column value from the DbDataReader using the properties name. By reading it, we’re pushing it onto the stack.
// Load the T instance, SqlDataReader parameter and the field name onto the stack il.Emit(OpCodes.Ldloc_0); il.Emit(OpCodes.Ldarg_0); il.Emit(OpCodes.Ldstr, pi.Name); // Push the column value onto the stack il.Emit(OpCodes.Callvirt, typeof(DbDataReader).GetMethod("get_Item", new Type[] { typeof(string) }));
Now’s the ugly part. Depending on the type, there are different ways to convert it into the corresponding .NET type, i’ve made a switch statement handling most common types, although it does lack support for nullable types, guids and various other numeric formats. It should show to idea though. Converting the value will push the resulting correctly typed value onto the stack, and pop the original value in the process.
// Depending on the type of the property, convert the datareader column value to the type switch (pi.PropertyType.Name) { case "Int16": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt16", new Type[] { typeof(object) })); break; case "Int32": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt32", new Type[] { typeof(object) })); break; case "Int64": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt64", new Type[] { typeof(object) })); break; case "Boolean": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToBoolean", new Type[] { typeof(object) })); break; case "String": il.Emit(OpCodes.Callvirt, typeof(string).GetMethod("ToString", new Type[] { })); break; case "DateTime": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToDateTime", new Type[] { typeof(object) })); break; case "Decimal": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToDecimal", new Type[] { typeof(object) })); break; default: // Don't set the field value as it's an unsupported type continue; }
And finally we set the properties value, thereby popping the value from the stack.
// Set the T instances property value il.Emit(OpCodes.Callvirt, typeof(T).GetMethod("set_" + pi.Name, new Type[] { pi.PropertyType }));
After we’ve mapped all the properties, we’ll load the T instance onto the stack and return it.
// Load the T instance onto the stack il.Emit(OpCodes.Ldloc_0); // Return il.Emit(OpCodes.Ret);
To improve performance, let’s cache this mapper method as it’ll work for the type T the next time we need it. Notice that what we’re caching is not the method itself, but a delegate to the method – enabling us to actually call the method.
private delegate T mapEntity<T>(DbDataReader dr); private static Dictionary<Type, Delegate> cachedMappers = new Dictionary<Type, Delegate>(); // Cache the method so we won't have to create it again for the type T cachedMappers.Add(typeof(T), dm.CreateDelegate(typeof(mapEntity<T>))); // Get a delegate reference to the dynamic method mapEntity<T> invokeMapEntity = (mapEntity<T>)cachedMappers[typeof(T)];
Now all we gotta do is loop the DbDataReader rows and return the mapped entities.
// For each row, map the row to an instance of T and yield return it while (dr.Read()) yield return invokeMapEntity(dr);
And of course, here’s the final method/class. Remember that this is more a proof of concept than a complete class. It ought to handle all necessary types. Also, it might be relevant to consider whether one should map public as well as private properties. Whether it should handle type validation errors, missing columns or not, I’m not sure. As it is now, it’ll throw a normal ArgumentOutOfRangeException in cases of missing columns, and relevant type conversion errors – all those can be handled by the callee using try/catch blocks.
using System; using System.Collections.Generic; using System.Data.Common; using System.Reflection; using System.Reflection.Emit; namespace Improve.Framework.Data { public class EntityMapper { private delegate T mapEntity<T>(DbDataReader dr); private static Dictionary<Type, Delegate> cachedMappers = new Dictionary<Type, Delegate>(); public static IEnumerable<T> MapToEntities<T>(DbDataReader dr) { // If a mapping function from dr -> T does not exist, create and cache one if (!cachedMappers.ContainsKey(typeof(T))) { // Our method will take in a single parameter, a DbDataReader Type[] methodArgs = { typeof(DbDataReader) }; // The MapDR method will map a DbDataReader row to an instance of type T DynamicMethod dm = new DynamicMethod("MapDR", typeof(T), methodArgs, Assembly.GetExecutingAssembly().GetType().Module); ILGenerator il = dm.GetILGenerator(); // We'll have a single local variable, the instance of T we're mapping il.DeclareLocal(typeof(T)); // Create a new instance of T and save it as variable 0 il.Emit(OpCodes.Newobj, typeof(T).GetConstructor(Type.EmptyTypes)); il.Emit(OpCodes.Stloc_0); foreach (PropertyInfo pi in typeof(T).GetProperties()) { // Load the T instance, SqlDataReader parameter and the field name onto the stack il.Emit(OpCodes.Ldloc_0); il.Emit(OpCodes.Ldarg_0); il.Emit(OpCodes.Ldstr, pi.Name); // Push the column value onto the stack il.Emit(OpCodes.Callvirt, typeof(DbDataReader).GetMethod("get_Item", new Type[] { typeof(string) })); // Depending on the type of the property, convert the datareader column value to the type switch (pi.PropertyType.Name) { case "Int16": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt16", new Type[] { typeof(object) })); break; case "Int32": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt32", new Type[] { typeof(object) })); break; case "Int64": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToInt64", new Type[] { typeof(object) })); break; case "Boolean": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToBoolean", new Type[] { typeof(object) })); break; case "String": il.Emit(OpCodes.Callvirt, typeof(string).GetMethod("ToString", new Type[] { })); break; case "DateTime": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToDateTime", new Type[] { typeof(object) })); break; case "Decimal": il.Emit(OpCodes.Call, typeof(Convert).GetMethod("ToDecimal", new Type[] { typeof(object) })); break; default: // Don't set the field value as it's an unsupported type continue; } // Set the T instances property value il.Emit(OpCodes.Callvirt, typeof(T).GetMethod("set_" + pi.Name, new Type[] { pi.PropertyType })); } // Load the T instance onto the stack il.Emit(OpCodes.Ldloc_0); // Return il.Emit(OpCodes.Ret); // Cache the method so we won't have to create it again for the type T cachedMappers.Add(typeof(T), dm.CreateDelegate(typeof(mapEntity<T>))); } // Get a delegate reference to the dynamic method mapEntity<T> invokeMapEntity = (mapEntity<T>)cachedMappers[typeof(T)]; // For each row, map the row to an instance of T and yield return it while (dr.Read()) yield return invokeMapEntity(dr); } public static void ClearCachedMapperMethods() { cachedMappers.Clear(); } } }
-
To avoid the big switch (pi.PropertyType.Name), I usually generate a cast or unbox instruction, depending on whether pi.PropertyType is a reference or value type respectively. Granted, that implies that the actual type returned from the datareader must be equal to the property type, not just convertible to it. But that’s what I usually want anyway. It works for all types (supported by the datareader) automatically, and it’s faster at execution time.
Also, I store all MethodInfo’s etc. I need in static readonly variables, that speeds up the code generation.
-
Oh, and you want to verify if the property has a public setter method before you generate code to set it. Don’t forget that readonly properties are not an error condition, they can be used to calculate derived data.
BTW, you should get the Set method using pi.GetSetMethod().
And you might want to consider making your cache access thread-safe, as this is likely to be used server-side.
-
Good comment/idea regarding the cast|unboxing trick instead of my switch, in most cases that’ll probably work out better.
I can see that storing the MethodInfo’s might be more efficient, but I don’t really think it’ll change the results much as we cache the mapper function between calls, so it’ll only help us on the very first call in this domain.
You’re right again regarding the read only properties, but that’s where we move away from the proof-of-concept code that I’ve made and into the more serious realm, imho :)
A lock on the cachedMappers Dictionary, when creating new mapper functions, ought to take of the synchronization issues that might occur.
-
Good
Linq to SQL is the same thoery -
can you introduce the property lazy load about Linq to SQL ?
and how to imple the property load after invoke DbReader.Read()
.
thanks -
Rice,
Regarding lazy loading, please see the following URLs:
http://www.davidhayden.com/... http://www.thedatafarm.com/...
I’m now sure what you’re asking in regards to the DbReader.Read() method?Mark
-
thanks
I have resoved the first question via dbdatareader’s DbDataRecord.
the second lazy loading
how to implement the property access use the CustomerAttribute.
Nhibernate use the dynamic proxy , the property of instance must be virtual . the castle dynamic generate the entity proxy, when invoke the property ,the castle intercept the invoke ,then invoke the realy property.
but the linq to sql is not , i want to know the implement theory,similar to AOP.
thanks … -
Rice,
What is the CustomerAttribute? I’m not sure how LINQ implements lazy loading internally as I haven’t worked with it yet. I’ve only used LINQ with custom mapped POCOs that have no LINQ collection relations and thus are not able to lazy load.
Mark
-
Hi all
I am still learning however I can understand most of the code above. Nevertheless, I am having difficulties making it work.
Could anyone send me a working example with a database please?
Thanks
Rob -
Your current write-up offers verified beneficial to me. It’s quite
informative and you’re simply certainly extremely well-informed of this type. You get popped my own face to varying views on this topic together with intriguing, notable and reliable content material.