13 December 2011
Another area of this migration proof-of-concept work I'm doing at the day job involves investigating the best way to swap out a load of COM components for C# versions over time. The plan initially is to define interfaces for them and code against those interfaces, write wrapper classes around the components that implement these interfaces and one day rewrite them one-by-one.
Writing the interfaces is valuable since it enables some documentation-through-comments to be generated for each method and property and forces me to look into the idiosyncracies of the various components.
However, writing endless wrappers for the components to "join" them to the interfaces sounded boring! Even if I used the .Net 4.0 "dynamic" keyword it seemed like there'd be a lot of repetition and opportunity for me to mistype a property name and not realise until debugging / writing tests. (Plus I found a problem that prevented me from using "dynamic" with the WSCs I was wrapping - see the bottom of this post for more details).
I figured this is the sort of thing that should be dynamically generatable from the interfaces instead of writing them all by hand - something like how Moq can create generate Mock<ITest> implementations. I did most of this investigation back in Summer, not long after the AutoMapper work I was looking into, and had hoped I'd be able to leverage my sharpened Linq Expression dynamic code generation skills. Alas, it seems that new classes can not be defined in this manner so I had to go deeper..
I was aware that IL could be generated by code at runtime and executed as any other other loaded assembly might be. I'd read (and chuckled at) this article in the past but never taken it any further: Dynamic... But Fast: The Tale of Three Monkeys, A Wolf and the DynamicMethod and ILGenerator Classes
As I tried to find out more information, though, it seemed that a lot of articles would make the point that you could find out how to construct IL by using the IL Disassembler that's part of the .Net SDK: ildasm.exe (located in C:\Program Files\Microsoft SDKs\Windows\v7.0A\bin on my computer). This makes sense because once you start constructing simple classes and examining the generated code in ildasm you can start to get a reasonable idea for how to write the generation code yourself. But it still took me quite a while to get to the point where the following worked!
What I really wanted was something to take, for example:
public interface ITest
{
int GetValue(string id);
string Name { get; set; }
}
and wrap an object that had that method and property such that the interface was exposed - eg.
public class TestWrapper : ITest
{
private object _src;
public TestWrapper(object src)
{
if (src == null)
throw new ArgumentNullException("src");
_src = src;
}
public int GetValue(string id)
{
return _src.GetType().InvokeMember(
"GetValue",
BindingFlags.InvokeMethod,
null,
_src,
new object[] { id }
);
}
public string Name
{
get
{
return _src.GetType().InvokeMember("Name", BindingFlags.GetProperty, null, _src, null)
}
set
{
_src.GetType().InvokeMember("Name", BindingFlags.SetProperty, null, _src, new object[] { value });
}
}
}
It may seem like using reflection will result in there being overhead in the calls but the primary objective was to wrap a load of WSCs in C# interfaces so they could be rewritten later while doing the job for now - so performance wasn't really a massive concern at this point.
The first thing to be aware of is that we can't create new classes in the current assembly, we'll have to create them in a new one. So we start off with
var assemblyBuilder = Thread.GetDomain().DefineDynamicAssembly(
new AssemblyName("DynamicAssembly"), // This is not a magic string, it can be called anything
AssemblyBuilderAccess.Run
);
var moduleBuilder = assemblyBuilder.DefineDynamicModule(
assemblyBuilder.GetName().Name,
false
);
// This NewGuid call is just to get a unique name for the new construct
var typeName = "InterfaceApplier" + Guid.NewGuid().ToString();
var typeBuilder = moduleBuilder.DefineType(
typeName,
TypeAttributes.Public
| TypeAttributes.Class
| TypeAttributes.AutoClass
| TypeAttributes.AnsiClass
| TypeAttributes.BeforeFieldInit
| TypeAttributes.AutoLayout,
typeof(object),
new Type[] { typeof(ITest) }
);
The TypeAttribute values I copied from the ildasm output I examined.
Note that we're specifying ITest as the interface we're implementing by passing it as the "interfaces" parameter to the moduleBuilder's DefineType method.
The constructor is fairly straight forward. The thing that took me longest to wrap my head around was how to form the "if (src == null) throw new ArgumentNullException()" construct. If seems that this is most easily done by declaring a label to jump to if src is not null which allows execution to leap over the point at which an ArgumentNullException will be raised.
// Declare private _src field
var srcField = typeBuilder.DefineField("_src", typeof(object), FieldAttributes.Private);
var ctorBuilder = typeBuilder.DefineConstructor(
MethodAttributes.Public,
CallingConventions.Standard,
new[] { typeof(object) }
);
// Generate: base.ctor()
var ilCtor = ctorBuilder.GetILGenerator();
ilCtor.Emit(OpCodes.Ldarg_0);
ilCtor.Emit(OpCodes.Call, typeBuilder.BaseType.GetConstructor(Type.EmptyTypes));
// Generate: if (src != null), don't throw new ArgumentException("src")
var nonNullSrcArgumentLabel = ilCtor.DefineLabel();
ilCtor.Emit(OpCodes.Ldarg_1);
ilCtor.Emit(OpCodes.Brtrue, nonNullSrcArgumentLabel);
ilCtor.Emit(OpCodes.Ldstr, "src");
ilCtor.Emit(OpCodes.Newobj, typeof(ArgumentNullException).GetConstructor(new[] { typeof(string) }));
ilCtor.Emit(OpCodes.Throw);
ilCtor.MarkLabel(nonNullSrcArgumentLabel);
// Generate: _src = src
ilCtor.Emit(OpCodes.Ldarg_0);
ilCtor.Emit(OpCodes.Ldarg_1);
ilCtor.Emit(OpCodes.Stfld, srcField);
// All done!
ilCtor.Emit(OpCodes.Ret);
Although there's only a single property in the ITest example we're looking at, we might as look ahead and loop over all properties the interface has so we can apply the same sort of code to other interfaces. Since we are dealing with interfaces, we only need to consider whether a property is gettable, settable or both - there's no public / internal / protected / private / etc.. to worry about. Likewise, we only have to worry about properties and methods - interfaces can't declare fields.
foreach (var property in typeof(ITest).GetProperties())
{
var methodInfoInvokeMember = typeof(Type).GetMethod(
"InvokeMember",
new[]
{
typeof(string),
typeof(BindingFlags),
typeof(Binder),
typeof(object),
typeof(object[])
}
);
// Prepare the property we'll add get and/or set accessors to
var propBuilder = typeBuilder.DefineProperty(
property.Name,
PropertyAttributes.None,
property.PropertyType,
Type.EmptyTypes
);
// Define get method, if required
if (property.CanRead)
{
var getFuncBuilder = typeBuilder.DefineMethod(
"get_" + property.Name,
MethodAttributes.Public
| MethodAttributes.HideBySig
| MethodAttributes.NewSlot
| MethodAttributes.SpecialName
| MethodAttributes.Virtual
| MethodAttributes.Final,
property.PropertyType,
Type.EmptyTypes
);
// Generate:
// return _src.GetType().InvokeMember(property.Name, BindingFlags.GetProperty, null, _src, null)
var ilGetFunc = getFuncBuilder.GetILGenerator();
ilGetFunc.Emit(OpCodes.Ldarg_0);
ilGetFunc.Emit(OpCodes.Ldfld, srcField);
ilGetFunc.Emit(OpCodes.Callvirt, typeof(Type).GetMethod("GetType", Type.EmptyTypes));
ilGetFunc.Emit(OpCodes.Ldstr, property.Name);
ilGetFunc.Emit(OpCodes.Ldc_I4, (int)BindingFlags.GetProperty);
ilGetFunc.Emit(OpCodes.Ldnull);
ilGetFunc.Emit(OpCodes.Ldarg_0);
ilGetFunc.Emit(OpCodes.Ldfld, srcField);
ilGetFunc.Emit(OpCodes.Ldnull);
ilGetFunc.Emit(OpCodes.Callvirt, methodInfoInvokeMember);
if (property.PropertyType.IsValueType)
ilGetFunc.Emit(OpCodes.Unbox_Any, property.PropertyType);
ilGetFunc.Emit(OpCodes.Ret);
propBuilder.SetGetMethod(getFuncBuilder);
}
// Define set method, if required
if (property.CanWrite)
{
var setFuncBuilder = typeBuilder.DefineMethod(
"set_" + property.Name,
MethodAttributes.Public
| MethodAttributes.HideBySig
| MethodAttributes.SpecialName
| MethodAttributes.Virtual,
null,
new Type[] { property.PropertyType }
);
var valueParameter = setFuncBuilder.DefineParameter(1, ParameterAttributes.None, "value");
var ilSetFunc = setFuncBuilder.GetILGenerator();
// Generate:
// _src.GetType().InvokeMember(
// property.Name, BindingFlags.SetProperty, null, _src, new object[1] { value }
// );
// Note: Need to declare assignment of local array to pass to InvokeMember (argValues)
var argValues = ilSetFunc.DeclareLocal(typeof(object[]));
ilSetFunc.Emit(OpCodes.Ldarg_0);
ilSetFunc.Emit(OpCodes.Ldfld, srcField);
ilSetFunc.Emit(OpCodes.Callvirt, typeof(Type).GetMethod("GetType", Type.EmptyTypes));
ilSetFunc.Emit(OpCodes.Ldstr, property.Name);
ilSetFunc.Emit(OpCodes.Ldc_I4, (int)BindingFlags.SetProperty);
ilSetFunc.Emit(OpCodes.Ldnull);
ilSetFunc.Emit(OpCodes.Ldarg_0);
ilSetFunc.Emit(OpCodes.Ldfld, srcField);
ilSetFunc.Emit(OpCodes.Ldc_I4_1);
ilSetFunc.Emit(OpCodes.Newarr, typeof(Object));
ilSetFunc.Emit(OpCodes.Stloc_0);
ilSetFunc.Emit(OpCodes.Ldloc_0);
ilSetFunc.Emit(OpCodes.Ldc_I4_0);
ilSetFunc.Emit(OpCodes.Ldarg_1);
if (property.PropertyType.IsValueType)
ilSetFunc.Emit(OpCodes.Box, property.PropertyType);
ilSetFunc.Emit(OpCodes.Stelem_Ref);
ilSetFunc.Emit(OpCodes.Ldloc_0);
ilSetFunc.Emit(OpCodes.Callvirt, methodInfoInvokeMember);
ilSetFunc.Emit(OpCodes.Pop);
ilSetFunc.Emit(OpCodes.Ret);
propBuilder.SetSetMethod(setFuncBuilder);
}
}
The gist is that for the getter and/or setter, we have to declare a method and then assign that method to be the GetMethod or SetMethod for the property. The method is named by prefixing the property name with either "get_" or "set_", as is consistent with how C# generates it class properties' IL.
The call to
_src.GetType().InvokeMember(
property.Name,
BindingFlags.SetProperty,
null,
_src,
new object[] { value }
);
is a bit painful as we have to declare an array with a single element to pass to the method, where that single element is the "value" reference available within the setter.
Also worthy of note is that when returning a ValueType or setting a ValueType. As we're expecting to either return an object or set an object, the value has to be "boxed" otherwise bad things will happen!
Like the TypeAttributes in the Constructor, the MethodAttributes I've applied here were gleaned from looking at IL generated by Visual Studio.
We're on the home stretch now! Methods are very similar to the property setters except that we may have zero, one or multiple parameters to handle and we may or may not (if the return type is void) return a value from the method.
foreach (var method in typeof(ITest).GetMethods())
{
var parameters = method.GetParameters();
var parameterTypes = new List<Type>();
foreach (var parameter in parameters)
{
if (parameter.IsOut)
throw new ArgumentException("Output parameters are not supported");
if (parameter.IsOptional)
throw new ArgumentException("Optional parameters are not supported");
if (parameter.ParameterType.IsByRef)
throw new ArgumentException("Ref parameters are not supported");
parameterTypes.Add(parameter.ParameterType);
}
var funcBuilder = typeBuilder.DefineMethod(
method.Name,
MethodAttributes.Public
| MethodAttributes.HideBySig
| MethodAttributes.NewSlot
| MethodAttributes.Virtual
| MethodAttributes.Final,
method.ReturnType,
parameterTypes.ToArray()
);
var ilFunc = funcBuilder.GetILGenerator();
// Generate: object[] args
var argValues = ilFunc.DeclareLocal(typeof(object[]));
// Generate: args = new object[x]
ilFunc.Emit(OpCodes.Ldc_I4, parameters.Length);
ilFunc.Emit(OpCodes.Newarr, typeof(Object));
ilFunc.Emit(OpCodes.Stloc_0);
for (var index = 0; index < parameters.Length; index++)
{
// Generate: args[n] = ..;
var parameter = parameters[index];
ilFunc.Emit(OpCodes.Ldloc_0);
ilFunc.Emit(OpCodes.Ldc_I4, index);
ilFunc.Emit(OpCodes.Ldarg, index + 1);
if (parameter.ParameterType.IsValueType)
ilFunc.Emit(OpCodes.Box, parameter.ParameterType);
ilFunc.Emit(OpCodes.Stelem_Ref);
}
var methodInfoInvokeMember = typeof(Type).GetMethod(
"InvokeMember",
new[]
{
typeof(string),
typeof(BindingFlags),
typeof(Binder),
typeof(object),
typeof(object[])
}
);
// Generate:
// [return] _src.GetType().InvokeMember(method.Name, BindingFlags.InvokeMethod, null, _src, args);
ilFunc.Emit(OpCodes.Ldarg_0);
ilFunc.Emit(OpCodes.Ldfld, srcField);
ilFunc.Emit(OpCodes.Callvirt, typeof(Type).GetMethod("GetType", Type.EmptyTypes));
ilFunc.Emit(OpCodes.Ldstr, method.Name);
ilFunc.Emit(OpCodes.Ldc_I4, (int)BindingFlags.InvokeMethod);
ilFunc.Emit(OpCodes.Ldnull);
ilFunc.Emit(OpCodes.Ldarg_0);
ilFunc.Emit(OpCodes.Ldfld, srcField);
ilFunc.Emit(OpCodes.Ldloc_0);
ilFunc.Emit(OpCodes.Callvirt, methodInfoInvokeMember);
if (method.ReturnType.Equals(typeof(void)))
ilFunc.Emit(OpCodes.Pop);
else if (method.ReturnType.IsValueType)
ilFunc.Emit(OpCodes.Unbox_Any, method.ReturnType);
ilFunc.Emit(OpCodes.Ret);
}
The boxing of ValueTypes when passed as parameters or returned from the method is required, just like the property accessors.
The only real point of interest here is the array generation for the parameters - this also took me a little while to wrap my head around! You may note that I've been a bit lazy and not supported optional, out or ref parameters - I didn't need these for anything I was working on and didn't feel like diving into it at this point. I'm fairly sure that if they become important features then whipping out ildasm and looking at the generated code there will reveal the best way to proceed with these.
Now that we've defined everything about the class, we can instantiate it!
var wrapper = (ITest)Activator.CreateInstance(
typeBuilder.CreateType(),
src
);
This gives us back an instance of a magic new class that wraps a specified "src" and passes through the ITest properties and methods! If it's applied to an object that doesn't have the required properties and methods then exceptions will be thrown when they are called - the standard exceptions that reflection calls to invalid properties/methods would result in.
But I think that's pretty much enough for this installment - it's been a bit dry but I think it's been worthwhile!
There are a lot of extension points that naturally arise from this rough-from-the-edges code - the first things that spring to my mind are a way to wrap this up nicely into a generic class that could create wrappers for any given interface, a way to handle interface inheritance, a way to possibly wrap the returned values - eg. if we have
public interface ITest
{
IEmployee Get(int id);
}
public interface IEmployee
{
int Id { get; }
string Name { get; }
}
can the method that applies ITest to an object also apply IEmployee to the value returned from ITest.Get if that value itself doesn't already implement IEmployee??
Finally, off the top of my head, if I'm using these generated classes to read interact with WSCs / COM components, am I going to need to pass references over to COM components? If so, I'm going to have to find a way to flag them as ComVisible.
But these are issues to address another time :)
The code here doesn't use the .Net 4.0 "dynamic" keyword and so will compile under .Net 2.0. I had a bit of a poke around in the IL generated that makes use of dynamic since in some situations it should offer benefits - often performance is one such benefit! However, in the particularly niche scenario I'm working with it refuses to work :( Most of the components I'm wrapping are legacy VBScript WSCs and trying to set properties on these seems to fail when using dynamic.
I've pulled this example from a test (in xUnit) I wrote to illustrate that there were issues ..
[Fact]
public void SettingCachePropertyThrowsRuntimeBinderException()
{
// The only way to demonstrate properly is unfortunately by loading an actual wsc - if we used a
// standard .Net class as the source then it would work fine
var src = Microsoft.VisualBasic.Interaction.GetObject(
Path.Combine(
new FileInfo(this.GetType().Assembly.FullName).DirectoryName,
"TestSrc.wsc"
)
);
var srcWithInterface = new ControlInterfaceApplierUsingDynamic(src);
// Expect a RuntimeBinderException with message "'System.__ComObject' does not contain a definition
// for 'Cache'"
Assert.Throws<RuntimeBinderException>(() =>
{
srcWithInterface.Cache = new NullCache();
});
}
public class ControlInterfaceApplierUsingDynamic : IControl
{
private dynamic _src;
public ControlInterfaceApplierUsingDynamic(object src)
{
if (src == null)
throw new ArgumentNullException("src");
_src = src;
}
public ICache Cache
{
set
{
_src.Cache = value;
}
}
}
[ComVisible(true)]
private class NullCache : ICache
{
public object this[string key] { get { return null; } }
public void Add(string key, object value, int cacheDurationInSeconds) { }
public bool Exists(string key) { return false; }
public void Remove(string key) { }
}
The WSC content is as follows:
<?xml version="1.0" ?>
<?component error="false" debug="false" ?>
<package>
<component id="TestSrc">
<registration progid="TestSrc" description="Test Control" version="1" />
<public>
<property name="Config" />
</public>
<script language="VBScript">
<![CDATA[
Public Cache
]]>
</script>
</component>
</package>
I've not been able to get to the bottom of why this fails and it's not exactly a common problem - most people left WSCs back in the depths of time.. along with VBScript! But for the meantime we're stuck with them since the thought of trying to migrate the entire codebase fills me with dread, at least splitting it this way means we can move over piecemeal and re-write isolated components of the code at a time. And then one day the old cruft will have gone! And people will consider the earlier migration code the new "old cruft" and the great cycle can continue!
Update (2nd May 2014): It turns out that if
var src = Microsoft.VisualBasic.Interaction.GetObject(
Path.Combine(
new FileInfo(this.GetType().Assembly.FullName).DirectoryName,
"TestSrc.wsc"
)
);
is altered to read
var src = Microsoft.VisualBasic.Interaction.GetObject(
"script:"
Path.Combine(
new FileInfo(this.GetType().Assembly.FullName).DirectoryName,
"TestSrc.wsc"
)
);
then this test will pass! I'll leave the content here for posterity but it struck me while flipping through this old post that I had seen and addressed this problem since writing this.
Posted at 22:01
Dan is a big geek who likes making stuff with computers! He can be quite outspoken so clearly needs a blog :)
In the last few minutes he seems to have taken to referring to himself in the third person. He's quite enjoying it.