August 16, 2007

DynamicMethod, IL Code, CreateDelegate

In a previous post, dotNET - Under the Hood, IL Assembly, I briefly described how to write some IL Code to an Assembly, but what if at a later occasion you'll want to invoke these methods from the dll.
This could be achieved by loading the assembly in an AppDomain and then invoking the methods within ????
Or you could use what are known as DynamicMethods to invoke the methods.

DynamicMethod is a class that also resides in the System.Reflection.Emit namespace. Why would you want to use DynamicMethod? it's useful when you want to load methods into memory or write methods in memory and invoke later.
We can write IL code in memory in the exact same was as we wrote IL code earlier using the MethodBuilder.

In my experience DynamicMethod is a bit tricky to get right especially when you try to invoke your Dynamic method. The problem I found was with the constructor parameters, what do they mean;
There are 6 constructor overloads in all;
half of these are associated with an class instance i.e. you supply a Type as one of the constructor parameters, the significance of this is that it gives you access to reference private data within the type.
The other half of the constructor overloads are associated with the Module i.e. the dll and not a type, the significance of this is that it does not give you access to private types data.
from msdn:
DynamicMethod (String, Type, Type[], Module) Creates a dynamic method that is global to a module, specifying the method name, return type, parameter types, and module.
DynamicMethod (String, Type, Type[], Type) Creates a dynamic method, specifying the method name, return type, parameter types, and the type with which the dynamic method is logically associated.

There's also another caveat which is a boolean option to skip the JIT checking. This means some verification of the parameter types which is usually done by the JIT is skipped.
DynamicMethod (String, Type, Type[], Module, Boolean) Creates a dynamic method that is global to a module, specifying the method name, return type, parameter types, module, and whether just-in-time (JIT) visibility checks should be skipped for members of all types in the module.
DynamicMethod (String, Type, Type[], Type, Boolean) Creates a dynamic method, specifying the method name, return type, parameter types, the type with which the dynamic method is logically associated, and whether just-in-time (JIT) visibility checks should be skipped for members of other types in the module.

And there's one more caveat, CallingConvention,......

DynamicMethod dynamicMethod = new DynamicMethod(

"MyMethod"),
typeof(object),
new Type[] { typeof(object[]), typeof(object[]) },
typeof(MyClass));

The instance of DynamicMethod results in a method with the following signature
MyClass
{
object MyMethod(object[], object[])
{
}
}
You now have to insert your own IL code is the methods like Emit, again see my other post, dotNET - Under the Hood, IL Assembly, you write the IL code in the same manner.

Now you want to Invoke the method you've just created. This can be done in 2 ways, one call Invoke on your instance i.e.
dynamicMethod.Invoke();
or create a Delegate and then call Invoke on that delegate. The advantage of this over the first option is that you supply the correct parameters here and then just call Invoke on it later, the first option requires you to call Invoke here and now.
I'll explain the second option here:
DynamicMethod.CreateDelegate is a method used to invoke a piece of IL code. It creates a Delegate as the name suggests. The Delegate must have the identical signature to the DynamicMethod instance which is calling CreateDelegate.

public delegate object CompiledMethod(object[] arguments);

//As the delegate above defines, our DynamicMethod must return an object, and take an array of objects as parameters, it's also tied to the type "Example".
dynamicMethod = new DynamicMethod(
"",
typeof(object),
new Type[] { typeof(Example), typeof(object) },
typeof(Example)
);

// Get a FieldInfo for the private field 'id'.
FieldInfo dynamicMethodFid = typeof(Example).GetField(
"id",
BindingFlags.NonPublic | BindingFlags.Instance
);

ILGenerator dynamicMethodIlg = d.dynamicMethod.GetILGenerator();

dynamicMethodIlg.Emit(OpCodes.Ldarg_0);
dynamicMethodIlg.Emit(OpCodes.Ldfld, dynamicMethodFid);
dynamicMethodIlg.Emit(OpCodes.Ldarg_0);
dynamicMethodIlg.Emit(OpCodes.Ldarg_1);
dynamicMethodIlg.Emit(OpCodes.Stfld, dynamicMethodFid);
dynamicMethodIlg.Emit(OpCodes.Ret);

CompiledMethod result = (CompiledMethod)d.dynamicMethod.CreateDelegate(typeof(CompiledMethod), d.ex);

The CreateDelegate has 2 constructor overloads, one which takes and instance of an object as the second parameter (Instance) and another which does not (Static).
The difference being at invokation time, if you created your delegate using the instance method you do not have to supply the instance at invokation time.
If you created your delegate using the Static method you must supply the instance at invokation time e.g.

Instance
public delegate int UseLikeInstance(int newID);//delegate declaration,i.e. when your invoking the delegate you must supply a value only, no instance.
UseLikeInstance uli = (UseLikeInstance) d.changeID.CreateDelegate(typeof(UseLikeInstance, d.ex));//instantiation, instance required

uli(1492); //Invokation, no instance

Static
public delegate int UseLikeStatic(Example ex, int newID);//delegate declaration,i.e. when your invoking the delegate you must supply an Instance and a value.
UseLikeStatic uls = (UseLikeStatic) d.changeID.CreateDelegate(typeof(UseLikeStatic));//instantiation, NO instance required
uls(d.ex, 1492); //Invokation, instance required

The following I'm not too sure about!

The resultant delegate looks the same, it has a Method and a Target. The Method is the DynamicMethod you've just created and the Target is the instance you provided. When the Delegate is invoked if the IL code within the DynamicMethod requires an instance of an object in order to run then it looks in the Target. If at invokation time this Target is null or is of the wrong type an Exception will be thrown.
The Target is created when you use the CreateDelegate, at this time the instance must exist, you can later set this same instance to null before invokation of the delegate, this must mean that a copy of the instance is added to the Target at runtime and the original can be done away with.

The Target object (instance) is a point of confusion for me! when is it required and when is it not? It's when the DynamicMethod's IL uses instances that are already on the stack, this is a achieved with
Idlem.ref
this gets a reference to an object which resides on the stack.
This is where the Target comes it, it is the instance that gets used.
Then this instance is used for something such as a call to another external method.



The above is not yet complete

August 14, 2007

Enumerator(s)

You've all used the foreach loop in .NET to get the values in a container. The foreach actually uses that objects Enumerator.
In order to allow the use of the foreach on an object the class must implement the IEnumerable.GetEnumerator method which also means you have to implement the IEnumerable interface. Within the GetEnumerator override you need to return an instance of IEnumerator, so you need to create a class that implements the IEnumerator interface. see the code.

This is useful if you class contains a member variable which is an or even a container. Let's say it's an array. When an instance of your class is create the array is also instantiated. But you don't want the array to be visible but you do want clients to be able to use a foreach on your class instance, this can be achieved by implementing the IEnumerable interface and overriding the GetEnumerator method.


using System;
using System.Collections.Generic;
using System.Text;
using System.Collections;

namespace Enumerator
{


public class Person
{
public Person(string fName, string lName)
{
this.firstName = fName;
this.lastName = lName;
}

public string firstName;
public string lastName;
}
public class People : IEnumerable
{
private Person[] _people;
public People(Person[] pArray)
{
_people = new Person[pArray.Length];

for (int i = 0; i < position =" -1;" _people =" list;" position =" -1;" peoplearray =" new" peoplelist =" new">


System.Collections.IEnumerable

Yield

This is a keyword in .NET, it's placed before the return value where your implementation of IEnumerable.
When you use yield your actually replacing the requirement for an Enumerator, Yield acts as the Enumerator. So if your iterating through your own type (a class made by yourself) you's usually have to create an implemetation of the IEnumerator interface to tell the foreach for example how to move to the next item in the list i.e. MoveNext(). Now you can use the Yield keyword instead which saves you work.

The Yield actually returns a value from the collection and then goes back to the loop and gets the next value in the loop, remembering the context.

The Yield keyword can be used in your IEnumerable implementations GetEnumerator() method or in the foreach itself.
This can be done in the IEnumerable's GetEnumerator() method
e.g.
public IEnumerator GetEnumerator()
{
for(int i=0;i<_people.length;i++)>
yield return _people[i];
}
instead of
public IEnumerator GetEnumerator(){
return new PeopleEnum(_people);
}
which uses an implementation of IEnumerator, which in turn returns the array of _people, to return the enumerator

If you want to return an Enumerable list from a method i.e. you want to return a list of object which you can iterate through from a method you'll need to use the yield keyword.

The keyword yield is required when returning an IEnumerable object from a method like so


public static IEnumerable DoSomething()
{
int counter = 10;
int result = 1;
while (result <>

August 13, 2007

Security in Assemblies

Code Security is all about allowing and preventing code from running.

The .NET Security Model works by the assemblies each having their own Evidence embedded in the Assembly by the Assembly writer.
When the CLR loads the assembly it then reads and applies this Evidence to a Security model and this in turn returns Permissions, depending on what Permissions are returned will determine is the assembly is permitted to execute or not.
So it's Evidence in (on the assembly) -> Code Groups -> Permissions.

Code Access Security (CAS) is the the mechanism used by .NET to manage all of this. It's function is to process assemblies and determine the runtime permissions they should have, e.g. should they code within a certain assembly be allowed to run or not.

All assemblies have Evidence. As the CLR is loading the assembly it looks at this Evidence and processes it using the current machines .NET Code Group. Depending on where the assembly is being loaded from, the Evidence it has, the Code Group configuration on the current machine; will determine the Permissions that assembly gets.

Evidence: Evidence takes into account the assemblies strong name, publisher. There are 2 types of evidence Host Evidence and Assembly Evidence. Host Evidence is all about where the code is being loaded from, internet, local machine these are identified by URI, Site and Zones (Zones are for non internet)

Code Group: You can group permissions for assemblies i.e. all assemblies that have Internet permissions can do such and such. These are setup on the machine, when the CLR is processing the assemblies Evidence it tries to match the output with a Code Group.

Permissions: FullTrust, Internet. Have special meaning to the CLR, these restrict the access of code to resources, resources such as printers, applications etc.


August 08, 2007

dotNET - Under the Hood, IL Assembly, MSIL, ILGenerator, MethodBuilder

IL (Intermediate Language) is also referred to as MSIL (Microsoft Intermediate Language) or even ILAsm which is the same thing. IL is the code which your source code (C#, VB.NET etc) is compiled into. When you EXE or DLL is run these IL code is converted to a machine language particular to the present machine i.e. if the current machine being used to run the application is a Windows 2000 machine the IL will be converted into Windows instructions before running on the x86 processor (processor architecture used for Windows machines).

When you're developing with .NET languages with a Development tool such as Visual Studio 2005 the compiler you use will generate the correct/valid IL for you, but what if you want to create your own IL for some reason! One reason could be that you've developed your own Language and would like it to run on .NET, in this case you'll need to write out some sort of Assembly, be it EXE or DLL, with IL code. To do this you can use some Classes available in .NET API, these classes reside within the
System.Reflection.Emit namespace. I'll explain how this is achieved later.

There are also some tools which are used to compile and decompile the IL code, ILASM (all caps means this is the compiler and not the language ILAsm) is the IL compiler and ILDASM is the IL Decompiler (allows the reverse of compilation, you can create .NET language source code from an IL assembly).

An IL assembly contains 2 things, Metadata and Managed Code. Metadata is information which describes the structures and methods within the assembly. The Managed Code is the actual IL code, it is stored in the assembly in Binary form, managed means that the Runtime controls it.

The assembly has 2 main components, the metadata and the code. At runtime the assembly is loaded, the metadata is read first to find the descriptions of the structures and methods, the JIT compiles the IL code in the assembly into machine code using the metadata. When a method is required the machine code for that method is executed (incidentally this is what differentiates this from an Interpreter).

PE and COFF are additional data embedded in the assembly to describe an EXE, we'll not worry about them here.

Contents of the Assembly:
The assembly contains one or more Modules. An Assembly may or may not have multiple modules but it must have at least one i.e. the Prime Module. The Prime Module contains a
Metadata section which describes the contents of the Assembly, An Assembly Identity section and maybe some actual IL code. The Assembly may also contain additional Modules which each have their own Metadata and IL Code sections.


Boxed/Boxing and UnBoxed/UnBoxing:
You'll see these two terms appearing whenever you deal with IL, for you C++ developers you may have heard of the term before. It's related to Reference types and Value types, reference types are objects on the Heap to which a variable points, value types are not on the Heap and the variable contains the value in it's own memory location. This can be seen in the IL code. Reference types are more expensive, in terms of processor and memory, than Value types.

Boxing is the conversion of a value type to a reference type at compile time i.e. the value is contained in a variables memory location, for some reason (such as passing the value by reference, maybe as a method parameter) you want to convert your value type to a reference type. This means that a bit copy is made of the value type and instance of a Class is created with that copied value.

UnBoxing in the conversion of reference type to value type at compile time i.e. in your source code a variable points to a memory address on the Heap, you new-up an Object, when you compile the object is Boxed meaning the IL code is written for that Object, the memory required is allocated in IL code.
Int32 unBoxed = 20;//Unboxed
Object boxed = unBoxed;//Boxed
Int32 unBoxedBoxed = (Int32)boxed;

Here's the IL code generated for the C# source code above
.entrypoint
// Code size 19 (0x13)
.maxstack 1
.locals init ([0] int32 unBoxed,
[1] object boxed,
[2] int32 unBoxedBoxed)
IL_0000: nop
IL_0001: ldc.i4.s 20
IL_0003: stloc.0
IL_0004: ldloc.0
IL_0005: box [mscorlib]System.Int32
IL_000a: stloc.1
IL_000b: ldloc.1
IL_000c: unbox.any [mscorlib]System.Int32
IL_0011: stloc.2
IL_0012: ret
Here's a great article on Boxing from msdn magazine.


When playing around with IL code and reflection you may want to invoke a method within an assembly. If you are writing the assembly you may want to write it using IL code, to do this you would use the MethodInfo class found in the System.Reflection.Emit namespace. Once you'd created your methodinfo you can write IL code directly to that method and then save all to a dll later using the ILGenerator class, this ILGenerator can be thought of as the writer of the IL code and a reference to this is gotten from the MethodInfo instance you've just created
// create a dynamic assembly and module
AssemblyName assemblyName = new AssemblyName();
assemblyName.Name = "HelloWorld";
AssemblyBuilder assemblyBuilder = Thread.GetDomain().DefineDynamicAssembly(
assemblyName, AssemblyBuilderAccess.RunAndSave);

ModuleBuilder module;
module = assemblyBuilder.DefineDynamicModule("HelloWorld.dll");

// create a new type to hold our Main method
TypeBuilder typeBuilder = module.DefineType(
typeof(Example).Name,
TypeAttributes.Public TypeAttributes.Class);




// create the Main(string[] args) method
MethodBuilder methodbuilder = typeBuilder.DefineMethod(
DynamicMethod.GetCurrentMethod().Name,
DynamicMethod.GetCurrentMethod().Attributes,
DynamicMethod.GetCurrentMethod().CallingConvention,
changeID.ReturnType,
new Type[] { typeof(Example), typeof(int) });

Type t = changeID.DeclaringType;
// generate the IL for the Main method
ILGenerator ilGenerator = methodbuilder.GetILGenerator();


// Push the current value of the id field onto the
// evaluation stack. It's an instance field, so load the
// instance of Example before accessing the field.
ilGenerator.Emit(OpCodes.Ldarg_0);
ilGenerator.Emit(OpCodes.Ldfld, fid);

// Load the instance of Example again, load the new value
// of id, and store the new field value.
ilGenerator.Emit(OpCodes.Ldarg_0);
ilGenerator.Emit(OpCodes.Ldarg_1);
ilGenerator.Emit(OpCodes.Stfld, fid);


// The original value of the id field is now the only
// thing on the stack, so return from the call.
ilGenerator.Emit(OpCodes.Ret);

// bake it
Type helloWorldType = typeBuilder.CreateType();
assemblyBuilder.Save("HelloWorld.dll");


A useful reference for deeper understanding of how .NET works under the hood is the Microsoft opensource .NET project "Shared Source Common Language Infrastructure" codenamed "Rotor", it's an opensource version of .NET and can be used for discovering more about .NET code, there's also a book and it's available from googledocs, see my bookmarks.