Wednesday, September 14, 2011

ICorProfilerCallBack and return to unmanaged code

As a .net developer if you just want your code to be run by the .net runtime and get your work done please don’t read this further.This is for the developers who really want to know what is happening inside the computer when they run a .net application. After reading you may think, why we need .net which is managed by somebody else and we need to code within it’s boundaries rather than exploring the system’s capabilities.So be careful.

Practical feeling v/s theories

From my childhood days itself, I have a habit of practically feeling things before confirming the theories.When my parents said there is shock so don’t touch the electric wire ,I didn’t believe.But realized when I touched myself.(Its 250volts in India).I am sure that ,this nature will be there with many people. The same problem appeared when I entered into the computer programming world. I started programming in my school days directly in the BASIC language where it was really hard for me to write program because I don’t know how this T.V is going to add 2 numbers which I am entering through keyboard. The teachers were simply saying computer will identify your command and process accordingly.But again how it knows how to manage carry and all.

All these questions were answered when I learned about the logical gates in digital electronics during the second year of diploma.Then I learned how these gates techniques can be leveraged to 8085 processor instructions which I executed in a small kit which has 7 segment display .There was no software simulators for 8085  available at that time for us.Third year I wrote and executed assembly instructions in an actual computer by compiling that into .exe.That put me in a stage to think about a compiler or interpreter which can understand a high level language and convert to machine instructions.The high level languages which was confortable for me was C,C++& VC++ in which we can easily plan how our code is going to execute in the physical machine. JAVA and .Net I learned as backups.

Things will not be always good for us.When I came out of the college the world needs .Net and Java programmers than traditional unmanaged coders. Since the bread and butter is important than the passion, I started with C# and .Net since it allows me to write C++ type language in my favorite development tool Visual Studio.But the same issue came there. How my code is being executed by the .Net?

As like any other .net fresher in India, my starting days were hard. So many things to do.No time to think whether its logical or not.That compels people to swallow the theories like how they believe in God. I had to believe that there is something called IL code to which we are compiling our source code which is machine independent. This helps us to write code in different languages such as VB.Net and C# in a single solution.It was really interesting for me.

Also theories said there is a garbage collector which will collect all the free memory so no need to worry about the memory management while coding. That’s great,but I got so many memory leak issues which I was not able to understand at that time. As a developer support engineer I was not supposed to fix the issues, only forward to the development team.

Another doubt was when we double click on the exe how the .net runtime takes control of the exe and manage execution of it.In Java when we run we specify Java <AppName>.class which clearly says we are executing Java.exe which is responsible for executing the IL code.

Things became clear when I tackled my developer support job. In free time googles I came to know there is a tool called ILDasm.exe in the .net framework which is present in my machine itself using which I can see the IL code.This made the concept of language interoperability strong in my mind.ie I realized myself that all our code which is written in VB or C# will compile into this same format.Or even we can write code in IL itself and compile using ILAsm. Then I thought about reverse engineering the IL to C# since its in a readable form which lead me to a nice tool called Reflector to reverse engineer the .Net code. This tool also showed me scenarios where the compiler optimize our code because some code which I wrote came in different form on reflection.

Again the season changed to busy. Microsoft introduced new technology called WPF for presentation purpose.As a control vendor my company also jumped into WPF where I was coordinating the development team and the release. As a .net engineer I had to learn one more technology keeping so many queries in mind.WPF introduced more queries into my mind like where is the WndProc function in WPF?,how the DP is working internally,who is invoking the PropertyChanged event in INotifyPropertyChanged,Is WPF really making use of my graphics card etc…But WPF helped me in one way which is nothing but to understand the working of GC. Since WPF comes with memory leaks if we use bindings ,I got opportunities to explore the .net process memory using another nice tool called WinDbg which showed me what are gen0 objects and when they goto  gen1 and gen2 also what is LOH and why its getting fragmented which leads to OutOfMemory exception.

But still so many queries are left.When a type is being initialized? if our assembly contains so many types will it decrease the performance due to loading time? How the .net runtime takes control when we double click on the .net exe? When a function is being JITted? When the GC starts its operation? etc… and most of the queries were answered when I work with Microsoft for my current company.

Working with Microsoft people

As I mentioned in some of the previous posts ,I got an excellent opportunity to work with Microsoft in their Charlotte,NC campus which leveraged my knowledge about the internals of .Net .Especially working with Wade Mascia who codes the DebugDiag was really wonderful.He introduced me one interface called ICorProfilerCallBack which has enough functions which will be called by the CLR when it performs its different operations.

Using ICorProfilerCallback in managed code

But fortunately or unfortunately we cannot implement ICorProfilerCallback in managed code such as C# or VB.net.I am looking for anyways to do so which I know not logical since writing profiler in managed code to profile managed code makes no sense.We are going to see when our method is going to be JITted and if we write the profiler in managed code how CLR can call the profiler callback method which needs JITting itself. That means we are back to unmanaged world.

Implementing ICorProfilerCallback

Its my continuous 6th year, I am writing  managed .net code.That made it difficult for me to write the VC++ code in a day.So as a normal developer, I googled for the code. There was not much luck except one article in CodeProject.It was simple and has basic functionalities of ICorProfileCallback interface like logging method calls.So I took that code and started scratching it to add my requirements.Hope scott_hackett will not mind, I am modifying his codeSmile.

When we implement the ICorProfilerCallback, we need to know about one more interface which is ICorProfilerInfo which is used as a helper when we deal with the profiler such as to get the method name from the ID and all.We will get an object of ICorProfilerInfo as parameter in the initialize method of ICorProfilerCallback interface.The Initialize method is called by CLR.

ICorProfilerCallback2 & ICorProfilerInfo2

These are the types introduced in .Net2.0 which contains some more functionalities. These are inherited interfaces of their counterparts in  old framework versions. So to get the new features cast the argument coming though Initialize method to ICorProfilerInfo2.

Steps to implement ICorProfilerCallback

For detailed description please see the original article.Below is a summary

  • Implement the interface ICorProfilerCallback /ICorProfilerCallback2 in a VC++ Com project as normal.
  • If you want your profiler to be recognized by the CLR and call the profiler methods,you need to set environment variables appropriately.
  • Simple implementation of ICorProfilerCallback will not give us the call back to our methods.For that we need to register which events we need using the SetEventMask method of ICorProfilerInfo object.Do that in the ICorProfilerCallback::Initialize itself.It accepts a DWORD and each bit controls one callback feature.
  • Now our methods in the implementation class will get called based on the mask we set except the MethodEnter and Leave.For that we need to hook separately using the ICorProfilerInfo::SetEnterLeaveFunctionHooks method.I don’t know why method execution callbacks need a separate way.

About the attached sample

The sample which I got from code project article doesn’t filter namespaces based on our needs. What I wanted was to filter how my application is getting JITed and the method calls.So I modified the ProfilerLauncher to accept namespace and passed that namespace to profiler through the environmental variable ‘NS_FILTER’. Environment variables are the only one way we can communicate to the profiler from outside world. Files are there, but I don’t want to use.

Also modified the profiler to log based on the namespace and added logging mask for the JITting events.As I told earlier its my second visit to C++ after 6 years.Things seems changed / I forgot C++ which made me write code like a fresher.So don’t expect a professional style coding in the sample.

Running the sample

Before running my modified sample I would recommend running the sample attached in the code project article which gives you an idea how many methods are being called when we run a simple win forms application. There are 3 files needed to run the sample.

  • DotnetProfiler.dll :This is the unmanaged profiler.
  • ProfilerLauncher.exe : This is the runner wrote in .Net which accepts target .net exe path and the namespace to filter.Then it create a process with required environment variables and starts execution of the target .net application.
  • Helloworld.exe : Just a .net application which we are going to be profiled.Put your application instead of this. Don’t forget to enter the correct namespace in the launcher screen.

This logs the profile information in a text file in the same folder.Download the sample from here.

No comments: