Improving performance using page guards

The problems we’re facing today is, a little bit, unique. Given:

  • n contiguous arrays
  • Each array has m cells
  • Each cell is a Boolean flag

We receive a stream of signals, each signal is an absolute offset from the first array. For each signal we need to set the correct flag AND the first flag of the array. The motivation for setting the first flag is to enable quick filtering of arrays having some flags set.
For example, we have a usage tracking system for n websites and m users. If user i visited website j we’d like to signal that by setting the ith flag in the jth array. After some time, we’d like to query which sites had any visit and who visited them.

The intuitive solution

Assuming you don’t care too much for the performance the solution is straight forward. Whenever setting a flag in an array set also the array in offset 0. If the input is index, then the array index is index / m and the item index is index % m. Pretty simple. For simplicity the source of indexes will be an array named items and the address of the first array will be baseAddress:

for (int i = 0; i < numOfItems; ++i)
{
char* hitAddress = baseAddress + items[i];
*
hitAddress = 1;
char* blockStartAddress = hitAddress - (hitAddress - baseAddress) % dwPageSize;
*
blockStartAddress = 1;
}

It is clear that the first action, *hitAddress = 1, is impossible to avoid. But, what about the set of the signal at index 0? We can replace it with a condition but it is clear it won’t affect much the performance. So, how can we improve that part?

Enabling page guards

Windows provides several memory protections, one of them is the page guard. When allocating a new memory scope we can declare it as protected. Defining it as protected means that each page (page is an arbitrary partition of the memory based on OS page size) will throw an exception on the first access to it. After throwing the exception the protection is removed. We would like to use this mechanism to avoid re-setting the flag at index 0.

In order to define such a scope, we will use the VirtualAlloc method:

VirtualAlloc(NULL, TOTAL_SIZE,
MEM_RESERVE | MEM_COMMIT
PAGE_READWRITE | PAGE_GUARD)

It returns a pointer to the memory scope with size of TOTAL_SIZE in bytes. If a page size P then the new scope has TOTAL_SIZE / P pages.

Tracking page hit

As mentioned, at the first time the memory inside a page is accessed an exception is being thrown. We would like to catch it. In order to do so in the fastest way, we will use windows SetUnhandledExceptionFilter API. The filter is a simple method receiving the exception information and deciding how to treat it. Treating it has three options:

  1. Handling it
  2. Handling it and continue the code execution
  3. Pass the decision to other handler

As a simple filter we can request the runtime to ignore all page guards exceptions:

LONG WINAPI SmartFilter(_EXCEPTION_POINTERS *ep)
{
if (ep->ExceptionRecord->ExceptionCode != STATUS_GUARD_PAGE_VIOLATION)
{
return EXCEPTION_CONTINUE_SEARCH;
}

return EXCEPTION_CONTINUE_EXECUTION;
}

So after setting it as the filter all page guards exceptions will be ignored:

SetUnhandledExceptionFilter(&SmartFilter);

Extending the exceptions filter logic

Let’s assume that all arrays are smaller than the opration system page size and we’ll assume that we don’t care about reserving extra space to pad each array. We’ll denote the page size with dwPageSize.
Now, we can make our SmartFilter really smart. We will add to it the logic for setting the first flag on each array. Assuming baseAddress is some global variable:

LONG WINAPI SmartFilter(_EXCEPTION_POINTERS *ep)
{
if (ep->ExceptionRecord->ExceptionCode != STATUS_GUARD_PAGE_VIOLATION)
{
return EXCEPTION_CONTINUE_SEARCH;
}

char* hitAddress = (char*)ep->ExceptionRecord->ExceptionInformation[1];
char* blockStartAddress = hitAddress - (hitAddress - baseAddress) % dwPageSize;

*
blockStartAddress = 1;

return EXCEPTION_CONTINUE_EXECUTION;
}

We extract the exact address being touched by accessing ep->ExceptionRecord->ExceptionInformation[1]. Through it it’s easy to get the start address of the page. When having this filter method registered we can be sure that whenever we set a flag in the array the first flag will be set too.
Now, we can alter the original code which was in charge of setting the first signal whenever a flag was set:

for (int i = 0; i < numOfItems; ++i)
{
char* hitAddress = baseAddress + items[i];
*
hitAddress = 1;
}

Comparing the results

In order to make our comparison interesting let’s assume that we have 10000 arrays (websites in the tracking system) and each array has 25000 flags (users for example). In order to make it intense we’ll assume that during a short period 10% of the arrays were visited, for exmaple having 250000000 signals sent through the stream (repeating actions in a website by same users are allowed). On average, the time it took to run:

Seconds
Straight forward 1.744
Page guards 0.78

As can easily seen, the page guards solution saves ~50% of the runtime.

Conclusion

The operating system provides a few very fast facilities which can be exploited. Even though most of those facilities are designed for different purposes they can still be useful in different cases, like the one here requiring single time signal for a scope. Since Windows puts a lot of focus in being backward compatible, those exploitations are not too risky. As usual – if it doesn’t require performance optimization, don’t do it. The price of maintaining the code might not worth it.

Advertisements

Conditional attribute and arguments evaluation

What is the conditional attribute?

The conditional attribute enables including/omitting methods calls during compilation depending on compilation symbols. For example, we can condition that a specific log method calls will be included only when we compile in debug. The compiler in this case will omit the calls to the method. Looking at the next code:

public class Logger
{
[
Conditional("DEBUG")]
public void LogDebugMessage(string str)
{

}
}

And the code calling it:

class MyClass
{
private readonly Logger logger = new Logger();

public void Foo()
{
logger
.LogDebugMessage("Foo");
}
}

We expect the compiler to omit the body of Foo(). As we can see with a disassembler this is exactly what happens:

.method public hidebysig instance void  Foo() cil managed
{
// Code size 1 (0x1)
.maxstack 8
IL_0000: ret
} // end of method MyClass::Foo

How method arguments are treated?

Temp variable assignment optimization

Regardless the conditional attribute, in release mode the compiler performs many optimization which one of them is skipping local variable assignment. You’re most likely to notice it when you assign a value into a local variable and pass it to a method as an argument (while this is the only variable usage). For example:

public void Foo()
{
var foo = "Foo";
logger
.LogDebugMessage(foo);
}

Translates into:

0527C028  mov         edx,dword ptr ds:[2EE78B0h]  	// Load the string address
0527C02E mov ecx,dword ptr [ecx+4] // Load the logger instance
0527C031 cmp dword ptr [ecx],ecx // Null check
0527C033 call dword ptr ds:[50C5BD8h] // Call LogDebugMessage
0527C039 ret

While:

public void Foo()
{
logger
.LogDebugMessage("Foo");
}

Translates into:

0552C028  mov         ecx,dword ptr [ecx+4]  		// Load the logger instance
0552C02B mov edx,dword ptr ds:[31478B0h] // Load the string address
0552C031 cmp dword ptr [ecx],ecx // Null check
0552C033 call dword ptr ds:[5375C30h] // Call LogDebugMessage
0552C039 ret

Which are basically the same. So in case we’re not using the conditional attribute we shouldn’t care about local assignment. We can expect to have no difference in runtime.

Temp variable sent to omitted call optimization?

So an interesting question is what happens to an argument we’re about to send to a conditional method? If call to LogDebugMessage are omitted, what should we expect in this case:

public void Foo()
{
var method = MethodBase.GetCurrentMethod().Name;
logger
.LogDebugMessage(method);
}

And in this case:

public void Foo()
{
logger
.LogDebugMessage(MethodBase.GetCurrentMethod().Name);
}

The answer can be easily found by looking at the methods IL. The first version with temp assignment to a variable compiles into:

.method public hidebysig instance void  Foo() cil managed
{
// Code size 12 (0xc)
.maxstack 8
IL_0000: call class [mscorlib]System.Reflection.MethodBase [mscorlib]System.Reflection.MethodBase::GetCurrentMethod()
IL_0005: callvirt instance string [mscorlib]System.Reflection.MemberInfo::get_Name()
IL_000a: pop
IL_000b: ret
} // end of method MyClass::Foo

While the second version compiles into:

.method public hidebysig instance void  Foo() cil managed
{
// Code size 1 (0x1)
.maxstack 8
IL_0000: ret
} // end of method MyClass::Foo

As we can see, in this case the argument was not even evaluated and the whole statement was omitted from the IL. Meaning that in this case, inlining the variable would have influence on the performance. It didn’t happen by chance, this is the defined behavior of the compiler as stated in the Conditional attribute documentation:

“If the symbol is defined, the call is included; otherwise, the call (including evaluation of the parameters of the call) is omitted.”

Conclusion

The most common scenario in which the conditional attribute is involved is logging. Since the main advantage of omitting the logs is usually to avoid performance hit in production it is important to take into consideration the price of evaluating the arguments values. The simplest solution is to inline the variable. This can be done easily when the argument is string.Format() or similar. In case it is more complicated or unreadable it can always be solved by preprocessor directive such as #if.

Field initializers and base constructors order

In this post we’ll explore what happens behind the scenes when we place code in field initializer compared to constructor body.

Field initializer and constructor

We’ll start from a simple example:

public class MyClass
{
private static int index = 1;

public int x = index++;
public int y;

public MyClass()
{
y = index++;
}
}

The result here is very intuitive, this code:

var instance = new MyClass();
Console.WriteLine("x = " + instance.x);
Console.WriteLine("y = " + instance.y);

Will output:
image

So far there’s no surprise, the field initializers code is performed prior to code of the explicit constructor.

Field initializer and base constructor

Let’s observe the next code:

public class Base
{
protected static int index = 1;

public int w = index++;
public int x;

public Base()
{
x = index++;
}
}

public class Derived : Base
{
public int y = index++;
public int z;

public Derived()
{
z = index++;
}
}

There are two outputs which one may expect:

  • The base class fields will be initialized first and the derived ones will be initialized afterwards.
  • The field y, which has a field initializer will be initialized first, then the base fields and lastly the derived constructor will be called.

What actually happens?

var instance = new Derived();
Console.WriteLine("Base.w = " + instance.w);
Console.WriteLine("Base.x = " + instance.x);
Console.WriteLine("Derived.y = " + instance.y);
Console.WriteLine("Derived.z = " + instance.z);

Outputs:
image

We can clearly see that the derived field initializer is called first, prior to the base constructor.

Why it happens?

Looking at the disassembled IL we can see that the field initializers code is placed at the beginning of the constructor code, prior to the base constructor call:

.method public hidebysig specialname rtspecialname 
instance void .ctor() cil managed
{
// Code size 48 (0x30)
.maxstack 8
IL_0000: ldarg.0
IL_0001: ldsfld int32 ConsoleApplication1.Base::index
IL_0006: dup
IL_0007: ldc.i4.1
IL_0008: add
IL_0009: stsfld int32 ConsoleApplication1.Base::index
IL_000e: stfld int32 ConsoleApplication1.Derived::y
IL_0013: ldarg.0
IL_0014: call instance void ConsoleApplication1.Base::.ctor()
IL_0019: nop
IL_001a: nop
IL_001b: ldarg.0
IL_001c: ldsfld int32 ConsoleApplication1.Base::index
IL_0021: dup
IL_0022: ldc.i4.1
IL_0023: add
IL_0024: stsfld int32 ConsoleApplication1.Base::index
IL_0029: stfld int32 ConsoleApplication1.Derived::z
IL_002e: nop
IL_002f: ret
} // end of method Derived::.ctor

We can see, obviously, that the compiler is protecting us from doing silly mistakes like accessing base fields before they’re initialized:

public class Base
{
protected int baseField = 100;
}

public class Derived : Base
{
public int x = baseField;
}

Leads to a compilation error, while this code works perfectly:

public class Derived : Base
{
public int x;

public Derived()
{
x = baseField;
}
}

Does it really matter?

Understanding our code

I guess that in almost all cases this will not cause noticeable difference. It will however make a difference when we’re depending on that order, or, when the field initializers call methods that depend on side effects. So for start it’s important that we understand how our code works.

Make use of it

The next thing we can do is to use the field initializers as a hack to precede base constructors code. For example, if we have this 3rd party logger:

public class MyLogger
{
public MyLogger()
{
if (!Directory.Exists(@"C:\Logs"))
{
throw new InvalidOperationException("Can't start logs");
}

File.WriteAllText(@"C:\Logs\Session.log", "Initialized");
}
}

And we want to change it’s behavior without rerouting all it’s methods (of course this is not the only way), we can initialize a dummy field with method call that creates the directory:

public class MyLoggerHack : MyLogger
{
private bool dummy = CreateDirectory();

private static bool CreateDirectory()
{
if (!Directory.Exists(@"C:\Logs"))
{
Directory.CreateDirectory(@"C:\Logs");
}

return true;
}
}

So using this hack, we can “add” new code to the beginning of an existing call constructor without actually changing the original class.

AOP without weaving

In this post I’ll present a usage of runtime method replacer in AOP context. The idea behind it is to change the behavior of an application without changing the IL of its methods. In this post I’ll show how to log an exception from a method.

This post is based on the work of Ziad Elmalki who posted the original method replacer. It is also based on the updated code for the method replacer by Chung Sung which is compatible with the new .NET framework versions. Lastly thanks to Roy Osherove who mentioned those recently.

Replacing methods

The method replacer uses the following concept – after a method is jitted it receives a pointer of the jitted code. You can see how to extract that address in the original post. After extracting the addresses, we can simply replace one method with another:

public static void ReplaceMethod(IntPtr srcAdr, IntPtr destAdr)
{
unsafe
{
if (IntPtr.Size == 8)
{
ulong* d = (ulong*)destAdr.ToPointer();
*d = (
ulong)srcAdr.ToInt64();
}
else
{
uint* d = (uint*)destAdr.ToPointer();
*d = (
uint)srcAdr.ToInt32();
}
}
}

As a simple example, if we have these two methods:

public class MyClass
{
public static void Foo()
{
Console.WriteLine("In Foo");
throw new Exception("I am done here!");
}

public static void Bar()
{
Console.WriteLine("In Bar");
}
}

Then executing Foo in the following context:

MethodInfo barMethod = typeof (MyClass).GetMethod("Bar");
MethodInfo fooMethod = typeof (MyClass).GetMethod("Foo");
MethodUtil.ReplaceMethod(barMethod, fooMethod);
MyClass.Foo();

Will actually lead to the next result:

image

Which is… Cool!

Catching exceptions in Foo

What I’d like to present is a simplified example of how to catch an exception in business code without modifying it. A similar functionality to PostSharp exception handling. What we’re about to do is to hijack the original calls to Foo and redirect those to our new wrapper method. Our new wrapper method will call the original one inside a try/catch block.

Storing the original Foo

Since we’re about to intercept calls to Foo based on its address, we’d like to store a “way” to call the original method later. The “way” to do it is simple, we’ll extract the method address before starting the interception and create a delegate to it using marshaling. The delegate will be stored on a field:

MethodInfo fooMethod = typeof (MyClass).GetMethod("Foo");
IntPtr fooAdress = MethodUtil.GetMethodAddress(fooMethod);
OriginalFoo =
Marshal.GetDelegateForFunctionPointer(fooAdress, typeof (Action));

Creating the wrapper

For the purpose of this example we could prepare a stub in the project istelf. But, in order to prove that it is likely possible to create a more general solution, we will generate the wrapper at runtime.

Since the wrapper is going to receive the calls instead of Foo it must have the same signature. Besides, our wrapper will retrieve the original Foo delegate from a static field named OriginalFoo. The delegate will be called from the method inside a try/catch block.

We will generate a dynamic method that replaces the original method:

// The field holding the delegate to the original Foo
FieldInfo originalFooDelegateField = typeof (FooProtector).GetField("OriginalFoo");

MethodInfo invokeDelegateMethod = OriginalFoo.GetType().GetMethod("DynamicInvoke");
MethodInfo innerExceptionGetter = typeof(Exception).GetProperty("InnerException").GetGetMethod();
MethodInfo exceptionMessageGetter = typeof(Exception).GetProperty("Message").GetGetMethod();

var dynamicMethod = new DynamicMethod("FooProtector", typeof(void), new Type[0]);
ILGenerator ilGenerator = dynamicMethod.GetILGenerator();

Label beginExceptionBlock = ilGenerator.BeginExceptionBlock();

// Preparing the call to the original Foo -
// Load the original Foo
ilGenerator.Emit(OpCodes.Ldsfld, originalFooDelegateField);
// Load "no arguments" to invoke the delegate
ilGenerator.Emit(OpCodes.Ldnull);
// Invoke the delegate and call original Foo
ilGenerator.Emit(OpCodes.Callvirt, invokeDelegateMethod);
ilGenerator.Emit(
OpCodes.Pop);

ilGenerator.Emit(
OpCodes.Leave, beginExceptionBlock);
ilGenerator.BeginCatchBlock(
typeof (Exception));

// Extract the exception message
ilGenerator.Emit(OpCodes.Callvirt, innerExceptionGetter);
ilGenerator.Emit(
OpCodes.Callvirt, exceptionMessageGetter);

// Print the exception message
MethodInfo info = typeof (Console).GetMethod("WriteLine", new[] {typeof (string)});
ilGenerator.Emit(
OpCodes.Call, info);

ilGenerator.Emit(
OpCodes.Leave, beginExceptionBlock);
ilGenerator.EndExceptionBlock();
ilGenerator.Emit(
OpCodes.Ret);

// Trigger method compilation
dynamicMethod.CreateDelegate(typeof (Action));

This wrapper calls the original method through a delegate. In case an exception is thrown, it extracts the original exception and prints to the console the message.

Is it working?

Let’s revisit the original code and update it to the protecting code:

FooProtector.ProtectFoo();
MyClass.Foo();

The expected result is two messages printed, where the second one is the exception message “I am done here!”. As we can happily see, this is the exact result:

image

Conclusion

The concept of replacing methods using their jitted versions can be useful. It can be used to for AOP where it can be used for logging, exception handling and basically applying any custom aspect. It can also be used to modify some 3rd party code behavior for which we have no source code. Additionally, as Roy says is can be used as an engine for mocking frameworks.

But there are some disadvantages too. Firstly, it is very dependent on the compilation outcome which makes it quite fragile. Secondly, it is sensitive to optimizations, for example inlined methods cannot be handled. Thirdly, when it is used extensively it requires generation and JIT of many dynamic methods which might lead to a performance hit.

How to use switch statement with class

The switch statement controls the execution flow using comparison of constant values. This implies that in each case, the possible values are of primitive or Enum types. What can we do if we want to use a custom class as the switch expression? Clearly this is impossible by default, since custom classes are not primitives or Enums. The solution is to use the implicit cast.

Implicit cast

Implicit cast can is implemented using the implicit cast operator. Implicit cast enables, for example, assignment of instance from one type into a variable of other type.

Example of implicitly castable class:

public class Person
{
public string ID { get; set; }

static public implicit operator int (Person person)
{
return int.Parse(person.ID.Substring(0, 1));
}
}

This casting allows this simple assignment:

var person = new Person() {ID = "0-12345678-9"};
int quality = person;

Switch on class

In order to use a class instance as the switch expression we must choose a governing type for the cases (primitive or Enum) and make the original class castable to that type. For example, if we choose a class of type Person and governing type int then we must have an implicit cast from Person to int.

For example, using the class Person:

var person = new Person() {ID = "0-12345678-9"};

switch (person)
{
case 0:
Console.WriteLine("This is a top level person");
break;
case 9:
Console.WriteLine("This is a bottom level person (probably a developer)");
break;
default:
Console.WriteLine("Just a regular person");
break;
}