X64 MANUAL STACK RECONSTRUCTION AND STACK WALKING

In this installment I would like to expand a bit on a previous post of mine called Challenges of Debugging Optimized x64 Code.  In that post I discussed the nuances of the x64 calling convention (thankfully of which there is only one) and how it is used in optimized builds of software.  The calling convention is sometimes referred to as the Application Binary Interface (ABI).  In this post, I would like to discuss the x64 unwind metadata and how you can use it in the debugger to manually walk a stack.

In some cases, you may have a corrupted stack that the debugger simply cannot effectively walk for you.  This often happens because the debugger walks a stack from the top down (assuming the stack grows upwards as if it were a stack of places on a table), and if the stack is sufficiently trashed then the debugger cannot find its bearing.  In the x86 world, a large percentage of the time, you can spot the stack frames by following the chain of base pointers and then build a crafty stack backtrace command to display the stack at some point in time.  But in the x64 calling convention there is no base pointer.  In fact, once a function’s prolog code has executed the rsp register generally never changes until the epilog code.  To read more about x64 prolog and epilog code conventions, go here.

Moreover, the syntax for creating a crafty stack backtrace command in the x64 environment is currently undocumented, and I aim to shed some light on that near the end of that blog post. J

The Example Code

For this blog post I have used the following example C# code that requires the .NET 4.0 framework and can be easily built from a Visual Studio 2010 command prompt.  You can find the code below:

using System;
using System.Numerics;
using System.Threading;
using System.Threading.Tasks;
using System.Collections.Concurrent;

class EntryPoint
{
   const int FactorialsToCompute = 2000;

   static void Main() {
       var numbers = new ConcurrentDictionary<BigInteger, BigInteger>(4, FactorialsToCompute);

       // Create a factorial delegate.
       Func<BigInteger, BigInteger> factorial = null;
       factorial = (n) => ( n == 0 ) ? 1 : n * factorial(n-1);

       // Now compute the factorial of the list
       // concurrently.
       Parallel.For( 0,
                     FactorialsToCompute,
                     (i) => {
                       numbers[i] = factorial(i);
                     } );
   }
}

The spirit of this code is to concurrently compute the first 2000 factorials and store the results in a dictionary.  This code uses the new Task Parallel Library to distribute this work evenly across the multiple cores on the system.  To compile the example (assuming the code is stored in test.cs), you can execute the following command from a Visual Studio 2010 command prompt:

csc /r:system.numerics.dll test.cs

Read more: NTDEBUGGING BLOG

Posted via email from jasper22's posterous