How to replace the inbox Serial.sys driver with the Serial sample driver from the WDK

Windows Resource Protection (WRP) prevents badly behaved applications from overwriting essential Windows operating system files. Unfortunately, WRP can create difficulties for a driver developer who wants to develop and test a driver to replace a Windows inbox driver. The .kdfiles kernel debugger command is currently the only supported way to replace an inbox driver (and circumvent WRP) in Windows 7 and Windows Vista.

I discovered this fact the hard way when I recently tried to install the Serial sample driver from the WDK on my Windows 7 computer. At that time, the readme file for this sample described an out-of-date technique for replacing the inbox Serial.sys driver with the Serial sample driver. This technique is discussed in a white paper titled Windows File Protection and Windows, which was published when Windows XP was released in 2001. In Windows XP, Windows File Protection—the predecessor to WRP—can be disabled by changing the value of a registry key named SFCDisable, but this technique does not work in Windows 7 and Windows Vista.

The .kdfiles command can be used to replace files only on a target computer that is being controlled by the Windows kernel debugger. Typically, the debugger runs on a host computer that communicates with the target computer through a debugging cable.

In my debugging configuration, a null-modem cable connects a COM (serial communication) port on the target computer to a COM port on the host computer. However, a serial connection might be too slow if you intend to copy large files between the target and the host. If you need faster communication, the Windows debugging tools also support IEEE 1394 and USB 2.0 connections.

Because I wanted to use the kernel debugger to step through the Serial sample driver, which controls a COM port, I needed a target computer that had a second COM port to connect to the kernel debugger on the host computer. I dedicated COM2 to the debugging connection, but left COM1 under the control of the installed serial driver.

Windows has intrinsic support to enable a computer to serve as a target for a kernel debugger. The debugging support module that runs on the target computer has a built-in serial driver that it uses to communicate with the host computer. In other words, the debugging connection does not use the installed serial driver. Thus, communication with the kernel debugger on the host computer continues without interruption if the debugger hits a breakpoint in the installed serial driver.

On the target computer, Device Manager does not display the COM port that the debugger is using. However, if the target computer is restarted with kernel debugging disabled, this COM port is again displayed by Device Manager and is again controlled by the installed serial driver.

Read more: Windows Driver Kit (WDK) Documentation Blog

Posted via email from jasper22's posterous