Back in November 2009, I wrote a Christmas list describing my hopes and dreams for the future of Windows driver development (see What I Want for Christmas (2011)). Since the release of the Windows Developer Preview, a number of people have asked me how the Win8 WDK stacks up against that list. So, let’s see!
The first and most important item on my Christmas wish list was a driver development environment that was integrated with Visual Studio. I wrote (in part):
While the VS editor might suck ass (might?) there are just too [many] useful things in VS to ignore it any longer. Too many add-ins. Too many features. So, I want a driver development environment that’s both supported by Microsoft and fully integrated with the VS IDE. This means I want VS to be able to successfully parse ntifs.h (and his include files), reasonably handle the conditionals therein, and provide me IntelliSense, F1 for help, and all the other VS features and wonderments. I want the various VS add-ins that provide cool features such as smart refactoring (including the ability to rename variables in a project… man, do you have any idea how great that is) to work properly.
I need the VS IDE to front-end the kernel debugger, preserving all its features, so I don’t have to suffer the slings and arrows of WinDbg’s arbitrary behavior.
It seems that this wish, my first and most important wish, has come true. We finally have a WDK that is fully integrated with Visual Studio. And while there’s certainly room for a bit of smoothing out of that integration, what we’ve seen in the Win8 WDK and VS11 is pretty much the core of what I asked for.
Note, too, that the new VS-integrated WDK actually does more than what I asked for on my list. Not only are the driver coding and debugging processes integrated, but so are the signing, deployment, and testing processes built into this same environment. This is a level of integration that I, quite frankly, didn’t even think to ask for. So, kudos to the WDK team on reaching beyond what I had hoped.
But I didn’t only ask for a new, VS integrated, WDK build environment, I also wished for backward compatibility with BUILD. I wrote:
I need to be able to build driver projects seamlessly in either VS or using “build” without any inconvenience. I need to be able to seamlessly “round trip” between VS and traditional build, so that any files I add to a “sources” file are easily (or, better yet, automatically) included in my VS project, and any files that I add to my VS project easily (or, better yet, automatically) show up in my sources file.
This is a wish that has not come true. And that’s a darn shame. While we understand that we all face resource constraints, I think this is a feature that’s far too important to be “below the line” and not implemented.
Why is it important to be able to “round trip” between the old build environment and the new VS integrated environment? One reason is that this would allow us to support “down level” build environments much more easily. In case you didn’t notice, the Win8 WDK does not support building drivers that are targeted for Windows XP. If you’re like us here at OSR, you still need to support Windows XP and probably Windows 2000 as well. This means that you’ll have to manage and support two entirely separate build environments that use entirely separate sets of project metadata. Which totally and completely sucks. When you add a file to your project, change the compiler flags, or define some new preprocessor value, you’ll need to do this in two separate places: You’ll need to do it in VS for your new Vista and later build environment and you’ll need to do it in your sources file for your XP build environment.
However, having to maintain two entirely separate build environments is not the only reason that not having the ability to “round trip” between VS and sources is important. The most important reason is that the ability to convert from sources to VS and back to sources again allows you to check that your driver project was properly converted. Unless you happen to be a Visual Studio and/or MSBUILD expert, when you import or convert your driver project into the Win8 WDK environment, you’ll have no clue if the import was done correctly. Sure, you’ll be able to tell if the resultant project builds and runs. But that doesn’t necessarily mean all the little settings you painstakingly established in your sources file have been properly brought forward. Given that you do understand the build/sources system, if you had the ability to convert your project to VS and then back again, you’d at least be able to check to see if the project is the same as the one you converted.
So, in summary… for my first and most important Christmas wish for Visual Studio integration, I got about 85% of what I asked for. Plus, I got a bunch of stuff (like automated signing, deployment, and testing) that I didn’t even think to ask for. Darn good work, by anybody’s standards.
The next thing I asked for was DMA virtualization. I wrote:
I want to see DMA scribbles eliminated for all time. IOMMUs are here, and I want Windows to use them. For all DMA operations. This would also mean that I can stop writing articles about why people need to use Map Registers and not just call MmGetPhysical Address; and you, dear reader, would be saved from having to read those articles.
I still think this is a laudable goal. Unfortunately, I’m not aware of any details that have been made public about DMA virtualization in Win8.
So we’ll have to mark this particular wish, DMA Virtualization, as “wait and see.”
The third item of which I dreamed was a more advanced, cleaner, architecturally more complete and rigorous KMDF. I wrote (again, in part):
What we have from KMDF today is not enough. I want the next major revision of KMDF [with] aggressive, continued, forward architecture and development.
I want the rough edges smoothed out. I want somebody with a solid knowledge of driver development, and a broad architectural view, to spend serious time reviewing KMDF with the goal of making the level of abstraction consistent, and on average, just a bit higher than it is today. [And] I want KMDF to include a new set of features and extensions that ease common programming patterns.
My wish list actually described several specific things that I hoped would be changed, enhanced, and smoothed out in KMDF.
When I wrote this, I quite frankly didn’t expect that I’d get it. And my expectations were met. There are a few very nice tweaks to KMDF in this new release of Windows, but in general, KMDF remains what it was. I’m sure I know why this is: The KMDF/UMDF group was plenty busy adding other features. But, as much as I like KMDF – and as I said back in the original article, I am a major KMDF fan – KMDF remains a good, but sometimes disappointingly ragged and incomplete, framework. The most coherent and compelling part of KMDF is its elegantly simple, integrated implementation of PnP and Power Management. And while this alone makes KMDF worth using, there is so much more that KMDF could be. The abstractions could be so much more coherent and clear. The ease-of-use could be so much greater. And I don’t think it would take that much work to elevate KMDF from “good” to “really really great” – I just wish somebody with adequate time and vision would step-up and get this done. Maybe what Microsoft needs to do is hire OSR to sketch out a vision for this. Give me a call anytime, guys. You have my number 🙂
A dynamically advanced and cleaned-up KMDF? Not in Win8. So this wish remains unfulfilled.
Another item that was on my Christmas list that I didn’t get, and that I’m not surprised at, was a new implementation of UMDF. My original request was:
I would like the ability to take my KMDF driver unchanged and compile it for use in either user mode or kernel mode. I very strongly believe that having entirely different programming models for user-mode and kernel-mode unnecessarily complicates WDF and actively hinders the movement of drivers out of the OS.
But, as far as I know, there’s been no work done in this area at all for Win8.
So, in terms of our getting an integrated KMDF and UMDF, the answer for Win8 is no.
When I made my next request, I knew I was pushing the envelope. But I figured if I didn’t aim high, I’d probably never get anything. So, I asked for:
I want real-time PFD [Prefast syntax checking] from within the [VS] IDE. You call a function, and you get a squiggly underline and a message saying you’re trying to call it at the wrong IRQL. Ever use tools like ReSharper from Jet Brains? Visual Assist by Whole Tomato? I have, and they’re pretty freakin’ cool. They provide real-time feedback on the code you write, telling you all sorts of things about your usage.
So, I asked, if ReSharper can do it, why can’t we have the same kind of analysis done in real-time by Prefast?
Apparently, the answer is that we can’t have it, at least not yet. In fairness, I have no idea what type of effort would be required to implement this feature. It’s a user-mode thing, so I have no clue. And it is important to note that we did get a pretty reasonable level of PFD integration in the Win8 WDK. When you build your driver project, it can be set to automagically run PFD from within the VS IDE and display the warnings or errors. And this is pretty cool.
So, in terms of dynamic PFD integration within VS, let’s say that I got 50% of what I asked for.
My penultimate request was for drivers that can do hardware access from user-mode. I wrote:
[I want] drivers using interrupts and register access in user mode. It is time to just bite the bullet and do this work, folks. If I get the things I’ve asked for above, it is beyond doubt that Windows drivers will be more reliable. But we don’t just need reliability. We need to limit the consequences of driver failure. To do that, we need to be able to move all drivers (at least those that are not necessary to boot the machine) to user mode.
I am very happy to say that, on this big and complicated request, I got my wish! Or, at the very least, most of it. UMDF does indeed support the ability to process interrupts and do direct register access in user-mode. There are a few restrictions, and some caveats (like, do you really want to handle PCI level-triggered interrupts in your user-mode driver?), but this very critical feature was implemented! Thanks!
User-mode hardware access? I’d say I got 95% of what I asked for. Yay!!
What’s a Christmas list without dreams, right? So, not being content to limit my Christmas wish list to just things that were possible within a reasonable timeframe, I just had to add one final item. I asked for the ability to write Windows drivers in C#:
I never liked C (I think it’s a terrible language). And you’ve doubtless read my columns over the last year or so talking all about how great it would be to write drivers in C#. You want to really, seriously, eliminate driver bugs? Do away with pool leaks, and memory scribbles, and reference counting forever? Bring on Windows drivers in C#.
I didn’t expect to get the ability to write Windows drivers in C#, and I didn’t. But it’s nice to dream.
When you add everything up, where does that leave us? Were my Christmas dreams fulfilled, or have I been left holding my Christmas stocking, sadly disappointed?
Well, dreams are just that: Fantasies that could all come true only in a perfect world. In our dreams, there are no constraints, no limits, to what we can obtain. However, in the real world, one cannot wave a magic wand and make wishes come true. Progress is often only attainable in incremental steps.
Having our driver development environment fully integrated, from code, to build, to sign, to deploy, to test, to debug – is a level of integration that I didn’t even dare hope for two years ago. The community has been asking for driver kit integration with Visual Studio for years, and somebody finally tackled the job, and it is here and it works! This is very obviously outstanding work. Plus, we have the integrated ability to run PFD within VS automatically.
And we got user-mode hardware access. This is epic, and it’s critical to a more stable Windows in the future.
When I look at what we got, overall, in the Win8 WDK I’m thrilled with what’s come to fruition. The Win8 WDK release is the most dramatic and important release since the compiler and linker were added to the DDK. It’s courageous, it’s visionary. And I’m sure it’ll serve as the basis for all sorts of new features in the future. The WDK team deserves a hearty “well done” for their work on this project.
I just really wish they had done VS to sources “round trip” conversion support. That’s the one thing that I’ll really miss.
As far as getting a dynamically revised KMDF, and a driver model that spans user-mode and kernel-mode without changes: I guess I’ll simply have to wait and hope.
There’s always Windows 9, right?
Peter Pontificates is a regular column by OSR Consulting Partner, Peter Viscarola. Peter doesn’t care if you agree or disagree, but you do have the opportunity to see your comments or a rebuttal in a future issue. Send your own comments, rants or distortions of fact to PeterPont@osr.com.
OSR Open Systems Resources, Inc.