It Passes Code Analysis

It’s great that your driver passes Code Analysis! But do you know that even the “Driver Recommended” rule set leaves out a lot of useful tests?

We are indeed extremely fortunate to have the tooling that is available to us for Windows driver development. Static tools such as Code Analysis (CA) and Static Driver Verifier (SDV), coupled with dynamic tools like Windows Driver Verifier and WDF Verifier make it much easier to avoid problems that aren’t always evident during basic in-house testing.

While there are lots of decent tools available across operating systems for general C/C++ code quality — tools such as clang-tidy and even CodeQL — I’m not aware of another operating system that has anything that comes close to the driver-specific tests that are available on Windows. This became clear to me recently when I spent some time working on a driver for a non-Windows operating system (an experience on which I will expound at some length at a future time, but in the meantime just let me advise you to never let anybody tell you that bazel is either simple or pleasant to use). But I digress.

Do You Know What’s Being Checked?

When using the Windows tools, what I see most people do is select the minimal tests — or the ones that Microsoft specifically recommends or requires — and leave it at that. Their driver passes and they think all is well. While this is certainly a reasonable starting strategy, it’s far from optimal. What’s not obvious (until you look) is that the default tests often do not select many tests that might be useful.

Let’s take CA as an example: Open one of your recent driver projects, select Properties, and go the Code Analysis section and select the Microsoft sub-section. If you’re like most folks, you’ll have selected either the Microsoft Driver Recommended Rules or the Microsoft Driver Must Fix Rules rule set.

From the property page above, click “Configure.” This will allow you to examine the rules that comprise the rule set that you have selected. If you’ve configured the Microsoft Driver Recommended Rules rule set, you’ll see something that looks like this:

Take your time to scroll through the list of rules, noting which ones are enabled and, more importantly, which ones are not enabled by the rule set that you’ve selected. Some of what you find might surprise you. You might be surprised to discover, for example, that many of the more advanced concurrency rules are not enabled by default, even in the Microsoft Driver Recommended Rules rule set. If you use many (er, any?) features of Modern C++, you might be disappointed that many of the C++ specific tests are not enabled by default.

My point here is NOT that Microsoft hasn’t done a good job of selecting rules to enable. Rather, my point is that most folks don’t realize which rules are enabled and which aren’t. So, when they’re running their driver through CA (to continue with that as our example) they can easily get a false sense of security. They can quite easily think “I’ve done my due diligence, I’ve tested my driver, I’ve passed CA and I can therefore feel confident that I have found all the errors that I could find.” Unless you know which rules your driver is actually being tested against, all you’ve really gotten is a false sense of security.

What We Recommend: Go Subtractive

For quite some time, our approach at OSR has been different to the one that’s provided by default. What we do (regardless of the test tool involved: CA, SDV, Driver Verifier) is that we enable all the available tests. We then disable tests, one by one, as we decide that they are not applicable to our situation or not useful to what we are doing.

For CA, specifically, we select the Microsoft All Rules rule set which, if you check as you should, does indeed enable all the available CA rules. This even includes the tests for following the recommendations specified by the Microsoft Guidelines Support Library (GSL) in support of the CPP Core Guidelines. We then include a header file that globally disables those specific CA warnings that don’t meet our needs.

Because each dev here has slightly different practices and preferences, each dev customizes the warning suppression header for their projects. I, for example, have come to embrace the use of nullptr. Scott has not. My projects, therefore, leave the rule that throws warning C26447 enabled. Scott typically disables this warning in his projects.

Note again that I’m using CA here as an example, but our recommended practice applies to the compiler’s general set of warnings (we always build at Warning Level 4 and disable the very few things that don’t apply), as well as to SDV, Windows Driver Verifier, and WDF Verifier. Enable everything, and then turn off the tests or constraints that you specifically don’t want applied to your project. By doing this, you’re likely to get much more comprehensive testing of your projects. And you will certainly be taking a much more deliberate, considered, approach to the tests that are enabled.