Over the past several years, we have promoted the isolation filter model as a solution to a number of data virtualization projects. While we haven’t returned to discussing isolation recently, we have gained considerable experience building isolation filters. One of these projects that we are excited to be telling you about is our Monadnock Project, named after the most popular mountain to climb in the US.
For the past ten years, we have been working on the problems of transparent encryption and compression in a Windows environment. It’s been an amazing experience and our understanding of the complexities and nuances of the environment have continued to grow. In addition, the world has changed dramatically in the ensuing years. Data encryption isn’t something just for Windows any more and solutions need to be able to work across numerous platforms.
Even on Windows we know there are challenges. For example, interactions with the native file systems can lead to unexpected performance behaviors. And those folks in Redmond sure seem to work hard thinking up new ways to make our lives more difficult as each new Windows release contains new and interesting features to which we must adapt.
So a couple years ago we started planning a new version of our data transformation solution kit and after quite a bit of time and planning – along with diversions to deal with the complexities of the WHCK tests and do other activities necessary to make a living – we reached the conclusion that what we needed wasn’t a new version of our existing solution, but rather a different product. The key features we saw that our customers needed were:
- Encryption focus. As much as we love compression, there was no way to avoid the harsh reality that it is expensive (performance wise) to support and complex to implement. Plus, it turns out none of our customers are using it. Encryption is where it’s at.
- High performance. In the past we had added some cool features, like alternate data streams for all file systems, but that in turn caused a lot of extra I/O overhead and a corresponding drop in performance. Given that customers didn’t really care about streams for their products it doesn’t make sense to pay the performance penalty – not to mention the code complexity. So we decided it was time to simplify: focus on the features our customers need for building a robust, high performance encryption product.
- Simpler programming model. As much as we love kernel programming, it turns out that it’s difficult to do correctly and our customers struggle to find and keep qualified kernel developers. So our goal was to build a framework in which it was possible to build an entire encryption product without any kernel programming.
- Multiple platform support. This is a vital part of building products these days: customers want to share data between their devices, be it PC, Mac, iPhone, Android device or even Windows tablet or phone. Of course, since some of these platforms lock us kernel developers out, we have to find new ways to provide at some measure of functionality.
Bottom line for us was the realization that this really was a different Solution Kit. It’s very different than what we’ve done in the past. And yet it also allows us to leverage the insight and understanding we have gained in helping dozens of different customers implement their encryption products over the years. Thus was born Project Monadnock.
To better help you visualize the technology behind Monadnock, let’s describe some of its key components in conjunction with the architectural diagram:
Monadnock: Isolation at Heart
At the heart of Monadnock is our isolation filter (which really is a mini-filter, albeit a complex one). While we do expect to make an isolation filter kit available in the near-term future, for now, we’ve put it through its paces in the Monadnock project.
An isolation filter is a filter that focuses on isolating the “view” of a file from the “contents” of a file. From an implementation perspective, this is similar to the technique used by NTFS to provide transactional isolation: each such “view” is managed as a distinct virtual memory section – in this case, by the filter itself. It does this by managing the data in virtual memory and the file system data cache, so it’s certainly not a simple mini-filter solution.
When an application opens a file, the responsibility of the isolation filter is to properly manage what that application sees as the contents of the given file. The specific choice of views is actually not something that the isolation filter does – this is deferred to other components. In the case of Monadnock, that would be the “Data Transformation” layer. While isolation permits multiple views per file, Monadnock uses just two views: the “encrypted” view and the “decrypted” view. Thus, Monadnock based solutions can choose, on a per open basis, what exactly an application will “see”.
For example, suppose you are building an encryption product that wants to send documents attached to an Outlook e-mail in their encrypted form but you also want to ensure the Outlook files are themselves encrypted. In that case, the policy would be to give “encrypted” views to files other than those identified as Outlook files – such as the “OST” and “PST” files that Outlook uses for its cache and archives. Such decisions can be made based upon various attributes of the file open operation including the fully qualified path name of the file, the access requested, the disposition requested, the process and/or thread opening the file, the security credentials of the thread and/or process. This gives considerable control to those building real-world products with Monadnock.
Directory Caching
One of the challenges in building any filter on Windows is that applications frequently rely upon the size information within a directory listing in order to determine “how big” a file is. But because encryption changes the size of a file by changing its padding and adding additional “header information”, failing to provide the correct size information to applications can lead to incorrect behavior.
While it is possible to “correct” these entries on the fly, our experience in the past tells us that it’s important for performance reasons to cache these size corrections in order to minimize their impact on application performance. Every unnecessary I/O that we introduce in the system leads to dramatically lower performance. Thus, our directory caching layer is optimized to ensure we do not open files that aren’t encrypted and to cache both the physical and logical size information for files that are encrypted.
Key Management and Encryption using CNG
Another important decision for us was to use the Cryptography API: Next Generation. This library is supported with Vista and more recent and permits use of existing encryption implementations (notably AES) as well as the many optimizations that Microsoft has included (AES-NI instruction support for those CPUs that can exploit it, for example). CNG has also been certified to US FIPS 140-2 standards, making it easier for our customers to qualify their products for Common Criteria certification, for example.
CNG can be used from both kernel mode as well as user mode and Microsoft documents how to build extensions to CNG, components known as Cryptographic Providers. These providers can implement not only encryption algorithms but also Key Storage Providers.
Our goal in choosing this paradigm was to further simplify development for those using the Monadnock framework, while also making it easier to build robust, performant products that meet stringent security requirements for current and future environments.
Flexible Data Storage Layer
In our previous work we had a very feature rich data storage layer. After analyzing how our customers were using it, we determined that most of them did not need many of the features that we were providing, but they were paying a performance penalty associated with those features. So, while we love technically rich projects, we decided to look at ways to simplify our data storage model, while at the same time permitting those customers that do need to support additional features the flexibility to do so.
Thus, Monadnock’s data storage layer is a simplified, minimal state interface where, by default, we defer complex processing to a user mode service. For example, when an existing file should be encrypted, rather than perform that complex work in kernel mode (as we did in the past) we now refer the operation to a user mode service.
The user mode service can then take advantage of things like the ability to display “progress bars” to the user encrypting a file and opening the file for exclusive access while the conversion is ongoing. This has the added benefit of keeping the kernel mode implementation simpler, which in turn translates into a more resilient end user product.
One key to our model, is to use a shared source library for data storage management between user mode and kernel mode. Thus, a subset of the same library is used in kernel mode to access the physical contents of the file, ensuring compatibility between the user mode and kernel mode code. This technique also greatly simplifies the process of developing this layer, as most of the development and debugging work can be done in user mode.
In addition, this has also permitted us to structure this critical library in such a way that we can support non-Windows platforms. Thus, this is a key piece permitting interoperability between platforms.
Shifting from Kernel to User
One thing we have seen in working with prospective customers is that frequently they do not have kernel level expertise. Our existing customers often tell us they have a difficult time finding such individuals and when an organization does invest to train someone, they often leave the organization because they can do better elsewhere.
We live, eat and breathe kernel development at OSR, so it took a while for us to wrap our heads around the idea of pushing more processing work into user mode. So while we might be slow, we have realized that in order to make Monadnock easier to use we have to make it possible to use it without any kernel programming at all. Thus, a developer using the Monadnock framework can leave the kernel programming to us, while they focus on adding the key management, administration, and other unique product features, all in user mode.
Of course, for customers that need to have control of the kernel components, they can certainly do so, as Monadnock will remain available (under license) in full source code form.
Developer Version
In the past, we’ve always struggled with the idea of providing developer versions of our Solution Kits. This was at least in part due to the difficulty in defining “best practices” example implementations that matched the widely varying policies and product features that customers were asking to see. The shift in focus to enabling customers to develop in user mode gives us a much clearer model for providing a base example, and allowing developers to “try out” their own customizations to Monadnock to match their product needs.
Thus, we will make a developer’s version available for Monadnock. We will include source code to the user mode components we expect developers will customize, with the other components provided in binary form, either as drivers, executables and binary libraries.
Monadnock: Our Newest Framework
We’re excited about Monadnock, as it will broaden the appeal of our encryption solutions far beyond what we’ve been able to accomplish in the past. For our existing customers, it offers a path forward as we continue to evolve and improve our product offerings.
We expect to make Monadnock available as part of a “technical preview” to help us showcase the solution and select a limited number of companies to work with closely during a Beta period. The full release will still focus on Windows, but it will include non-Windows platform support, which we will continue to enhance as we learn more about our customer’s needs.
For those interested in joining the Monadnock Beta program, please sales@osr.com.
OSR Open Systems Resources, Inc.