Data Modification Kit (DMK)

Products that encrypt, compress, or otherwise modify file-based data require some of the most complex kernel software developed for Windows. Even for seasoned kernel-mode programmers, the complexity of delivering per-file data modification solutions on Windows is significant. For a development team that is not fortunate enough to include multiple Windows file system and internals experts, the task of actually getting such a product to work can be daunting.

OSR has years of experience working with clients and other members of the development community to deliver such solutions. OSR has gone a step further with the Data Modification Kit (DMK) . Read on, to find out how the OSR DMK can facilitate the development of per-file data modification solutions for Windows that encompass a broad range of features while maintaining correctness, flexibility, and ease of use.

The Challenge

A common use for a Windows file system filter driver is to extend the base OS to provide support for encryption, compression, or other custom modifications on a per-file basis. We call this "per-file data modification." Often, supporting per-file data modification requires changing the size and format of the data as well as storing additional meta-data with the file. Unfortunately, implementing this level of functionality on Windows is a task riddled with numerous difficulties. Some of the most common difficulties are:

• Support for executable images - Because the executable image is memory mapped, file system filter drivers are bypassed when they interact with the Windows Virtual Memory system regarding the size of the file-backed section. This can lead to incorrect behavior of the application.
• Support for memory mapped data files - Any application that uses the Windows file mapping APIs can cause problems. Such applications are common, including Microsoft Word and Notepad.
• Support for arbitrary data modification - To simplify the task, existing solutions typically restrict their meta-data additions to affixing a header or trailer to the data. This does not provide them with the ability to perform arbitrary data modifications (such as compression, for example).
• Support for all standard Windows file systems - Each Windows file system comes with its own set of idiosyncratic behaviors that must be accommodated. And of course, this results in an increased testing burden because each unique scenario has to be scripted, tested, and have its results validated.
• A changing file system filter model - With Microsoft slowly moving to a newer filter model, most development teams are now faced with the task of having to support the older legacy platforms and yet coexist and work well with the new platforms of the future.
• Subtle, yet crucial, differences among Windows versions - Each version of Windows has introduced changes that affect Windows file systems. Just discovering these changes can be difficult enough; properly accommodating them in a product can be maddening.

There are a number of partial solutions to these issues. However, such solutions can compromise the capabilities of the product and still involve considerable specialized development.

The Solution

The OSR Data Modification Kit (DMK) addresses these complex issues by providing a comprehensive solution for supporting per-file data modification on Windows.

The DMK's primary value is that it provides a ready-made infrastructure that properly deals with Windows' complexities. As a result, regardless of the specifics of your data modification, you need only provide the actual routines that perform the data modification. This simplifies the development process, increases reliability, and decreases the time required to deliver your product to market. Additionally, because the DMK includes source code, it is suitable for use even in highly sensitive projects - everything is available for your inspection, modification, and customization.

Features

The following key features are supported by OSR's DMK:

• Customizable data modification algorithms. You decide the minimum block size for your algorithm. The DMK calls your data modification function, providing the data requiring modification. In response, your data modification function returns an optional header block plus the transformed data block. Note that the transformed data block need not be the same size as the input data block- you can make it smaller or larger.

  Benefit - Complete flexibility in choice of algorithm for your solution, regardless of minimum block size.

• Support for multiple, sequential, transformations. Because the OSR DMK allows the output data returned by your library to be any size relative to the input data, you can easily implement multiple transformations.

  Benefit - Implement compression and encryption in your solution (for example), gaining both disk space and performance, and giving your product another advantage for your customer.

• Support for all file systems. The OSR DMK supports FAT, NTFS, network redirectors, and other Windows file systems.

  Benefit - Your product works across all common Windows client and server releases, with no source-code changes to your data transformation library.

• Data safety. Even if the data modification toolkit is removed from a system that contains transformed files, each transformed file appears as a single file to the base file system, so it can be safely backed up, copied to removable storage, and later re-transformed if required.

  Benefit - Data integrity, including added meta-data, is maintained.

• Case-by-case file access control. Each time the file is opened, the DMK calls an authorization function that you provide, to perform custom access checks (if desired) and indicate whether file data should be transformed as it is accessed. Alternatively, your transformation library can indicate to the DMK that for a given open file instance the data should be provided as is - without performing any transformation. If your data modification driver is present on both a server and client system, you can (if you wish) shift the work required to transform (or reverse transform) to the client system, distributing workload, potentially decreasing network overhead (e.g., compressed data) and increasing security (e.g., encrypted data).

  Benefit - Implement your own access control scheme to augment the existing Windows implementation as well as control which version of the data will be seen by individual callers.

• Policy Change. The DMK allows you to change the specific data modification policy for a given file in a reliable and transparent fashion. Thus, the handling of existing files can be changed simply by indicating to the DMK that this should be done.

  Benefit - Provides tremendous flexibility in reconfiguring the product dynamically based upon policy changes.