Last reviewed and updated: 10 August 2020
The example driver and test app accompanying this article can be found in OSR’s GitHub repo at https://github.com/OSRDrivers/Inverted.
One of the most common questions we see from students, clients, and new Windows driver writing colleagues is, “How can a driver perform a callback to a user-mode program?” The answer to this is simple: it can’t. There is no architected way in Windows for a driver to call a given callback in a user program. It’s simply not something you can do.
If it’s that simple, should we just end this article here? Probably not. Because when we investigate further, people who ask this question typically don’t actually want to know how a driver can call a user-mode callback. Rather, what they really want to know is, “How can my driver notify a user-mode program that some event has occurred in the driver.” It seems that devs who mostly work in user mode just naturally think “perform a callback” when they have the requirement “notify about an event.”
Enter the Inverted Call Model
So, what is the best way for a driver to notify a user-mode program that a given event has occurred? The answer to that question is simple: The driver should use what is referred to as the Inverted Call Model. We first described this technique in The NT Insider back in January of 2002: http://www.osronline.com/article.cfm?id=94.
Using this technique is much easier than its fancy name makes it sound. The application sends one or more requests (typically device controls, also known as IOCTLs) to the driver, with a pre-determined control code. The driver keeps these requests pending. When the driver needs to notify the application of the occurrence of an event, the driver simply completes one of the pending IOCTLs. The application discovers that the event has occurred by noticing that the previously issued IOCTL has been completed by the driver. Having received the notification, the application sends the IOCTL back to the driver where the driver once again holds the request pending until it needs to notify the application of another event.
That’s all there is to it. I did tell you this was simple, didn’t I?
One objection that new Windows driver writers often raise to this approach is that it seems like “an awful lot of overhead” to use an I/O completion as an event notification mechanism. But, in fact, this isn’t true at all. The Windows I/O Subsystem and its companion Win32 are both highly optimized for handling I/O completions. If you’re using asynchronous I/O (supplying an OVERLAPPED structure) to send multiple IOCTLs to the driver to hold for event notification purposes, using Completion Ports for handling the notification is particularly efficient. And, when you think about it, using something like a shared event between user-mode and kernel-mode has at least as much overhead as, and many more disadvantages than, the Inverted Call Model.
Implementing the Driver
The pattern for implementing the Inverted Call Model in your driver couldn’t be simpler. You define an IOCTL control code that the driver and the application will use for notification purposes:
#define IOCTL_OSR_INVERT_NOTIFICATION \ CTL_CODE (FILE_DEVICE_INVERTED, 2049, METHOD_BUFFERED, FILE_ANY_ACCESS)
In the driver’s EvtIoDeviceControl Event Processing Callback, the driver takes each IOCTL Request it received with the designated control code, and forwards it to a Queue with manual dispatching. The code for doing this is shown in the example below:
VOID
InvertedEvtIoDeviceControl(WDFQUEUE Queue,
WDFREQUEST Request,
size_t OutputBufferLength,
size_t InputBufferLength,
ULONG IoControlCode)
{
PINVERTED_DEVICE_CONTEXT devContext;
NTSTATUS status;
ULONG_PTR info;
devContext = InvertedGetContextFromDevice(
WdfIoQueueGetDevice(Queue) );
//
// Set the default completion status and information field
//
status = STATUS_INVALID_PARAMETER;
info = 0;
switch(IoControlCode) {
//
// This IOCTL are sent by the user application, and will be completed
// by the driver when an event occurs.
//
case IOCTL_OSR_INVERT_NOTIFICATION: {
//
// We return an 32-bit value with each completion notification.
// Be sure the user's data buffer is at least long enough for that.
//
if(OutputBufferLength < sizeof(LONG)) {
//
// Not enough space? Complete the request with
// STATUS_INVALAID_PARAMETER (as set previously).
//
break;
}
status = WdfRequestForwardToIoQueue(Request,
devContext->NotificationQueue);
//
// If we can't forward the Request to our holding queue,
// we have to complete it. We'll use whatever status we get
// back from WdfRequestForwardToIoQueue.
//
if(!NT_SUCCESS(status)) {
break;
}
//
// *** RETURN HERE WITH REQUEST PENDING ***
// We do not break, we do not fall through.
//
return;
}
// ... other cases omitted from example...
}
//
// Complete the received Request
//
WdfRequestCompleteWithInformation(Request,
status,
info);
}
In the above example, when the driver is presented with a Request that has a control code value of IOCTL_OSR_INVERT_NOTIFICATION, it checks to see if the length of the output buffer associated with the Request is at least the size of a LONG integer (4 bytes). It does this because in this example when the driver notifies the application of an event, it will also return a 32-bit value containing additional information about the event. If the output buffer length is less than the size of a LONG integer, the driver completes the Request with STATUS_INVALID_PARAMETER.
If the output buffer accompanying the Request is of sufficient length, the driver calls WdfRequestForwardToIoQueue to ask the Framework to place the Request on the driver’s notification queue. The driver created this Queue in its EvtDriverDeviceAdd Event Processing Callback and configured it with Manual Request Dispatching. The handle for the Queue was saved in the WDFDEVICE’s context area in the field named NotificationQueue. The only use of this Queue is for holding IOCTL_OSR_INVERT_NOTIFICATION Requests until they are completed to notify the application of an event. If the driver’s attempt to forward the Request to its notification queue is not successful, the Request is completed with STATUS_INVALID_PARAMETER.
If the driver successfully forwards the Request to its notification queue, it simply returns. The Request is not completed in the EvtIoDeviceControl Event Processing Callback, and thus remains in progress (i.e. pending) until it is completed later by the driver.
When an event occurs, and the driver wants to notify the application of the occurrence of that event, the driver simply removes the first pending IOCTL Request that it finds on its notification queue and completes that Request. Prior to completing the Request, the driver can optionally return additional information associated with the event such as an event code. This procedure is shown below:
VOID
InvertedNotify(PINVERTED_DEVICE_CONTEXT DevContext)
{
NTSTATUS status;
ULONG_PTR info;
WDFREQUEST notifyRequest;
PULONG bufferPointer;
LONG valueToReturn;
status = WdfIoQueueRetrieveNextRequest(DevContext->NotificationQueue,
¬ifyRequest);
//
// Be sure we got a Request
//
if(!NT_SUCCESS(status)) {
//
// Nope! We were NOT able to successfully remove a Request from the
// notification queue. Well, perhaps there aren't any right now.
// Whatever... not much we can do in this example about this.
//
return;
}
//
// We've successfully removed a Request from the queue of pending
// notification IOCTLs.
//
//
// Get a a pointer to the output buffer that was passed-in with the user
// notification IOCTL. We'll use this to return additional info about
// the event. The minimum size Output Buffer that we need is sizeof(LONG).
// We don't need this call to return us what the actual size is.
//
status = WdfRequestRetrieveOutputBuffer(notifyRequest,
sizeof(LONG),
(PVOID*)&bufferPointer,
NULL);
//
// Valid OutBuffer?
//
if(!NT_SUCCESS(status)) {
//
// Complete the IOCTL_OSR_INVERT_NOTIFICATION with success, but
// indicate that we're not returning any additional information.
//
status = STATUS_SUCCESS;
info = 0;
} else {
//
// We successfully retrieved a Request from the notification Queue
// AND we retrieved an output buffer into which to return some
// additional information.
//
valueToReturn = InterlockedExchangeAdd(&DevContext->Sequence,
1);
*bufferPointer = valueToReturn;
//
// Complete the IOCTL_OSR_INVERT_NOTIFICATION with success, indicating
// we're returning a longword of data in the user's OutBuffer
//
status = STATUS_SUCCESS;
info = sizeof(LONG);
}
//
// And now... NOTIFY the user about the event. We do this just
// by completing the dequeued Request.
//
WdfRequestCompleteWithInformation(notifyRequest, status, info);
}
As shown above in the function InvertedNotify, the driver starts by attempting to remove a Request from its notification queue, by calling WdfIoQueueRetrieveNextRequest. The Request the driver is attempting to remove is an IOCTL with the control code IOCTL_OSR_INVERT_NOTIFICATION. This Request would have been forwarded to the notification queueby the driver’s EvtIoDeviceControl Event Processing Callback that was described previously.
If the driver is not successful in getting a Request from its Notification Queue the driver just returns from this function. This would be the case when, for example, the notification queue is empty (because the user application has failed to send a sufficient number of IOCTL_OSR_INVERT_NOTIFICATION Requests to the driver). In this case, the driver will either need to store the event in some driver-specific way, or the user application will simply not be notified of this event occurrence. If it’s important for the application to not miss any event notifications, it will need to be sure to always keep one or more IOCTL_OSR_INVERT_NOTIFICATION Requests pending in the driver. The application would do this by calling DeviceIoControl multiple times, most likely using asynchronous completion with an OVERLAPPED structure for each request, and by immediately re-issuing the IOCTL each time it is completed by the driver to notify the application of an event occurrence.
If the driver successfully removes a Request from its notification queue, it calls WdfRequestRetrieveOutputBuffer to get a pointer to the output buffer associated with the Request. Recall that on receiving the IOCTL, the driver verified that an output buffer was supplied with the Request and that the output buffer was at least sizeof(LONG) bytes in length. If the driver is successful in getting a pointer to application’s specified output buffer, it returns into that output buffer additional information associated with the event. In the example, this information is an event sequence number – a signed 32-bit value that increases by one for each event signaled back to the user.
Whether the driver was successfully able to get a pointer to the application’s output buffer or not, the driver next completes the Request by calling WdfRequestCompleteWithInformation. The application interprets this completion as notification of the event occurring. If an output buffer was specified, the application can examine the additional information about the event returned by the driver into that buffer.
Looks Too Simple
Well, of course it looks simple. Because it is simple. We’re using WDF, after all. Note that one very important, but perhaps easily overlooked, feature of the example is that it keeps the Requests to be completed when an event occurs on a WDFQUEUE. This isn’t chance, and you couldn’t (for example) easily substitute a WDFCOLLECTION or a vector of WDFREQUESTS for that WDFQUEUE. This is because the WDFQUEUE automatically handles request cancellation. Thus, when the application exits with Requests in progress on the driver’s notification queue, the Framework will automatically complete those Requests with STATUS_CANCELLED. The driver doesn’t even have to get involved.
Another detail that you’ll notice when you read the full example is the notification queue is setup so that it is not power-managed. That is, the PowerManaged field of the WDF_IO_QUEUE_CONFIG structure is set to FALSE. This is actually pretty important. If the notification queue was power managed, the driver’s ability to remove Requests from that Queue would change based on the device’s power state (D State). So if an event occurred that the driver wanted to notify the application about while the device was in a low power D State, that call to WdfIoQueueRetrieveNextRequest would fail with a return status of STATUS_WDF_PAUSED.
The Application Side?
A gentleman was asking in the NTDEV forum recently how to implement the user side of this model. Frankly, we don’t do user-mode here at OSR, so there’s not a lot we can tell you that’s both clever and definitive. However, we can repeat what we said earlier in this article: You probably want to implement this notification using asynchronous I/O and completion ports, and you almost certainly want to be careful in your application to send up enough notification requests to ensure that the driver always has one pending. That doesn’t seem very difficult to us.
Just to give you an idea of how you might approach the user-mode side of things, we threw together an example application that you can find in the ZIP archive accompanying this article. But be warned: It’s just test code. We don’t care that it’s sloppy or that it never calls free or CloseHandle, or that it’s got bugs. We already know its crap, so don’t email us to tell us about how badly our user-mode code is written. Of course, please do feel free to write something that’s proper, elegant, attractive, and well-documented… and then send it to us. We’ll be happy to share it with the community.
Natural Extensions
We’ve outlined just the basics in this article. There are several extensions to this model that should be relatively easy for you to implement. For example, on receiving the event notification you might want the user-mode application to do some processing and return some data back to kernel mode. To do this, the user-mode app could copy the sequence number that it received with the original event notification to a structure in its input buffer. Then the app would then add any data it wants to send to kernel mode to that structure, and sends the IOCTL_OSR_INVERT_NOTIFICATION back to the driver. When the driver receives this IOCTL it would process the information received from the application’s input buffer and then forward the received IOCTL Request to the notification queue. Part of the processing done by the driver might be to match the supplied sequence number with an event the driver has buffered. Or something. I hope you get the idea.
Summary
That’s the Inverted Call Model. When you want your driver to be able to notify a user-mode application about the occurrence of an event, have the driver complete a request back to the application! It’s an easy model to implement, it’s easy to test, and it’s remarkably efficient. Just be sure to have the application keep enough IOCTLs in progress at the driver, so there’s always one available when the driver wants to notify it of an event.
Code associated with this article:
OSR Open Systems Resources, Inc.