使用缓存管理器，尽你之能力(Cache Me if You Can: Using the NT Cache Manager )

在pediy.com发了一篇译文，简单转载这里。

缓存管理器属于NT内存管理器紧密集成的软件组件，与虚拟内存系统集成文件系统缓存数据。

好处是物理内存的使用和文件缓存和系统运行程序在缓存管理器下达到均衡应用。

另一个使用缓存关键原因是文件可被标准文件系统接口存取，如read和write等。或通过内存管理器形成一个“内存映射”文件。

缓存管理器数据结构：下段描述文件系统和缓存管理器共享的数据结构。

Buffer Control Bloxk (BCB)

Most of the buffer control block (BCB) is opaque. The first portion of the BCB is exposed to file systems:

typedef struct _PUBLIC_BCB {
    CSHORT NodeTypeCode;
    CSHORT NodeByteSize;
    ULONG MappedLength;
    LARGE_INTEGER MappedFileOffset;
} PUBLIC_BCB, *PPUBLIC_BCB;

 

Buffer控制块(BCB)

BCB用来跟踪文件，当此文件影射到系统地址空间。在文件系统在内存锁定(Pin)数据以作关键操作。结构后两个字段是指定了此BCB文件管理的范围。

File Size Information

The file system and Memory Manager each maintain information about the size of the file. Whenever the file system establishes mapping for a file it indicates the current size of the file. Any subsequent changes to the size of the file are similarly indicated to the Cache Manager.

文件系统和内存管理维护文件大小信息。所有后续文件大小改动都同样被cache manager显示处理。

There are three values used by the Cache Manager to indicate the current size of the file:

typedef struct _CC_FILE_SIZES {
    LARGE_INTEGER AllocationSize;
    LARGE_INTEGER FileSize;
    LARGE_INTEGER ValidDataLength;
} CC_FILE_SIZES, *PCC_FILE_SIZES;

AllocationSize段并非实际文件实际分配的物理空间。而是适于分配空间的数据量。

AllocationSize被内存管理器代表"section object"的大小。Section object 用于确定一个映射到内存文件，AllocationSize经常是最小为文件大小。

Cache Manager Callbacks

Interactions between the file system and the Cache Manager are manipulated via a series ofcallback functions. These callback functions are registered on a per file basis with the Cache Manager and are then used by the Cache Manager in order to ensure that the data structures are "locked" prior to performing a file system operation.

文件系统和缓存管理器通过一系列callback函数操纵和交互。

Windows NT assumes there is a strict ordering in how resources are acquired between the file system, Cache Manager, and Memory Manager. If followed, this ordering will ensure that deadlocks do not occur. Of course, if it is not followed, deadlocks can (and will) occur. Specifically, file system resources are acquired first. Then Cache Manager resources are acquired. Finally, Memory Manager resources are acquired.

Windows NT 假设在文件系统，资源获取有严格顺序。以保证不发生死锁。特别是文件系统的资源是在第一位获取，然后是缓存管理器资源，最后是内存管理器资源。

Thus, these callbacks are used by the Cache Manager to honor this hierarchy. The callbacks required by the Cache Manager are:

typedef BOOLEAN (*PACQUIRE_FOR_LAZY_WRITE) (
        IN PVOID Context,
        IN BOOLEAN Wait
        );

typedef VOID (*PRELEASE_FROM_LAZY_WRITE) (
        IN PVOID Context
        );

typedef BOOLEAN (*PACQUIRE_FOR_READ_AHEAD) (
        IN PVOID Context,
        IN BOOLEAN Wait
        );

typedef VOID (*PRELEASE_FROM_READ_AHEAD) (
        IN PVOID Context
        );

typedef struct _CACHE_MANAGER_CALLBACKS {
        PACQUIRE_FOR_LAZY_WRITE AcquireForLazyWrite;
        PRELEASE_FROM_LAZY_WRITE ReleaseFromLazyWrite;
        PACQUIRE_FOR_READ_AHEAD AcquireForReadAhead;
        PRELEASE_FROM_READ_AHEAD ReleaseFromReadAhead;
} CACHE_MANAGER_CALLBACKS, *PCACHE_MANAGER_CALLBACKS;

 

注意到callbacks用于Cache Manager 的两个不同部分。首先是Lazy writer有责任写回脏缓存数据到文件系统。第二是为预读功能。

需要保护non-cached用户I/O操作和用户修改文件大小操作。NT文件系统使用两个ERESOURCE结构，从Common header的PERESOURCEResource
PERESOURCE PagingIoResource两个字段指定。Cache Manager并不直接获取资源相反调用文件系统获取必须资源。

static BOOLEAN OwAcquireForLazyWrite(PVOID Context, BOOLEAN Wait)
{
POW_FCB fcb = (POW_FCB) Context;
BOOLEAN result;

// Take out the lock on the file.

result = OwAcquireResourceExclusiveExp(&fcb->Resource, Wait);
if (!result) {

// We did not acquire the resource.

return (result);
}

// We did acquire the resource. We need to:
// (1) Store away the thread id of this thread (for the release)
// (2) Set top level irp to a pseudo value
// In both cases, the previous value should be zero.

OwAssert(!fcb->ResourceThread);
fcb->ResourceThread = OwGetCurrentResourceThread();
return (TRUE);
}

Each of the file systems in the Microsoft IFS kit also contains examples of routines like these. For the Lazy Writer these routines are located in the following locations:

 

File System

File

Routine

FAT

resrcsup.c

FatAcquireFcbForLazyWrite

CDFS

resrcsup.c

CdAcquireForCache

RDR2

rxce\resrcsup.c

RxAcquireFcbForLazyWrite

Other similar routines (for read ahead for example) can be located in the same file.

另外相似例程预读等可在同一个文件找到。

CcCanlWrite

Because an application program can modify data in memory at a rate that exceeds the ability to write the data to disk, the Virtual Memory system can "fill up" with data. This in turn can then cause fatal out-of-memory conditions to occur within the VM system. To avoid this, the file system must cooperate with the VM system to detect these conditions. One of the key operations provided by the Cache Manager for this support isCcCanIWrite. The prototype for this call is:

应用程序可修改内存数据的速率超出写数据到磁盘，虚拟内存系统可被数据填满。

判断是否可以写入数据到缓存。

NTKERNELAPI BOOLEAN CcCanIWrite (
    IN PFILE_OBJECT FileObject,
    IN ULONG BytesToWrite,
    IN BOOLEAN Wait,
    IN BOOLEAN Retrying
    );