Performance: Disk Accesses

Optimized disk access is one of the greatest areas of advantage of the PostPath Server™ over Exchange™.

Rather than use a database, the PostPath Server™ writes data directly into the filing system, using a one-file-per-message approach, allowing clear visibility into disk operations and enabling careful optimization of performance.

The design goals included minimizing disk reads, minimizing disk writes, and especially minimizing disk-head seeks.

Optimizations adopted include:

One-shot read and write. Top level message files are always read in a single shot and written in a single shot, because it is almost as expensive to read or write one byte as it is to read or write several blocks of 10KB. Larger streaming files are read/written in 64KB blocks.
Fragmentation avoidance. Modern filing systems almost never fragment files smaller than 32KB and hardly ever fragment files smaller than 64KB. To avoid disk seeks caused by fragmentation, top-level message files are usually less than 32KB large and always less than 64KB, with larger properties extracted into their own streaming files as and when required.
File-length and file-name overloading. Certain information is encoded in the low bits of the file length (by rounding file size appropriately) and in the filename to enable that information to be retrieved very fast without requiring a file to be opened and read.
inode-use. Hand-chosen message information is included in each inode (the inode tree is the logical tree of disk blocks that the filing system uses to keep track of what files are where on the disk) so that information about each file can be retrieved merely by scanning a directory without having to open and read individual files.
Simplified indexes. In many index implementations, a lot of information is encoded per index row, so that a relatively small number of rows can be retained in memory. In the PostPath implementation, memory required to encode each row is small, so that very large numbers of rows can be retained in memory simultaneously, enabling high speed index traversal. The PostPath Server™ also uses lazy update (no update until index is accessed), and is able to make use of the inode information to avoid file-opens during updating of indexes. Each index is stored as a file in the folder to which it refers; the small size of the indexes also makes saving and restoring indexes fast, and is efficient from a disk-usage point of view.
Request throttling and elevator-filling. Where allowing complex disk requests to occur in parallel would slow down the system (because disk heads would spend time seeking backwards and forwards servicing fragments of each request) the number of requests allowed to reach the disk controller is reduced, causing all the requests (even the ones held off) to complete sooner. Conversely, in some other situations many simple requests are let through simultaneously, allowing the disk controller to service them with only short disk-head-seeks between each one (with many requests outstanding, the disk head operates an elevator algorithm, sweeping from one side of the platter to the other, servicing all requests as it goes; since there are many requests the distance between requests that are serviced consecutively on the platter is smaller, and hence disk head seeks are relatively fast).
Linking/single-instance behavior. Rather than copy files, the server links to them wherever possible. A link generally saves at least one seek and one write vs. copying, and can save many writes, depending on the file size. Of course, linking saves disk space as well as optimizing throughput.

Efficient disk access enables substantial savings on disk storage subsystems.