Disk I/O optimization

Disk I/O refers to the number of read and write operations performed by your
application on a physical disk or multiple disks installed in your server. Common
activities that can cause disk I/O-related bottlenecks include long-running file I/O
operations, data encryption and decryption, reading unnecessary data from database
tables, and a shortage of physical memory that leads to excessive paging activity.
Slow hard disks are another factor to consider.

To resolve disk-related bottlenecks:
- Start by removing any redundant disk I/O operations in your application.
- Identify whether your system has a shortage of physical memory, and,
if so, add more memory to avoid excessive paging.
- Identify whether you need to separate your data onto multiple disks.
- Consider upgrading to faster disks if you still have disk I/O
bottlenecks after doing all of above.

Configuration Overview
Microsoft Windows retrieves programs and data from disk. The disk
subsystem can be the most important aspect of I/O performance, but
problems can be masked by other factors, such as lack of memory.
Performance console disk counters are available within both the
LogicalDisk or PhysicalDisk objects.

Metrics
PhysicalDisk
- Avg. Disk Queue Length
- Avg. Disk Read Queue Length
- Avg. Disk Write Queue Length
- Avg. Disk sec/Read
- Avg. Disk sec/Transfer
- Disk Writes/sec


Tuning Options
If you determine that disk I/O is a bottleneck, you have a number of options:
- Defragment your disks. Use the Disk Defragmenter system tool.
- Use Diskpar.exe on Windows 2000 to reduce performance loss due to misaligned
disk tracks and sectors. You can use get the Diskpar.exe from the Windows 2000
Resource Kit.
- Use stripe sets to process I/O requests concurrently over multiple disks. The
type you use depends on your data-integrity requirements. If your applications
are read-intensive and require fault tolerance, consider a RAID 5 volume. Use
mirrored volumes for fault tolerance and good I/O performance overall. If you do
not require fault tolerance, implement stripe sets for fast reading and writing and
improved storage capacity. When stripe sets are used, disk utilization per disk
should fall due to distribution of work across the volumes, and overall throughput
should increase.
- If you find that there is no increased throughput when scaling to additional disks
in a stripe set, your system might be experiencing a bottleneck due to contention
between disks for the disk adapter. You might need to add an adapter to better
distribute the load.
- Place multiple drives on separate I/O buses, particularly if a disk has an
I/O-intensive workload.
- Distribute workload among multiple drives. Windows Clustering and
Distributed File System provide solutions for load balancing on different drives.
- Limit your use of file compression or encryption. File compression and
encryption are I/O-intensive operations. You should only use them where
absolutely necessary.
- Disable creation of short names. If you are not supporting MS-DOS for Windows
3.x clients, disable short names to improve performance. To disable short names,
change the default value of the \NtfsDisable8dot3NameCreation registry entry
(in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Filesystem) to 1.
- Disable last access update. By default, NTFS updates the date and time stamp
of the last access on directories whenever it traverses the directory. For a large
NTFS volume, this update process can slow performance. To disable automatic
updating, create a new REG_DWORD registry entry named
NtfsDisableLastAccessUpdate in HKEY_LOCAL_MACHINE\SYSTEM\CurrentContolSet\Control\Filesystem and set its value to 1.
Reserve appropriate space for the master file table. Add the
NtfsMftZoneReservation entry to the registry as a REG_DWORD in
HKEY_LOCAL_MACHINE \SYSTEM \CurrentControlSet\Control \FileSystem.
When you add this entry to the registry, the system reserves space on the volume
for the master file table. Reserving space in this manner allows the master file
table to grow optimally. If your NTFS volumes generally contain relatively few
files that are large, set the value of this registry entry to 1 (the default).
- Typically you can use a value of 2 or 3 for moderate numbers of files, and use a value of 4 (the maximum) if your volumes tend to contain a relatively large number of files. However, make sure to test any settings greater than 2, because these greater values cause the system to reserve a much larger portion of the disk for the master file table.
- Use the most efficient disk systems available, including controller, I/O, cabling,
and disk. Use intelligent drivers that support interrupt moderation or interrupt
avoidance to alleviate the interrupt activity for the processor due to disk I/O.
- Check whether you are using the appropriate RAID configuration. Use RAID 10
(striping and mirroring) for best performance and fault tolerance. The tradeoff is
that using RAID 10 is expensive. Avoid using RAID 5 (parity) when you have
extensive write operations.
- Consider using database partitions. If you have a database bottleneck, consider
using database partitions and mapping disks to specific tables and transaction
logs. The primary purpose of partitions is to overcome disk bottlenecks for large
tables. If you have a table with large number of rows and you determine that it is
the source of a bottleneck, consider using partitions. For SQL Server, you can use
file groups to improve I/O performance. You can associate tables with file groups,
and then associate the file groups with a specific hard disk.
- Consider splitting files across hard disks. If you are dealing with extensive
file-related operations, consider splitting the files across a number of hard disks
to spread the I/O load across multiple disks.