Storage Performance Testing
amps_bio_perf_test
AMPS contains a utility, amps_bio_perf_test, that measures the raw performance of storage devices for the type of sequential writes (and, optionally, reads) that are used by the AMPS transaction log.
The utility can be used to get a sense of the maximum throughput that a given device offers, and to compare one device to another.
Options and Parameters
| Option | Description |
|---|---|
-f, --file | Required. Path of the data file to create on the device to test. The file is allocated before the timed test begins and contains generated data. |
-b, --batch_size | Number of messages in each write batch, from 1 through 256. The default is 256 when neither batch size nor message size is specified; otherwise the value can be derived from message_size or adjusted to produce 4KB-aligned writes no larger than 128KB. |
-m, --message_size | Message size in bytes, from 256 through 128KB and a multiple of 256. The default is 256 when neither message size nor batch size is specified; otherwise the value can be derived from batch_size. |
-s, --file_size | Requested data file size in MB, from 1 through 8192. The default is 1024MB. The actual size is rounded down to contain only complete write batches. |
-r, --read_mix | Requested percentage of read operations, from 0 through 99. The default is 0. |
-u, --units | Units for write latency statistics: auto, ms, us, or s. The default is auto. In auto, each statistic is shown as integer milliseconds when it is at least 1ms, or integer microseconds otherwise. Explicit ms and us values are integers; s values have nine digits after the decimal. |
-h, --help | Print command-line help and exit. |
60East recommends running the amps_bio_perf_test tool on a system that is otherwise quiet. The --file specified is the file that the tool creates for writing data, so that file must be on the device to be measured.
The amps_bio_perf_test uses the O_DIRECT flag for writes (as does the AMPS transaction log writer), bypassing caching at the operating system level. Since tmpfs does not support O_DIRECT, the utility disables it for files under /dev/shm or /run/shm.
The amps_bio_perf_test tool attempts to fully use the I/O capacity of the target device. Do not run this tool on a system that is being used for another purpose, since the tool may impact the performance of any other process using the device being tested.
Output
The amps_bio_perf_test tool produces the following metrics:
| Metric | Explanation |
|---|---|
| elapsed time | Time in seconds during which the tool was actively writing and, when requested, reading data. |
| writer thread count | Number of threads writing data (always 1). |
| batch count | Number of messages included in each submitted write. |
| message byte count | Size of each generated message in bytes. |
| submit byte count | Total number of bytes in each submitted write (batch count multiplied by message byte count). |
| submit count | Total number of messages submitted during the test. |
| read count | Number of read operations performed during the test. |
| write count | Number of batch write operations completed during the test. |
| file byte count | Actual number of bytes written to the test file. |
| submit rate | Average number of messages submitted per second during the timed test. |
| write rate | Average number of batch writes completed per second during the timed test. |
| write byte rate | Aggregate write throughput, reported in MB/sec or GB/sec. |
| actual read mix | Percentage of read operations among the completed reads and writes. This can differ from the requested mix due to thread scheduling. |
| write latency mean | Arithmetic mean of the batch write completion latencies. |
| write latency stddev | Sample standard deviation of the batch write completion latencies. |
| write latency min | Minimum batch write completion latency. |
| write latency Q1 | First quartile (25th percentile) of the batch write completion latencies. |
| write latency median | Median (50th percentile) of the batch write completion latencies. |
| write latency Q3 | Third quartile (75th percentile) of the batch write completion latencies. |
| write latency max | Maximum batch write completion latency. |
Latency statistics use the units selected by --units. With the default auto setting, each statistic selects its unit independently, so a single report can contain both ms and us values.
Tips on Evaluating Overall Performance
The amps_bio_perf_test tool tries to simulate the AMPS transaction log workload under ideal conditions for maximum throughput. Before the timed test begins, the tool preallocates the file, prepares a generated data pattern, and allocates the write buffers and latency records. During the timed part of the test, it copies the pattern into the preallocated write buffer, submits writes, and records their completion latencies.
The submit rate is an estimate of the maximum number of messages per second that the device could sustain for the selected message and batch sizes under these test conditions. To approach this rate, the payload and metadata for each application message would need to fit within the configured message size, and network ingress would need to keep pace with storage without gaps in message flow or pushback from the device.
The write latency summary describes how long each batch write takes to complete. These measurements affect the overall latency of the system and are strongly correlated with performance (as reflected in the write rate). The quartiles and standard deviation also show variation that would be hidden by the mean alone.
For example, consider a result that shows a "submit rate" of 2500000 for 512 byte messages. This means that this device could (in principle, under absolutely ideal conditions), support a maximum write throughput of:
| 512 bytes | 2,500,000 per second |
|---|---|
| 1024 bytes (1KB) | 1,250,000 per second |
| 10KB | 125,000 per second |
| 100KB | 12,500 per second |
These are, of course, approximations of the expected maximum throughput. In practice, message sizes rarely fit perfectly into a batch, application activity sees bursts and lulls, the number of threads reading from the transaction log varies based on activity (AMPS tries to consolidate replays from the transaction log into the minimum number of threads to save CPU and I/O bandwidth), and the transaction log device may be used for other traffic (for example, SOW topic storage, statistics, or error and event logging) that may share I/O bandwidth. These factors (and others) can affect the actual performance in deployment.
Usage
60East typically runs the amps_bio_perf_test tool with a command line similar to the following:
$ amps_bio_perf_test -b 128 -m 512 -s 8192 -r 0 -u auto -f /mnt/fastdrive/ampsdir/bio.data
This provides a good baseline for the maximum write performance of the system.