1

I'm doing a fio nvme test on a Samsung gen 4 disk. The test results are similar to those in the datasheet, but randwrite differs by almost 5 times. I took the settings for the test from the datasheet 4KB (4,096 bytes) of data transfer size in Queue Depth=32 by 4 workers

The results I get

Sequential Read 6741 MB/s
Sequential Write 4027 MB/s
Random Read 1056K IOPS
Random Write 920K IOPS 
rand_write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.34
Starting 4 processes
Jobs: 4 (f=4): [w(4)][100.0%][w=3585MiB/s][w=918k IOPS][eta 00m:00s]
rand_write: (groupid=0, jobs=4): err= 0: pid=419160: Thu Jun 27 15:10:57 2024
  write: IOPS=929k, BW=3628MiB/s (3804MB/s)(425GiB/120001msec); 0 zone resets
    slat (nsec): min=1105, max=4923.5k, avg=2131.83, stdev=1574.82
    clat (usec): min=11, max=15470, avg=135.25, stdev=40.34
     lat (usec): min=13, max=15554, avg=137.38, stdev=40.54
    clat percentiles (usec):
     |  1.00th=[  109],  5.00th=[  117], 10.00th=[  121], 20.00th=[  125],
     | 30.00th=[  129], 40.00th=[  133], 50.00th=[  135], 60.00th=[  137],
     | 70.00th=[  139], 80.00th=[  141], 90.00th=[  143], 95.00th=[  153],
     | 99.00th=[  231], 99.50th=[  247], 99.90th=[  310], 99.95th=[  408],
     | 99.99th=[ 1221]
   bw (  MiB/s): min= 2110, max= 4220, per=100.00%, avg=3631.51, stdev=72.23, samples=956
   iops        : min=540390, max=1080339, avg=929666.36, stdev=18491.86, samples=956
  lat (usec)   : 20=0.01%, 50=0.01%, 100=0.31%, 250=99.28%, 500=0.38%
  lat (usec)   : 750=0.02%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=21.24%, sys=61.52%, ctx=19177737, majf=0, minf=90
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,111451144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=3628MiB/s (3804MB/s), 3628MiB/s-3628MiB/s (3804MB/s-3804MB/s), io=425GiB (457GB), run=120001-120001msec


Disk stats (read/write):
  nvme0n1: ios=92/111222969, merge=0/0, ticks=10/13512913, in_queue=13512924, util=99.87%

The results are in the datasheet

Random 4KB Read  (Up to) IOPS 1000K
Random 4KB Write (Up to) IOPS 200K

Random performance in Table 3 was measured by using FIO 2.7 in Linux Gen3: CentOS6(kernel 3.14), Gen4: CentOS7(Kernel 5.3.1) with 4KB (4,096 bytes) of data transfer size in Queue Depth=32 by 4 workers. Measurements were performed on a full Logical Block Address (LBA) span of the drive in sustained state. The actual performance
may vary depending on use conditions and environment.


Sequential 128KB Read  (Up to) 6700
Sequential 128KB Write (Up to) 4000


Sequential performance in Table 4 was measured by using FIO 2.7 in Linux Gen3: CentOS6(kernel 3.14), Gen4: CentOS7(Kernel 5.3.1) with 128KB (131,072 bytes) of data
transfer size in Queue Depth=32 by 1 worker.

The job I use.

[global]
ioengine=libaio
iodepth=32
blocksize=4K
direct=1
buffered=0
runtime=120
numjobs=4
time_based
filename=/dev/nvme0n1
stonewall
group_reporting

[rand_write]
rw=randwrite
Lingathoti Ashalatha
  • 49
  • 2
Олег Баталов
  • 21

1 Answers1

-1

Your fio result is random 4k writes. That spec sheet is sequential 128k writes. Very different, not comparable.

When IOPS is a limiting factor, a larger block size will be more data transferred per second.

Making this benchmark relevant to your capacity planning of real applications is another matter:

  • What is the read/write distribution? 100% writes seems unlikely.
  • Does the application use Linux libaio?
  • File system or no?
  • How many IOPS do you require for good performance? Hundreds of thousands of random IOPS is sufficient for some databases with a very large number of concurrent users.
John Mahowald
  • 36,071