Server Fault
Server Fault

Questions tagged [fio]

20 questions
9
votes
1 answer

Understanding ZFS NVMe benchmarks with FIO

I'm trying to benchmark an all-NVMe ZFS disk array. I'm familiar with extremely fast benchmark results, with very little disk activity due to efficient ZFS caching. I ran into the reverse: massive amount of ZFS disk activity, but FIO shows only…
Frank Vermeulen
  • 167
7
votes
1 answer

why read is faster when using O_DIRECT flag?

I copied a 10GB of file in my SSD which has read bandwidth of around 3.3GB/s, benchmarked using fio command. Here is the ref: https://cloud.google.com/compute/docs/disks/benchmarking-pd-performance I cleared out the cache using this "sync; echo 3 >…
Abdul Wadood
  • 73
7
votes
0 answers

Write performance is 5 times worse with LUKS on top of mdadm RAID10 than without LUKS

I have servers with many NVMe disks. I am testing disk performance with fio using the following: fio --name=asdf --rw=randwrite --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --size=10G --runtime=60 --group_reporting For a single disk, LUKS…
tacos_tacos_tacos
  • 3,260
6
votes
1 answer

fio 3.23 core dumps when bench-marking many small files

I have been asked to come up fio benchmark results for this test dataset: 1048576x1MiB. So, overall size is 1TiB. The set contains 2^20 1MiB files. The server runs CentOS Linux release 7.8.2003 (Core). It has sufficient RAM: [root@tbn-6 src]# free…
foss4me
  • 63
4
votes
1 answer

Terribly slow I/O on NVMe mdadm RAID array

I have a AMD EPYC 7502P 32-Core Linux server (kernel 6.10.6) with 6 NVMe drives, where suddenly I/O performance dropped. All operations takes too much time. Installing package updates takes hours instead of seconds (maybe minutes). I've tried…
Tombart
  • 2,523
2
votes
0 answers

fio gives an unexpected IOPS number

While running tests with fio, we encountered unexpected behavior of it. Test…
it800
  • 21
2
votes
1 answer

During continuous process of fio test, SSD sequential read bandwidth will slows down

I am using fio to test NVMe SSD max sequential read bandwidth, following is my fio command arguments. fio -direct=1 -filename=/dev/nvme0n1 -name=nvme0n1-read_bw -rw=rw -ioengine=io_uring -rwmixwrite=0 -iodepth=256 -numjobs=1 -bs=128k…
pan renhua
  • 21
2
votes
1 answer

Really slow FIO random 128 MB read values for a RAID 10 XFS

I've just installed two Samsung 970 EVO Plus drives, and configured the biggest partition as RAID 10 (far 2 layout). The problem is FIO only reports a read speed of 72 MB/s. I used the following command to build the array; mdadm --create --verbose…
James
  • 133
2
votes
0 answers

FIO test results wildly vary based on number of jobs

Take the following 3 commands: fio --name=write_throughput --numjobs=8 \ --size=10G --time_based --runtime=60s --ramp_time=2s --ioengine=libaio \ --direct=1 --verify=0 --bs=1M --iodepth=64 --rw=write \ --group_reporting=1 This results in a write…
KdgDev
  • 343
2
votes
1 answer

Interpreting fio test result

I ran fio on a VM provided by my VPS provider and got this result: $ fio --name=random-write --ioengine=posixaio --rw=randwrite --bs=4k --numjobs=1 --size=4g --iodepth=1 --runtime=60 --time_based --end_fsync=1 [--- snipped ---] Run status group 0…
pepoluan
  • 5,248
2
votes
2 answers

AWS XFS performance issues. Stripped setup vs single disk setup

I wanted to share an issue (I guess I may be misunderstanding some concepts) that I'm facing with some benchmarks I'm doing to XFS setups, as we are going to recently migrate a service to a new instance and we would like to have the max. amount of…
David Barranco
  • 149
1
vote
0 answers

Performance Issues on Broadcom 9600W-16e Controller When Using Disk Shelves Drop Perf

I can't understand why this is happening. I have a Broadcom 9600W-16e controller. There are two disk shelves, and the controller is connected with two SAS 4 cables to the disk shelf expander. From disk shelf 1, another shelf is connected with two…
Олег Баталов
  • 21
1
vote
1 answer

Big difference in fio randwrite my result and datasheet result for disk

I'm doing a fio nvme test on a Samsung gen 4 disk. The test results are similar to those in the datasheet, but randwrite differs by almost 5 times. I took the settings for the test from the datasheet 4KB (4,096 bytes) of data transfer size in Queue…
Олег Баталов
  • 21
1
vote
1 answer

How to read and decode fio --bandwidth-log?

I'm Looking for assistance with reading and decoding fio --bandwidth-log. I've run the below command and the output includes a few columns as listed below, how to read and decode each column? fio --invalidate=1 --filename=/dev/nvme0n1 --direct=1…
Or Lapid
  • 11
  • 1
  • 2
1
vote
1 answer

`fio` does not timeout even specified?

# fio --name=random-write --directory=/mnt/test/ --ioengine=posixaio --rw=randwrite -bs=4k --numjobs=1 --size=4g -iodepth=1 -runtime=600 --time_based --end_fsync=1 random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)…
HCSF
  • 255
1
2