0

I've created two zfs pools (one with ssds, the other rust) and run bonnie against each to try to understand their performance but I find the output somewhat confusing as the performance seems practically identical. I was wondering if I'm misinterpreting the output or if someone could offer advice on what might be going on?

Both pools have a single mirror vdev containing two disks.

SSD Pool - 2 x 500GB Samsung 850 Pro, ashift=13

bonnie -u root -d /ssdpool/images/

Version      1.98   ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
is7734 126G::8192:5  177k  99  475m  97  438m  98  376k  99  1.4g  99 +++++ +++
Latency             69204us     100ms     104ms   33890us     925us    6948us
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
      files:max:min  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
is77346          16 16845  98 +++++ +++  2463  10 +++++ +++ +++++ +++ 22164  98
Latency              6213us     708us    5089ms    2565us      42us     910us

bonnie++ -d /ssdpool/images/ -s 128G:4k -u root

Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
is77346     128G:4k  180k  99  343m  98  305m  99  396k  99  989m  99 +++++ +++
Latency             31352us     102ms     101ms   16948us     952us   12772us
Version  1.98       ------Sequential Create------ --------Random Create--------
is77346             -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16     0  98 +++++ +++ 724249376  10 +++++ +++ +++++ +++ 724249376  99
Latency              2523us     488us    5379ms    2562us      92us     926us


fio --name=random-write --ioengine=sync --iodepth=4 --rw=randwrite --bs=4k --direct=0 --size=256m --numjobs=16 --end_fsync=1
[...]
Run status group 0 (all jobs):
  WRITE: bw=212MiB/s (222MB/s), 13.2MiB/s-15.5MiB/s (13.9MB/s-16.2MB/s), io=4096MiB (4295MB), run=16565-19359msec

HDD Pool - 2 x 4TB WD Green, ashift=12

bonnie -u root -d /hddpool/images/

Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
is77346        126G  166k  94  391m  86  345m  82  387k  99  1.4g  99 11676 399
Latency             97954us     101ms     105ms   39728us     939us   10904us
Version  1.98       ------Sequential Create------ --------Random Create--------
is77346             -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16     0  73 +++++ +++ 724249376   7     0  78 +++++ +++ 724249376  16
Latency               100ms     667us    6853ms    3384us      20us     563ms

bonnie++ -d /hddpool/images/ -s 128G:4k -u root

Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
is77346     128G:4k  166k  99  315m  96  268m  96  355k  99  979m  99  6563 259
Latency             32958us     101ms     103ms   26880us    1019us   11142us
Version  1.98       ------Sequential Create------ --------Random Create--------
is77346             -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16     0  92 +++++ +++ 724249376   7     0  81 +++++ +++ 724249376  18
Latency              2562us     949us    6462ms    3451us      31us     626ms

fio --name=random-write --ioengine=sync --iodepth=4 --rw=randwrite --bs=4k --direct=0 --size=256m --numjobs=16 --end_fsync=1
[...]
Run status group 0 (all jobs):
  WRITE: bw=33.0MiB/s (35.6MB/s), 2175KiB/s-2623KiB/s (2227kB/s-2686kB/s), io=4096MiB (4295MB), run=99950-120548msec

In case it's also worth mentioning the other config in the system:

  • OS is Ubuntu 20.04 running on SSD
  • 64GB RAM
  • both pools use: atime=off, recordsize=64k, compression=lz4
  • I haven't explicitly configured zil or l2arc so unless they use the OS disk by default then they won't be configured.

On @francisaugusto suggestion in the comments I ran fio (see results above) and it shows a significant difference between the two pools. I'm still unsure why bonnie gives such similar results but I feel reassured that things are working properly. If anyone is reading this in the future, maybe give bonnie a miss and start out with fio.

James
  • 325

0 Answers0