2

I have a problem with my VPS and disk I/O. My server is running nginx + PHP-FPM + APC. The database is located on another dedicated VPS. I have several WordPress MU sites living on the web server. The average I/O rate is 6k block/second.

I'm trying to understand what's causing the high I/O.

Output of 'free -m':

            total   used   free   shared   buffers   cached
Mem:         1005    973     31        0        96      568
-/+ buffers/cache:   307    697
Swap:         255      8    247

Output of 'iotop':

Total DISK READ: 0.00 B/s | Total DISK WRITE: 3.90 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 2150 be/4 root        0.00 B/s    0.00 B/s  0.00 % 65.25 % [flush-202:0]
 6694 be/4 www-data    0.00 B/s   19.64 K/s  0.00 %  0.00 % php-fpm: pool www
 6700 be/4 www-data    0.00 B/s   23.56 K/s  0.00 %  0.00 % php-fpm: pool www
 8646 be/4 www-data    0.00 B/s  424.12 K/s  0.00 %  0.00 % php-fpm: pool www
10974 be/4 www-data    0.00 B/s   19.64 K/s  0.00 %  0.00 % php-fpm: pool www

The 'flush-202:0' process sometimes hits I/O of 99%. I've read this is the disk cache flushing process, but what causes it to run and how do I fix it?

Peter Mortensen
  • 2,327
Mem
  • 311
  • 2
  • 4
  • 5

3 Answers3

2

I'm not sure that iotop sample shows something unusual. It's not a problem for the flush process to be a high percentage of your I/O at any point in time if there isn't much I/O going on at that time.

I would install atop, which can present real-time data like iotop, but has the advantage of also logging samples throughout the day. A day after installing it, I would open the logged data with atop -r log_filename, then go through the samples with t until I found times when the I/O reported in the system-level output is high. Then I would switch the per-process output to disk with d to see what processes were generating the I/O activity.

Peter Mortensen
  • 2,327
sciurus
  • 12,958
  • 3
  • 33
  • 51
1

This may be a problem with APC.

If it works for you, in your PHP configuration, set:

apc.mmap_file_mask = /tmp/apc.shm.XXXXXX

If that doesn't work, remove the apc.mmap_file_mask setting altogether.

If not APC, then it's something else that uses disk backed caching. Like virtual memory for instance, or a varnish cache, or maybe something that uses DBM files. There are many possibilities there. Maybe even a database engine.

Users of APC should probably now migrate to Zend's opcache which comes with later free distributions of PHP (and earlier paid ones as I recall). Looking at the tuning of opcache might still be significant for managing your I/O load. https://www.sitepoint.com/understanding-opcache/ is a useful introduction, and links to several tools which can be used to check your cache performance (particularly the hit ratio).

Peter Mortensen
  • 2,327
mc0e
  • 5,979
0

You can achieve this with the pidstat program. Some distributions don't come with it installed. But you can download the sysstate package from here and compile it. Don't install it, but copy out the pidstat it compiles (or just run it in the current directory). You can pass the '-d' flag to get the output you want.

Peter Mortensen
  • 2,327
Matthew Ife
  • 24,261