Maximum Number of Files in a Single Directory for Netapp NFS mounts on Linux

Question

On an older Linux box I have, there is an NFS mount of a Netapp. Once there are 100k files in a single directory on this box, files can no longer be written. Anyone know what might be causing this? I have been told symbolic links can still be written.

I will add more specific information as I look it up. Please, skip the 'use a better a file / folder structure and/or database', I know ...

Answer 1

There are two limits that you may have hit.

• inodes (unlikely)

  You've consumed all of the available inodes for that volume. You can confirm this with the commands df -i and maxfiles.

• maxdirsize (more likely)

  ONTAP imposes a limit on size of directories. Size being a product of metadata/hardlinks, not file content. This limit defaults (assuming >6.5) to 1% of your system RAM. It does so to ensure that large directories don't impact system performance. Because linear directory scans require directory data structures to be loaded into memory. Quick overview:

  • You can check a directory size with ls -lkd.
  • Raising maxdirsize for a volume is a bit of a one way operation.
  • Only raise it in small increments.
  • You can't reduce a directory's size by deleting it's contents.
  • I have these two bookmarked links which contain a lot more information.
    • NOW: What is maxdirsize?
    • Communities: maxdirsize Recommended Setting and Performance Implications

If it's neither of these issues then I'd recommend checking and posting your ONTAP logs.

Answer 2

The system is hitting the link limit for a directory that is a fixed limitation of the NetApp ONTAP filesystem. From this knowledge base article:

Starting with Data ONTAP 6.5, the maximum number of subdirectories a single directory may have is 99998 (100K). Data ONTAP 6.4 and earlier versions were restricted to 65534 (64K) subdirectories. This number may not be changed. To understand the reason for this limit, see the section below on hard links and subdirectory implementation.

Answer 3

http://www.unixunderground.com/blog/?cat=11

A bit dated, but might be helpful. I believe the number of files is based on the number of disks/nodes.

Answer 4

On newer versions of NetApp (8.1+) you can overwrite the default limits per volume using this command:

vol options vol_name maxdirsize 150000

To see current settings run

vol options vol_name

There is no way to change these from the GUI client.