File system for a lot of small files

Question

I'm using NetApp device as a NAS storage. I have a lot of small files (30-60k) and dirs. For example I have: 30000 dirs with another 1000-2000 dirs.

A lot of dirs are empty, but most of them have many files (30k).

Should I change block size on Ext3?

Answer 1

If you're using the NetApp as a NAS, i.e. using NAS protocols such as NFS, CIFS/SMB, http etc. then you don't get to choose the filesystem, it uses NetApp's WAFL file system, you can't change it.

If you're using the NetApp to provide block-level sharing such as iSCSI/Fibre-Channel/FCoE then it's irrelevant that it's on a NetApp - you can assume it's on local disk for all intents. In which case please use our search function as this kind of question has been asked many times before.

Answer 2

Based on your requirements, I would recommend ReiserFS. Benchmarks show that ReiserFS performs optimally with a large number of small files compared to other file systems. Ext3 is far from optimal for your scenario, even with a smaller block size.

However, as has already been mentioned in one of the other answers, if you're using a NAS protocol you probably won't be able to choose the file system. If you're using it as a block device, take a look at the following extract from the Linux File System Primer:

EXT2

• Recommended to move to EXT3
• Not Journaled
• POSIX access control

EXT3

• Most popular Linux file system, limited scalability in size and number of files
• Journaled
• POSIX extended access control

JFS

• High performance and scalability
• Journaled
• POSIX extended access controls

ReiserFS

• Best performance and scalability when number of files is great and/or files are small
• Journaled
• POSIX extended access controls

XFS

• Best for extremely large file systems, large files, and lots of files
• Journaled (an asymmetric parallel cluster file system version is also available)
• POSIX extended access controls