Wednesday, December 4, 2013

Maximum Number of Files in a Single Directory for Netapp NFS mounts on Linux: maxdirsize

Hit file number limit on Netapp NFS mount.

The parameter is [maxdirsize]

Starting with Data ONTAP 6.5, the maximum number of subdirectories a single directory may have is 99998 (100K). Data ONTAP 6.4 and earlier versions were restricted to 65534 (64K) subdirectories. This number may not be changed. To understand the reason for this limit, see the section below on hard links and subdirectory implementation.

And the size of it also has limit which is 1% of Memory.
It has too be less than 80 MB on a 8GB Memory Netapp unit.

You can increase this parameter on line for individual volume.

For the best performance:

# The most important point is to avoid a high number of files in a single directory.
# Create sub-directory structures and place files at the bottom of the directory tree.
# Less than 1000 files per directory is ideal.
# Avoid deep directory structures.
# A depth of less than 5 is ideal, anything above 8 or 9 results in poor performance.

What are the performance impacts of changing the size of maxdirsize?

Performance issues are hard to quantify, but easy to state in a general sense. Lookups in a large directory consume lots of CPU. An additional
performance impact is that when a directory is loaded into memory, the entire directory tree is loaded. Parts of it may fall out of memory
through non-use, but there is a performance impact from reading from disk and finding space in-memory for the directory to be stored.
Hope this helps ,

References:
http://serverfault.com/questions/76018/maximum-number-of-files-in-a-single-directory-for-netapp-nfs-mounts-on-linux
http://source.kohlerville.com/2009/01/post-about-wafldirsizemax/
https://communities.netapp.com/message/5790;jsessionid=7D859DAC2D127BAF4C4D53A5157E060D

Emergency Solution:


find /home/lambert/backup -mtime +{days back} | xargs rm

1 comment:

  1. I feel that is one of the most important information for me.
    And i am satisfied reading your article. But wanna statement on some basic issues, The site style is ideal,
    the articles is truly great : D. Excellent activity, cheers

    ReplyDelete