cancel
Showing results for 
Search instead for 
Did you mean: 

Strange filesystem (full) problem

Former Member
0 Kudos

On our main server I have a really strange problem (SLES 9 SP3):

# df -h /usr/sap
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p6      21G   20G  331M  99% /usr/sap

If I step into this directory and use "du" to check the usage I get

# du -hsx /usr/sap
2.8G    /usr/sap

With "x" I excluded the (mounted) directories.

Any idea what the problem can be? Stale NFS handle? Any hints are appreciated.

--

Markus

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

Hello Markus,

A common problem I've found with /usr/sap/SID filesystems (since using Java instances), is the block size of the partition (or lvol working with LVM). The "default" block size for a big FS it's OK to use with Oracle datafiles (or other big files), but with the j2ee server there is a huge number of small files, and the real total size is smaller than the "occupied" size in disk.

Can you check your block size with

<b>/sbin/dumpe2fs /dev/cciss/c0d0p6 | grep 'Block size'</b>

...and paste the results?

Best Regards,

JC Llanes.

markus_doehr2
Active Contributor
0 Kudos

Hi Juan,

that filesystem is a "fixed" one - neither using LVM nor ext2, an xfs filesystem:

# mount | grep "/usr/sap "
/dev/cciss/c0d0p6 on /usr/sap type xfs (rw)

# find . -mount -type f | wc -l
47943

I think I'll put that filesystem on a SAN system and copy the content over, this seems to be really strange to me.

--

Markus

Former Member
0 Kudos

Hi Markus,

did you delete some files earlier on this partition?

There might be a problem with files which are still opend by a process.

Search with lsof for deleted but opened files.

like #lsof | grep deleted

Google for this Problem, it is a known issue.

Regards

Manuel

Answers (3)

Answers (3)

hannes_kuehnemund
Active Contributor
0 Kudos

Marked question as "assumed answered".

Former Member
0 Kudos

Hi,

Seems your trace files and stat files increased.

Please using sm50 to reset trace files and using ST03 delete statfiles of the applcation.

With Regards,

Lolla

markus_doehr2
Active Contributor
0 Kudos

thank you for your answer - but as you can see in my initial post, that the directory did not contain more than the given 2.6 GB data.

Problem the issue was a stale NFS handle from one of the application servers.

--

Markus

hannes_kuehnemund
Active Contributor
0 Kudos

Hi Markus,

XFS has a standard block size of 4k. I assume that in average we lose 2k space per file because of the 4k boundaries (e.g. a file wirh 6k allocates 8k on disk, and so on..).

As you have ~47.000 files under /usr/sap, this would only explain a loss of around 100MB, but not of 17.2GB!

df and du handle sparse files in a different way. Where du excludes the free space in sparse files, does df include it. But normally, there aren't any sparse files under /usr/sap.

Maybe the -x option does not work correctly. Do you have any SLES10/Solaris/other Unix system at hand?

Thanks,

Hannes Kuehnemund

SAP LinuxLab

hannes_kuehnemund
Active Contributor
0 Kudos

Additionally, as you exclude mounted directories, I assume that you definitely have them under /usr/sap. Are there any forgotten sparse files in these directories which get "over mounted" somehow?

markus_doehr2
Active Contributor
0 Kudos

Hi Hannes,

thank you for your answer.

I unmounted the share on the application servers, unmounted it locally and remounted it. Now it´s showing "normal" values:

# df -h /usr/sap
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p6      21G  2.8G   18G  14% /usr/sap

So apparently it was really a stale NFS handle (somewhere).

Thanx for your time!

Greetings,

--

Markus