Swap activity grows untill disk/system performance degrades ?

[Former Member]

Hi,

I have very serious performance problem. My system:

AIX 5300-05

DB2 8.1 fixpack 14

SAP R/3 Enterprise 620 level 0060

RAM 32 Gb

Swap size 25 Gb

DB size 200 Gb

users 300

For examle let me say that I restarted my system. Users begin to connect and do their work. Performance is good. Slowly but constantly swap activity begins to grow (I used iostat command to monitor rootvg native disks where swap partition is) and after about 3 hours, disk activity reaches 100% and never goes down. In this state system is terribly slow and all users complain a lot. I understand that this is complicated situation but maybe someone experienced similar things before ?

At least where should I start ?

thank you in advance

Vilius

[Former Member]

Hi Vilius,

Try reducing the amount of memory allocated to the Database and SAP System. The problem could be that the entire physical memory is getting consumed and the system still needs more memory which is resulting it to use the Swap space.

The sum of memory alocated to the Database and SAP should not exceed more that the physical memory available, infact leave a few GB of memory free at the OS level would be a good idea.

sappfpar check pf=<instance profile> should tell you the amount of memory consumed by SAP. I guess ST04 should give you the details of memory consumed by DB2 ( haven't worked much with DB2 ).

Try keeping this within the physical memory available - 2 to 3 gb free at OS level. That should help in reducing the Swap usage. This is assuming you have no other memory consuming application running on the same box.

Having lesser memory allocated to SAP is better that having it use the Swap space.

Thanks and Best Regards,

Sunil.

[Former Member]

Hi Sunil very thanks for your reply.

My SAP knowledge is very limited right now so I must be sure what I'm doing.

The following output is from sappfpar command - it contains a lot of memory amount parameters -

whitch one of those is maximum memory allowed ?

Shared memory disposition overview

================================================================

Shared memory pools

Key: 10 Pool

Size configured.....: 248000000 ( 236.5 MB)

Size min. estimated.: 243093652 ( 231.8 MB)

Advised Size........: 246000000 ( 234.6 MB)

Key: 40 Pool for database buffers

Size configured.....: 208000000 ( 198.4 MB)

Size min. estimated.: 203142528 ( 193.7 MB)

Advised Size........: 206000000 ( 196.5 MB)

Shared memories inside of pool 10

Key: 1 Size: 2500 ( 0.0 MB) System administration

Key: 2 Size: 7666144 ( 7.3 MB) Disp. administration tables

Key: 4 Size: 525448 ( 0.5 MB) statistic area

Key: 7 Size: 14838 ( 0.0 MB) Update task administration

Key: 11 Size: 500000 ( 0.5 MB) Factory calender buffer

Key: 12 Size: 6000000 ( 5.7 MB) TemSe Char-Code convert Buf.

Key: 13 Size: 60500000 ( 57.7 MB) Alert Area

Key: 17 Size: 2672386 ( 2.5 MB) Roll administration

Key: 30 Size: 40960 ( 0.0 MB) Taskhandler runtime admin.

Key: 31 Size: 4806000 ( 4.6 MB) Dispatcher request queue

Key: 33 Size: 33792000 ( 32.2 MB) Table buffer, part.buffering

Key: 34 Size: 20480000 ( 19.5 MB) Enqueue table

Key: 51 Size: 3200000 ( 3.1 MB) Extended memory admin.

Key: 52 Size: 40000 ( 0.0 MB) Message Server buffer

Key: 54 Size: 42975232 ( 41.0 MB) Export/Import buffer

Key: 55 Size: 8192 ( 0.0 MB) Spool local printer+joblist

Key: 57 Size: 1048576 ( 1.0 MB) Profilparameter in shared mem

Key: 58 Size: 4096 ( 0.0 MB) Enqueue ID for reset

Key: 63 Size: 409600 ( 0.4 MB) ICMAN shared memory

Key: 64 Size: 4202496 ( 4.0 MB) Online Text Repository Buf.

Key: 65 Size: 4202496 ( 4.0 MB) Export/Import Shared Memory

Shared memories inside of pool 40

Key: 42 Size: 12512992 ( 11.9 MB) DB TTAB buffer

Key: 43 Size: 72795064 ( 69.4 MB) DB FTAB buffer

Key: 44 Size: 34474936 ( 32.9 MB) DB IREC buffer

Key: 45 Size: 8564664 ( 8.2 MB) DB short nametab buffer

Key: 46 Size: 20480 ( 0.0 MB) DB sync table

Key: 47 Size: 21505024 ( 20.5 MB) DB CUA buffer

Key: 48 Size: 300000 ( 0.3 MB) Number range buffer

Key: 49 Size: 2968344 ( 2.8 MB) Spool admin (SpoolWP+DiaWP)

Shared memories outside of pools

Key: 3 Size: 13714400 ( 13.1 MB) Disp. communication areas

Key: 6 Size: 532480000 ( 507.8 MB) ABAP program buffer

Key: 8 Size: 134217828 ( 128.0 MB) Paging buffer

Key: 9 Size: 134217828 ( 128.0 MB) Roll buffer

Key: 14 Size: 15640576 ( 14.9 MB) Presentation buffer

Key: 16 Size: 22400 ( 0.0 MB) Semaphore activity monitoring

Key: 18 Size: 1992804 ( 1.9 MB) Paging adminitration

Key: 19 Size: 71984128 ( 68.6 MB) Table-buffer

Key: 41 Size: 25010000 ( 23.9 MB) DB statistics buffer

Key: 62 Size: 85983232 ( 82.0 MB) Memory pipes

Key: 1002 Size: 400000 ( 0.4 MB) Performance monitoring V01.0

Key: 58900102 Size: 4096 ( 0.0 MB) SCSA area

Nr of operating system shared memory segments: 14

Shared memory resource requirements estimated

================================================================

Nr of shared memory descriptors required for

Extended Memory Management (unnamed mapped file).: 55

Total Nr of shared segments required.....: 69

System-imposed number of shared memories.: 1000

Shared memory segment size required min..: 532480000 ( 507.8 MB)

System-imposed maximum segment size......: 35184372088832 (33554432.0 MB)

Swap space requirements estimated

================================================

Shared memory....................: 1401.6 MB

..in pool 10 231.8 MB, 98% used

..in pool 40 193.7 MB, 97% used

..not in pool: 968.6 MB

Processes........................: 430.8 MB

Extended Memory .................: 14000.0 MB

-

---

Total, minimum requirement.......: 15832.4 MB

Process local heaps, worst case..: 2038.0 MB

Total, worst case requirement....: 17870.4 MB

Errors detected..................: 0

Warnings detected................: 0

Vilius

Edited by: Vilius Mockunas on Oct 14, 2008 7:59 AM

Edited by: Vilius Mockunas on Oct 14, 2008 7:59 AM

Edited by: Vilius Mockunas on Oct 14, 2008 8:10 AM

[malte_schuenemann]

The first question to be answered is what causes the high disk activity. Suspicion of course is paging - can you confirm ?

Next, if there is a lot of paging, then we need to know which component is causing this. To track DB2, I suggest to run

db2mtrk -i -d -p -r 300

This will report usage of various memory areas in 5 minutes intervals. I suggest to report the output in a file. Terminate db2mtrk if the problem is visible and provide the file.

I suggest to add the DB CFG and DBM CFG plus data illustrating the increase in overall memory usage.