SAP ECC6 memory and paging issues
I have recently upgraded my 4.6C systems to an ECC 6 system (DB2 LUW 9.5 on AIX 5.3 TL9 64 Bit OS)
I have been running the LPAR with 14 GB of memory and we are around 100-200+ users using the system, I was monitoring using nmon and found that Physical Memory was around 99.8% Used (14311.8MB and 22.6MB was free) also the paging space was around 37.2% in result causing the system at times to run slow which can have a very negative effect on the users.
After further investigation I found that after a system restart the Physical Memory would start around 50.9% and increased at a steady pace until it reached 99.8% that is when the system would start using the paging space which would steadily increase, I found that the only solution was a system restart at least once a week to reduce the memory consumption.
At first glance it looked like a database manger memory leak with the process db2sysc, so I searched the net with the search words u201Cdb2 memory leaku201D and found the following APARs and notes.
APAR JR30285 - Pervasive memory leak when compiling SQL statements that use SQL/XML functions
APAR IZ35230 - There is a pervasive unix-specific private memory leak in the security component
Note 1288341 - Memory leak in APPLHEAPSZ -> SQL0954C
Note 1352361 - Memory leak in shared memory area abrfci
Note 1147821 - DB6: Known Errors and available Fixes in DB2 9.5 LUW
After reading the notes and APARs I decided to updated DB2 to the latest fix pack (5SAP), but after the fix pack was implemented it did not solve the memory problem
I started look at different problems with SAP ECC6, db2 and AIX with paging/memory problems and I found the following notes to do with AIX memory and paging but none of them helped as all parameters and settings were set accordingly
789477 - Large extended memory on AIX (64-bit) as of Kernel 6.20
191801 - AIX 64-bit with very large amount of Extended Memory
973227 - AIX Virtual Memory Management: Tuning Recommendations
884393 - AIX saposcol consumes large amount of memory.
856848 u2013 AIX Extended Memory Disclaiming
1048686 u2013 Recommended AIX settings for SAP
1121904 u2013 SAP on AIX: Recommendations for Paging
1086130 u2013 DB6: DB2 Standard Parameter Settings
After even more investigation I found the following evidence suggesting AIX Virtual Memory Manager might have a problem