Abnormal "% system ASP used" growth when running SAP after V5R4 upgrade

[franco_tassone]

Dear friends,

apparently again issues with the V5R4 upgrade. Let me summarize:

• upgraded sucessfully from V5R3 to V5R4

• II14126 successfully applied (8 October 2007)

• no problems running SAP at all (we run multiple instances with 46C and 47, so kernel 46C 2341 and kernel 640 174)

When the instances are running, we see an abnormal (too fast!) storage consumption, never seen anything similiar before... the journals are ok, we checked them all, also the non-SAP ones...

Any idea/suggestion will be sincerely appreciated.

Franco.

[jo_degraeve]

Hello

anyone knows if this issue is solved with the latest V5R4 cumpack ?

I don't find the PTF SI28358 (mentioned as a solution in this thread) in the SAP APAR II14126 ?

[franco_tassone]

Hi,

the definitive solution came from IBM with the 2nd re-spin (yeah they changed this PTF on the fly) of PTF SI29796.

This PTF went GA about 1 week ago.

Go for this PTF, this is the definitive solution for V5R4 and TEMPSTORAGE.

Cheers.

Franco.

[Former Member]

Yes, SI29796 definitely solved our Temp Stg and QXDA API errors in a iSeries/Linux landscape

Bernd

[franco_tassone]

Hi all,

definitivelly this is what IBM told us, PTF SI28358 will soon go in general availability. I can confirm that this issue arises only for those (like us) who have a mixed environment (eg: win app server+CI on iSeries).

Franco.

[franco_tassone]

Mark/all

sorry for this late reply, however IBM provided us an 'experimental' PTF (and they were aware of the TEMPSTORAGE) issue.

We've tried the PTF in our dev environment and it worked on our dev iSeries! The PTFthey provided us is the SI28358 and it is NOT in general availability. IBM has in plan to put this one in general availability very soon.

If you want to keep the TEMPSTORAGE under control, just use the woraround directions in this thread,.

When we'll apply the PTF in prod, we'll let you know the final resoults and close the thread.

Cheers and thank you to you all.

Franco.

[Former Member]

Hi Franco,

Did the extended memory problem ever get resovled? We upgraded to V5R4 two weekends ago and have also seen an increase in the rate of extended memory consumption. It's not the rapid growth that you're seeing but there is a definite increase. We're running ECC 5.0 on two i550s linked via opticonnect. One server is the database while the other functions as our sole application server. Most of our growth comes from the APIA* jobs in QSOC which link to the WPs on the application server. We also are working with IBM with a PMR.

thanx

Mark

[franco_tassone]

Volker,

so far we're continuing to work in prod using the WP's restart every 24 hours, no user complaining for performances or anything else... and we've manage to keep the tempstorage under control, it isn't growing bigger that 100Gb!

We have an open PMR with IBM, and YES it is definitivelly a PTF issue: it seems that there are other customers with 46C and V5R4 (last apar) that have the tempstorage issue.

We're waiting for them to provide a solution... At this point I'd discourage the SAPlings to go with SF99540 7226, stay lower if you can.

Apparently this problem does not affect the SAP 47 and ECC...

We'll keep the group posted.

Franco.

[Former Member]

Hi France,

I am not sure about the thing about SF99540. We upgraded to V5R4 at the beginning of the year and we are right now on SF99540 7282...and with five instances running on the LPAR, all temp storage is at around 40 GB's....

So then, is it hardware dependent? We are running i570's..Cud it be that this is limited to some models, although would not make sense generally.

We are also running 4.6C R/3 and BCS and BW70 & SCM41 and SOLMAN40 on this box.

Anyway, thanks for letting us know.

Good luck

Abhi

[Former Member]

Abhi,

Is this a development environment that is at V5R4 or production will normal activity? We upgraded to V5R4 on 10/20 in our development environment (Sandbox, Development, Test, Solution Manager, BW Development) and would have never been able to tell if we this Teraspace/Temp storage issue would be a problem in production without IBM running advanced analysis commands from Service Tools. We upgraded to V5R4 in production on 11/17 and had major temp storage/performance problems. We did apply the SI28358 (Test PTF) in our production system last night after testing (with IBM) in the development partition over the past week. It appears to have fixed the temp storage issue.

Regards,

Philip Stracener

[Former Member]

Philip,

No, I am talking about DEV/TEST/PROD servers all combined....and in operation for almost a year....

what hardware are you running?? This is really interesting...

But Thanks a lot for informing us about the PTF number...Hope it will be soon put on the infoAPAR....for SAP...

Thanks

Abhi

[Former Member]

Philip

Are u running V5R4M5 or V5R4M0?

Pls let me know...

Thanks

Abhi

[Former Member]

Abhi,

I'm running (2) 550s (Power 5). The problem is in a 3-Tier environment. The QXDA or APIA jobs on the database server depending on if you're using ethernet or opticonnect over HSL.

This is what IBM has identified.

Are you using an application server to connect to your database server? If not, you wouldn't see this problem.

Regards,

Philip Stracener

[franco_tassone]

Volker,

we went EXT since long ago, we had no issues until we ran V5R3, now we're in V5R4, last apar II14126 as of Oct 8, kernel EXT 2341 on CI and windows application servers and we're definitivelly in troubles !

The strange thing is that we have another identical iSeries, same kernels, same V5R4 tough wih an older CUM, no problems on that iSeries.

No useful suggestions from IBM and SAP so far.

IBM asked us to load also the following PTFs :

SI28671

MF42522

SI25219

we did it, no luck, nothing changed... the tempstorage is increasing at an avg speed of 10Gb per hour.

I've found some old threads in the saponiseries group, here a couple of workaround were suggested :

Note 182207 - DB2/390: Improving Virtual Storage Utilization

Note 101717 - Automatic restart of SAP R/3 work processes

See also http://tech.groups.yahoo.com/group/SAPoniSeries/message/39

Do you think we can adopt this temporary workaround ?

Franco.

[Former Member]

Hi Franco,

first of all I'm happy, that this parameter change with the roll memory didn't solve your issue with the increasing DASD, as it just should reduce the average use but not the steady increasement

Yes, you can try this note as workaround - I would suggest to set the time to about 4 hours - that means each WP will die each 4 hours (not good for performance.

If this helps a lot, the reason is to be found in the ODP areas of IBM. If nothing changes, it is a memory leak somewhere else (most likely) in the IBM code.

Good luck,

Volker Gueldenpfennig, consolut international ag

http://www.consolut.de - http://www.4soi.de - http://www.easymarketplace.de

[franco_tassone]

.... also, SAP adviced us to reduce immediatelly the ztta/roll_area from the current value ( 16773120 ) to 3000000, which is the default, and this should fix the things, according to them. Does is make sense to you ?

Cheers.

Franco.

[Former Member]

Hi Franco,

ok, the total temp storage will be reduced with this parameter change - that is normal.

But:

Isn't your problem a big performance issue ? I would not expect far better response times because of this change (except when your DASD was running to nearly 100%)

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.de - http://www.4soi.de - http://www.easymarketplace.de

[Former Member]

Hi Franco,

ok, I hope, that is fixed now.

But still: I cannot see a difference between V5R3 & V5R4 ...

Or, did you upgrade the kernel from COM to EXT kernel for 4.6 with this step ?`

Hope, all is OK on your site,

Volker Gueldenpfennig, consolut international ag

http://www.consolut.de - http://www.4soi.de - http://www.easymarketplace.de

[franco_tassone]

Hi ABHI,

there aren't any R/3 DB growing so fast ! We're taking this aspect under complete control.

I'm currently issuing the DSPTMPSTG every 25-20 seconds, here's the result:

Temporary storage used: 115725312 KB.

Temporary storage used: 115789824 KB.

Temporary storage used: 115823616 KB.

Temporary storage used: 115823616 KB.

Temporary storage used: 115893248 KB.

Temporary storage used: 115919872 KB.

Temporary storage used: 115977216 KB.

Temporary storage used: 116139008 KB.

The temporary storage is growing so fast !

Any advice ?

Franco.

[Former Member]

Hi Franco,

I am not sure if I understand you question that well...

What exactly u mean when u say "too fast" storage consumption...

If your system disk space is increasing too fast, check in DB02-->Detailed object analysis and figure out which objects have been growing the fastest,...you can see the growth rate daily/weekly/monthly...You can check out what tables or physical files those are and then you can ask the team, fi they know anything about it...wwould give you a starting point to investigate...

Maybe it is that there is some activity which is causing the system to grow "too fast" ...

Hope that helps...

Thanks

Abhi

[Former Member]

Also try checking the DSPTMPSTG command from the <SID>OFR menu...

That would give u an idea of the temp sapce being used...