Performance Issue

[Former Member]

We are getting a timed out error (after 10 mins) when i execute a report in production

server(high -availablity) environment

However the same report works fine in develeopment environmnet(central

system) with the same period

This is happening only after we have moved to high availablity environment(cluster) with high end servers

Ideally this should improve the performance of the system after the hardware migration and its not case here

Appreciate your help in analysing this issue

Background:

AM : 120 GB RAM

OS: Windows 2008

DB: MSSQL 2008

[former_member340078]

Hello Vadi,

you can try Runtime analysis (TCD SE30) on both DEV and PRD to find out at which step degradation happens. Maybe it will give a hint....

Regards,

Ilya

[Former Member]

You mention migration in your post,so is it a system migration from older to newer server with better specification?

let me rephrase your question:

with the same amount of data ( i guess you have copied the data since you are comparing the performance of both systems) & system version

but different hardware specifications ( one is higher / PRD )

the execution time is less in the lower spec system / DEV?

Is this correct?

[JPReyes]

Its simply impossible to give you a good answer without having the facts....  Lots of different factors will influence the performance of your system and even of a certain report, It could be as simple as an badly built index or as complicated as a mayor bottleneck in your hardware.

More information is needed.

Regards, Juan

[Former Member]

Dear Juan,

Thanks for your response

I have listed some information regarding the current prod landscape

High Availablity cluster:

Node 1: SAP Central Instance running

Node 2: DB running

OS : windows 2008 R2

DB: MSSQL 2008 R2

Kernel :291

SAP Product:ERP EHP3

RAM : 128 GB

Number of Application servers : 2

I have given response time comparision for both dev and prod systems in my previous reply

Happy to provide more information on request

Thanks,

Vadi

[Former Member]

Hey Vadi,

Do an SQL trace and find if some indexes are in play. May be there is in DEV but not in production.

Also after migration did you run the stats in production ?

Regards.

Ruchit.

[Former Member]

Dear Vadi

There will be more users in production system as compared to development.

So, more users more transactions. Just try to increase the value of a parameter suggested by Ratnajit earlier and let us know the result.

Also, It is always better to run long running reports in background for which a separate work process is dedicated. Dialog users will not be effected.

Rgds

Khalid

[Former Member]

Dear Khalid,

Thanks a lot for your response

yes as you said,the load is more when compared to dev system

However the issue occurs even when there is less load (off peak hours ) in prod system

Similarly there are many reports which are taking up long time now (after migration) and increasing the timeout may have negative impact on the performance

Please suggest

Thanks,

Vadi

[Former Member]

Vadi

Since you did system copy and tried to run the report.

Just wondering if the kernel level are same in both the systems?

Just check and confirm

this issue is not at all related to stand alone system or high availability.

Rgds

Khalid

[Former Member]

Hi,

I checked and kernel is same in dev and prod

Thanks,

Vadi

[Former Member]

Hello Vaidivambal,

Are  you facing performance issues for this report only or for all transactions?

You can have following general checks:-

1. Check for missing indexes.
2. Check if Datafiles are full:- Although Autogrow option is
   set, but autogrow can lead to performance problems and is mainly used for
   emergency cases.
3. Activate the user trace in ST05 while you run that report, it will give you some insight.
4. Try to find out the Long running queries in STAD and SM50.
5. ST02:- Buffer utilization
6. Check the response time for all the dialog steps in SMLG.
7. Checkout for the I/O latencies in ST04 or DBACOCKPIT

          Most optimal values:-
         Read:- Reading from the data files: <=10ms - 15ms
         write Log:- Writing into the transaction log <=5ms

Cheers,

Vishal

[Former Member]

Dear Vishal,

Thanks a lot for the valuable inputs

Yes we do have the performance issue for some of the reports

1. Checked in DB02 and there are no missing indexes

2. No we have enough free space available

PRDDATA1     102,280     0.14     150

PRDDATA2     106,820     8.73     150

PRDDATA3     107,930     8.65     150

PRDDATA4     102,850     9.10     150

PRDLOG1     13,903     99.86     250

PRDLOG2     51,750     99.86     250

3. Have activated the trace and its taking more DB time for sequential read for PA0001 table

4. Cudnt locate the query using STAD

5. ST02 looks fine : No swapping

6. SMLG dialog response time : Dont sure on how to check..pls guide me

7. I/O latencies are within the given range

Let me know for more information

Thanks,

vadi

[Former Member]

Hello Vadi,

1) Why are you using 2 log files on MSSQL Database?

   I think It is not recommended to use multiple log files for MSSQL.

Please check if you are using single / multiple log files on your Dev. I dont know what would be it's impact.

  PRDDATA1     102,280     0.14     150

Do you have 0.14 GB free on PRDDATA1?

It is recommanded that you have equivelent free disk space across all data files for better I/O.

2) Also check the event logs for some H/W issues.

3) Goto SMLG --> GOTO --> Load Distribution

Observe the Response time.

4) Make sure you have configured the DB memory parameters properly.

How much memory have you allocated for Database?

Cheers,

Vishal

[former_member189725]

Are you running the report in the foreground ?

If yes , check the value of the profile parameter using RZ11  rdisp/max_wprun_time in the production system and the dev system.In the dev system , the value might be high whereas in Production system it is 600.

You can increase the value of rdisp/max_wprun_time in the production system to 1800 (30 mins) and see if the report gives an output.

Regards

Ratnajit

[Former Member]

Hi Ratnajit,

Thanks a lot for your reply

Yes we are running it in the foreground

I understand that this can be resolved by incresing the timeout value for given parameters

However this doesn't help us in fidning the root cause of the issue

i)The given parameters having the same value in Dev and Prod

ii) The source code version is same in dev and prod

iii) The last change made was on 2009 and no change after that in both dev n prod

iv) the system copy was performed on dev system and the report executed for the same period in dev n prod and it has to retrive the same amount of data in dev n prod

Appreciate your response

Thanks,

vadi

[Former Member]

Hi Ratnajit,

Thanks a lot for your reply

Yes we are running it in the foreground

I understand that this can be resolved by incresing the timeout value for given parameters

However this doesn't help us in fidning the root cause of the issue

i)The given parameters having the same value in Dev and Prod

ii) The source code version is same in dev and prod

iii) The last change made was on 2009 and no change after that in both dev n prod

iv) the system copy was performed on dev system and the report executed for the same period in dev n prod and it has to retrive the same amount of data in dev n prod

Appreciate your response

Thanks,

vadi

[former_member189725]

You need to figure out the workload details of the report on the production system and then compare it against the same analysis in the development system . Please run the report on both the systems and use the Business transaction Analysis (STAD) to find the details of the response times .

Figure out the area of potential issue and try to work on it.

Regards

Ratnajit

[Former Member]

Dear Ratnajit,

Thanks a lot for your response

I have executed the report in background in prod server and it tool 894 sec to complete

Below is the St03 data for both dev n prod

Dev:

steps     T time   A T time      T proc   Avg Proc       Tot CPU   Avg CPU       T DB    Avg DB/dialog step              roll wait     Seq read

2     241     120,369.0     158     78,863.0     200     99,973.0     83     41,328.5     0.0     0     175.0     0     475,998

prod:

steps     T time   A T time      T proc   Avg Proc       Tot CPU   Avg CPU       T DB    Avg DB/dialog step              roll wait     Seq read

1     893     892983          258     258461          322     321970          635     634515                    0          485744

We are running SAP Central instance in node1 and SQL DB in another node (node 2)

Can this cause this issue

Please let me know your inputs

Thanks,

Vadi

[bharat_rathod2]

Dear,

I think your production data qty is large tha dev so please try to give restrict data fetching by giving more option in selection screen.