CE 7.1 Ehp1 SP5 on Oracle 10.2.0.4 Performance - XSS track import of SCAs

[Former Member]

Hi,

I'm currently building a new track on my newly built CE server which has the NWDI capability configured. I've had a few problems importing the SCA files into the track. I've been getting DTR internal server errors. Looking through the logs I couldn't see a specific reason as per OSS 774339 and the subsequent notes this refers to. After restarting the CE server I've imported the SCA files individually and this has at least enabled me to get the SCA files imported without error.

The time taken to import is very long - the ESS SCA file (290 MB) took 3.5 hours alone.

I've read OSS 1361525 which is related to DTR performance with Oracle but the Patch provided is for 7.01 / 7.02 releases of CE.

I have already patched my CE 7.1 Ehp1 SP5 server with the available patches for DICMS05, DICLIENTS05 & DICBS05* - there is none current available for download for the DIDTR05* CE 7.1 Ehp1 release.

Can anyone suggest what could be done to reduce the import runtimes ?.

I'm regularly updating the Oracle Statistics as per OSS 1229080 and the Oracle params have been set as per OSS 830576.

Regards,

Brian.

[ErvinSzolke]

Hi Brian,

yes, I remember now, we were talking about this issue with my developer colleagues via mail, and here the problem is apparently not due to the NWDI, but it at the end the message landed on a JAS component due to

Web Container failed to process the request [250].

[EXCEPTION]

java.lang.IllegalStateException: ShmWebSession is closed

at com.sap.bc.proj.jstartup.sadm.ShmWebSession.isDebug(ShmWebSession.j

For the record: the fix is in 720 and now considering to downport it to 711.

Thank you for sharing this information,

I consider then this thread as closed.

Thank you!

Best Regards,

Ervin

[Former Member]

Hi Ervin,

the message number is :-

738267 / 2010

Regards,

Brian.

[ErvinSzolke]

Hi Brian,

can you please tell me the message number you have created at us?

Thank you!

Best Regards,

Ervin

[Former Member]

I opened a message with SAP Support and sent them all the screenshots and error logs on the problems importing the last 3 XSS SCA files. They have reviewed the logs etc and responded saying it's a known issue that was fixed in CE 7.20 but hasn't been in 7.11 yet. My message is still with SAP waiting for the fix to be made available.

[Former Member]

Hi Ervin,

1. The NWDI Server is a distributed configuration - the DB instance is on it's own Solaris 10 zone and the central services & instance are on another Solaris zone. Both zones have plenty CPU/Memory resources and the load values on both are low. The systems are not overloaded.

2. THis is a new build and I'm setting up the first track to enable the customer to start a XSS development project. The first 10 or so SCs imported without error into the track. ESS, MSS & PCUI_GP SCA files are the ones remaining and are failing to import.

3. Our OS is Solaris 10 on Sun Sparc - so no windows type anti-virus software to contend with.

4. I've run the statistics a few times over the last day or so :-

brconnect -u / -c -o summary -f stats -o SAPSR3DB -t all -p 8 -f nocasc

5. Trace levels are default.

6. The logs & traces have no OutOfMemory errors as per OSS 1001502. The error received is :-

20100908171424 Info :Starting Step CBS-make at 2010-09-08 17:14:24.0632 +1:00

20100908171425 Info :wait until CBS queue of buildspace S01_Sandbox_D is completely processed, before starting the import

20100908171425 Info :CBS queue of buildspace S01_Sandbox_D is completely processed. Starting the import now.

20100908181425 Fatal :CBS throws exception during activation:

20100908181425 Fatal :caused by Exception:com.sap.tc.cbs.client.error.CommunicationException: Read timed out (Service call exception; nested exception is:

java.net.SocketTimeoutException: Read timed out):Read timed out (Service call exception; nested exception is:

java.net.SocketTimeoutException: Read timed out)

The interesting thing here is that the error is thrown exactly one hour after the CBS-make operation starts.

7. We've got Oracle DB.

Hopefully you'll be able to suggest how best I can resolve the error and continue.

Also - from the logs there was further error activity for this same import operation 3 hours later after I'd gone home :-

#2.#2010 09 08 21:34:50:064#+0100#Info#/Applications/CBS/Service#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200680000000E000004DB#14060650000000783##com.sap.tc.cbs.server.rt.rq.RequestOrc####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[CBS Request Orc,5,SystemThreadGroup]#Plain##

Request arranged for processing for BuildSpace S01_Sandbox_D. [bsID:3, rID:62, node:14060650]#

#2.#2010 09 08 21:34:50:086#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000006F000004DB#14060650000000783##com.sap.tc.cbs.server.dao.impl.Request####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Build number assigned: 67#

#2.#2010 09 08 21:34:50:088#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000070000004DB#14060650000000783##com.sap.tc.cbs.server.dao.impl.Request####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Change request state from QUEUED to PROCESSING#

#2.#2010 09 08 21:34:50:095#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000071000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

#

#2.#2010 09 08 21:34:50:096#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000072000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

REQUEST PROCESSING started at 2010-09-08 20:34:50.095 GMT#

#2.#2010 09 08 21:34:50:098#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000073000004DB#14060650000000783##com.sap.tc.cbs.server.proc.RequestHandler####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

ACTIVATION request in Build Space "S01_Sandbox_D" at Node ID: 14,060,650

[id: 62; parentID: 0; type: 4]

[options: IGNORE BROKEN DC, FORCE ACTIVATE PREDECESSORS, FORCE INTEGRATED, IGNORE COMPONENT INTERSECTION]#

#2.#2010 09 08 21:34:50:149#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000074000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

#

#2.#2010 09 08 21:34:50:150#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000075000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Waiting for access: 54 ms#

#2.#2010 09 08 21:34:50:151#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000076000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

#

#2.#2010 09 08 21:34:50:151#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000077000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

===== Pre-Processing ===== started at 2010-09-08 20:34:50.150 GMT#

#2.#2010 09 08 21:34:50:152#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000078000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

List of requested for activation activities:#

#2.#2010 09 08 21:34:50:184#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000079000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

''sap.com_SAPPCUI_GP_1'' compartment#

#2.#2010 09 08 21:34:50:184#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007A000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

CMS_sap.com_SAPPCUI_GP_590_aabfda99de4511d9b84800300530c04f#

#2.#2010 09 08 21:34:50:185#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007B000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

[ISN 2][created by MAIN at 2010-09-08 17:14:24.0][OID e4fdda79781d11dec2470030057116fe]#

#2.#2010 09 08 21:34:50:185#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007C000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

There is 1 activity in compartment sap.com_SAPPCUI_GP_1#

#2.#2010 09 08 21:34:50:186#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007D000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

1 activity will be processed by this request#

#2.#2010 09 08 21:34:50:647#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007E000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

#

#2.#2010 09 08 21:34:50:648#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000007F000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Analyze dependencies to predecessor activities... started at 2010-09-08 20:34:50.647 GMT#

#2.#2010 09 08 21:34:50:648#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000080000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Analyzing predecessors in compartment "sap.com_SAPPCUI_GP_1"#

#2.#2010 09 08 21:34:50:669#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000081000004DB#14060650000000783##com.sap.tc.cbs.server.proc.impl.ActivationPreProcessor####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

No dependencies to predecessor activities found.#

#2.#2010 09 08 21:34:50:669#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000082000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Analyze dependencies to predecessor activities... finished at 2010-09-08 20:34:50.669 GMT and took 22 ms#

#2.#2010 09 08 21:34:50:670#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000083000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

#

#2.#2010 09 08 21:34:50:670#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000084000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Analyze versions... started at 2010-09-08 20:34:50.670 GMT#

#2.#2010 09 08 21:34:50:712#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000085000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

===== Pre-Processing ===== finished at 2010-09-08 20:34:50.711 GMT and took 561 ms#

#2.#2010 09 08 21:34:50:712#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000086000004DB#14060650000000783##com.sap.tc.cbs.server.dao.impl.Request####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Change request state from PROCESSING to FAILED#

#2.#2010 09 08 21:34:50:713#+0100#Error#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000087000004DB#14060650000000783##com.sap.tc.cbs.server.proc.RequestHandler####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Error! The following problem(s) occurred during request processing:#

#2.#2010 09 08 21:34:50:713#+0100#Error#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D0582006900000089000004DB#14060650000000783##com.sap.tc.cbs.server.proc.RequestHandler####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Error! The following error occurred during request processing:Server error [500 Internal Server Error]#

#2.#2010 09 08 21:34:50:714#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000008B000004DB#14060650000000783##com.sap.tc.cbs.RequestPerformanceMeter####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

REQUEST PROCESSING finished at 2010-09-08 20:34:50.714 GMT and took 619 ms#

#2.#2010 09 08 21:34:50:728#+0100#Info#/Applications/CBSRequestLog/RequestProcessing/id67#

#BC-CTS-CBS#tc.CBS.Service#0021286D058200690000008C000004DB#14060650000000783##com.sap.tc.cbs.server.dao.impl.Request####97674940BB5111DFA9180021286D0582#97674940bb5111dfa9180021286d0582#97674940bb5111dfa9180021286d0582#0#Thread[SystemThreadGroup - 70,5,SystemThreadGroup]#Plain##

Regards,

Brian.

[ErvinSzolke]

Hello Brian,

let me give you a general answer, so others who finds this thread and have a different platform, may find these hints useful.

first of all can you please check from your side if NWDI performance can be improved in general?

1 Does the nwdi server have enough

a) CPU,

b) hard disk space,

d) DB space? (is the DB mapped via network, or is it located on the same server?)

e) Is the load low (if the nwdi system is hosted on unix, you can also use the command w).

The "load average" should be under value 1. If it is higher, you need to check your running processes using the "top" command and check which one is taking too much CPU time. If this is not the case then we can continue the investigation somewhere else, but please let me know the result of this test first.

2 Are you encountering general NWDI performance problem lately or in the past?

3 When building are all anti-virus software disabled? Especially live-scan feature which checks each touched file.

This is causing problem due to high number of temporary files when CBS builds.

4 Could you please ask your DB Colleagues to run UPDATE DB Statistics? There's a background job to update the database statistics for SAP's MaxDB. For all other databases (so also for Oracle/MSSQL) the update of the database statistics has to be done manually. (This can easily be the cause and the solution as well as we have experienced from other similar situations from the past.)

5 Check if you are using a pretty strong severity setup, and therefore too many traces will be written into the defaultTrace. This can cause overall performance issue as well. Could you please reset the severity setup in /nwa Configuration -- Log Configuration ?

6 If you are using Oracle, then check also the note #1001502

7 If you are using MSSQL, then check the note #985821

Some other general notes:

#889038 - Collective note: NWDI server component configuration

#737368 - Hardware requirements of Java Development Infrastructure

#754143 - CMS remarks/restrictions

Best Regards,

Ervin (http://wiki.sdn.sap.com/wiki/display/TechTSG/%28NWDI%29Home)