CMS: Shared Workspace Lock Error During Import Check for ESS

[Matt_Fraser]

Hello all,

I am setting up a 'compare' track in CMS for modifications to the ESS component, but I'm running into a small problem that I can't seem to get past. The 'Import Check' for the SAP_ESS component fails after about 45-50 minutes, and the CMS Log shows the error:

Exception:com.tssap.dtr.client.lib.deltavlib.VcmFailure: Conflict [(pre||post)-condition failed: x:could-not-obtain-shared-workspace-lock]

Digging into the log viewer through NWA, I can see objects being processed and DB updates starting and finishing, and then periodically a warning:

Precondition failed: Precondition could-not-obtain-shared-workspace-lock check failed for resource {resourceType='WORKSPACE', getPath()='/ws/604-13c/sap.com_SAP_ESS/dev/inactive', resourceID=....

This is followed by the informational statement 'workspace is currently locked, no integration possible.' Then there seem to be more objects processed, more DB updates finished, and the warning occurs a few more times. Eventually, after 45 minutes or so, there's the fatal error that I mentioned at the start and the entire process comes to a halt with a final status of 'Check failed.'

Searching turned up several very old discussions about this error, but none of them had any resolution.

Initially I followed the advice from (NWDI) (CBS) Import of Large Archives from the CMS Fails - Technology Troubleshooting Guide - SCN Wi... and Unable to check in large files into DTR - Technology Troubleshooting Guide - SCN Wiki, which led me to set in the ICM:

icm/HTTP/max_request_size_KB = 409600

and in the DTR's repository.properties:

com.tssap.dtr.pvc.propagation.maxRequestContentLength = 409600

com.tssap.dtr.deltav.propagation.maxRequestContentLength = 409600

No joy. From there I followed the guidance from the NWDI Troubleshooting Guide linked from the NWDI SCN space, following the path CMS import shows errors -> DTR import -> yes (to Is the DTR server available?) -> yes (to Are you importing a large software component (e.g. ESS/MSS)?) -> MS SQL (to What is the database system you use?).

This led me to Note 985821 (DTR Import Hangs on MSSQL Database), based upon which I set:

com.tssap.dtr.pvc.versionmg.DefaultIntegrator.fastFirstIntegrate = false

com.tssap.dtr.pvc.basics.sql.autobatchexec.maxbatchcount =50

No change. I set:

com.tssap.dtr.server.deltav.integrate.countForLargeSourceSetNameCollisionComputer = 200000

No change.

I proceeded on to Note 1096029 (Hanging initial DTR import of huge components on MSSQL). The patch mentioned is not applicable for me (I'm on a higher NetWeaver release and have the latest available SP -- see below), but I did then set

com.tssap.dtr.server.deltav.integrate.inClauseSize = 8

No change.

I looked at Note 990538 (WSAENOBUFS error on Windows) and it's not applicable to me (I am on a 64-bit platform, and the latest release of Windows). Likewise with 1014672 (CMS assembly fails with out of memory exceptions on MSSQL): this isn't my problem, plus the advice about the JDBC driver appears to be outdated.

So here we are. Each of the iterative attempts above required nearly an hour of waiting for the error to recur, of course.

So, the environment. This is a newly installed NWDI on NetWeaver 7.4 sps11. It is running on SQL Server 2012 sp2 cu5 and Windows 2012 R2. I configured the DI component using the configuration wizard. CMS, DTR, and CBS all reside on the same box. Communication with the central SLD seems to be just fine. Other than the default SANDBOX track setup by the wizard, this is the first track on the CMS, also setup via the Modify a Software Component CMS wizard. The track is intended as a 'compare' track for supporting ESS modifications during support pack updates and upgrades, so it has no runtime systems associated with it, and is a "development system only" track.

The track is meant for ESS on a NetWeaver 7.01 sps13 portal, so it has its own associated JDK (sapjvm 4.1) setup in a build variant, different from the NWDI system's own JDK (sapjvm6). I have successfully imported SAP_BUILDT, EP_BUILDT, SAP-JEE, and SAP_JTECHS all with version 7.01 sp13, and SAPPCUI_GP version 603 sp9. The component that is failing is SAP_ESS 603 sp12. These component versions and support pack levels match what is currently on our ESS portal, and they match the components used in our old (retired and soon to be deleted) NWDI system (which is also version 7.01 sp13). No custom components or modifications exist yet in the new NWDI.

No one else besides me has been using the box, since it's brand new and not yet ready for developers.

There it is. I'm about out of ideas, having struggled with this for the past few days.

Cheers,

Matt

[Matt_Fraser · Accepted solution]

The issue is now resolved, with the help of "Xinlei" at SAP Support, and the main cause appears primarily to have been one related to ICM timeouts, plus a little bit of tuning.

Following Xinlei's advice and that of Notes 1750758 (DC build in CBS fails due to ICM FCA connection broken with DTR server) and 2007212 (Tuning SAP Web Dispatcher and ICM for high load), I adjusted the instance profile as follows:

Changed the following line (added piece in bold):

icm/server_port_4=PROT=HTTPS, PORT=50001, TIMEOUT=60, PROCTIMEOUT=-1, SSLCONFIG=ssl_config_4

Added following lines:

icm/max_conn = 2000

icm/req_queue_len = 6000

icm/min_threads = 10

icm/max_threads = 500

mpi/total_size_MB = (min(0.06 * $(icm/max_conn) + 50, 2000))

mpi/max_pipes = ($(icm/max_conn) * 2)

In NetWeaver Administrator -> Configuration -> Infrastructure -> Java System Properties -> Services -> HTTP Provider, changed property FCAServerThreadCount to 30 (from default of 15).

Restarted the system and tried again, and voila!

[ErvinSzolke · Accepted solution]

Thanks for sharing, Matt, I am glad the problem has been resolved now.

[ErvinSzolke · Accepted solution]

btw, I'll recommend Xinlei to enhance the troubleshooting guide with this.

[Matt_Fraser]

More than a week has gone by since submitting the Customer Incident, and last night someone finally picked it up. However, I suspect the reply I got was actually intended for a different Incident than mine, as it didn't seem to relate at all, and it was addressed to someone else. Oh well, hopefully this at least means I'll get some attention now.

[ErvinSzolke]

Hi Matt,

to speed up things a little, I handed over your case to the developers.

Have a great day!

Best Regards,

Ervin

[Matt_Fraser]

I saw that. Thanks, Ervin! Though I admit I half-expected when you saw it you'd take one look and say "Oh, I see where you missed something. Just make this small change...."

[ErvinSzolke]

I wish I had time to dig into your issue, but currently I am really really busy, so I thought at least I try to help in a different way, like routing your case to my developer colleagues. Knowing you, I am sure you checked everything so I thought the best will be if your case is in the developers' hand.

[Matt_Fraser]

Additional troubleshooting steps I've applied:

• Applied the latest Java patches to all components for which such patches exist (none of which were DI components, however).
• Increased sysDS.maximumConnections in dbpool service to 80 from default of 50 (i.e., JDBC connections

Still no joy. I've submitted a Customer Incident.