VcmFailure received: Not Ready

[Former Member]

Hi,

I get an error which i cant figure out :-

steps followed :

1> upgrade of WebAS 640 to SP17 on the JDI server

2> successful import of SAP-JEE, BUILDT, & JTECH into a track

3> failure of MSS & ESS package imports into CMS

4> error message specified at the end of the post, which is when the import process completed it's run

5> upon repeated trials of import the portal hangs and is no longer accessible [http://server:port/irj/portal]

only option it then to reset it, to get it to work again, at this time the error log in not written

6> the jlauch process swells to 800M, there is 3.71 gb free space on the hdd, sql server has 1100 M free space

with respect,

amit

Info:Starting Step Repository-import at 2006-12-16 10:55:58.0860 +13:00

Info:Component:sap.com/SAP_MSS

Info:Version :MAIN_xss04VAL_C.20060707030624

Info:1. PR is of type TCSSoftwareComponent

Fatal Exception:com.sap.cms.tcs.interfaces.exceptions.TCSCommunicationException: communication error: VcmFailure received: Not Ready [(pre||post)-condition failed: Not Ready]:communication error: VcmFailure received: Not Ready [(pre||post)-condition failed: Not Ready]

com.sap.cms.tcs.interfaces.exceptions.TCSCommunicationException: communication error: VcmFailure received: Not Ready [(pre||post)-condition failed: Not Ready]

at com.sap.cms.tcs.client.DTRCommunicator.integrateChangelist(DTRCommunicator.java:382)

at com.sap.cms.tcs.core.RepositoryImportTask.processRepositoryImport(RepositoryImportTask.java:295)

at com.sap.cms.tcs.core.RepositoryImportTask.process(RepositoryImportTask.java:500)

at com.sap.cms.tcs.process.ProcessStep.processStep(ProcessStep.java:77)

at com.sap.cms.tcs.process.ProcessStarter.process(ProcessStarter.java:179)

at com.sap.cms.tcs.core.TCSManager.importPropagationRequests(TCSManager.java:376)

at com.sap.cms.pcs.transport.importazione.ImportManager.importazione(ImportManager.java:216)

at com.sap.cms.pcs.transport.importazione.ImportQueueHandler.execImport(ImportQueueHandler.java:585)

at com.sap.cms.pcs.transport.importazione.ImportQueueHandler.startImport(ImportQueueHandler.java:101)

at com.sap.cms.pcs.transport.proxy.CmsTransportProxyBean.startImport(CmsTransportProxyBean.java:583)

at com.sap.cms.pcs.transport.proxy.CmsTransportProxyBean.startImport(CmsTransportProxyBean.java:559)

at com.sap.cms.pcs.transport.proxy.LocalCmsTransportProxyLocalObjectImpl20.startImport(LocalCmsTransportProxyLocalObjectImpl20.java:876)

at com.sap.cms.ui.wl.Custom1.importQueue(Custom1.java:1170)

at com.sap.cms.ui.wl.wdp.InternalCustom1.importQueue(InternalCustom1.java:2162)

at com.sap.cms.ui.wl.Worklist.onActionImportQueue(Worklist.java:880)

at com.sap.cms.ui.wl.wdp.InternalWorklist.wdInvokeEventHandler(InternalWorklist.java:2338)

at com.sap.tc.webdynpro.progmodel.generation.DelegatingView.invokeEventHandler(DelegatingView.java:87)

at com.sap.tc.webdynpro.progmodel.controller.Action.fire(Action.java:67)

at com.sap.tc.webdynpro.clientserver.task.WebDynproMainTask.handleAction(WebDynproMainTask.java:101)

at com.sap.tc.webdynpro.clientserver.task.WebDynproMainTask.handleActionEvent(WebDynproMainTask.java:304)

at com.sap.tc.webdynpro.clientserver.task.WebDynproMainTask.execute(WebDynproMainTask.java:659)

at com.sap.tc.webdynpro.clientserver.cal.AbstractClient.executeTasks(AbstractClient.java:59)

at com.sap.tc.webdynpro.clientserver.cal.ClientManager.doProcessing(ClientManager.java:251)

at com.sap.tc.webdynpro.serverimpl.defaultimpl.DispatcherServlet.doWebDynproProcessing(DispatcherServlet.java:154)

at com.sap.tc.webdynpro.serverimpl.defaultimpl.DispatcherServlet.doContent(DispatcherServlet.java:116)

at com.sap.tc.webdynpro.serverimpl.defaultimpl.DispatcherServlet.doPost(DispatcherServlet.java:55)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.runServlet(HttpHandlerImpl.java:390)

at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.handleRequest(HttpHandlerImpl.java:264)

at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:347)

at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:325)

at com.sap.engine.services.httpserver.server.RequestAnalizer.invokeWebContainer(RequestAnalizer.java:887)

at com.sap.engine.services.httpserver.server.RequestAnalizer.handle(RequestAnalizer.java:241)

at com.sap.engine.services.httpserver.server.Client.handle(Client.java:92)

at com.sap.engine.services.httpserver.server.Processor.request(Processor.java:148)

at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)

at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)

at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)

at java.security.AccessController.doPrivileged(Native Method)

at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)

at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)

Info:Step Repository-import ended with result 'fatal error' ,stopping execution at 2006-12-16 23:30:24.0341 +13:00

[Former Member · Accepted solution]

Check the application log for DTR using Visual Admin. Looks like CMS is trying to import sources into DTR and DTR is "not ready". Check to see if DTR is running (simply open http://..../dtr/). Check for permissions, database issues, etc.

Regards,

Marc

[Former Member · Accepted solution]

Hi Marc,

thank you for the response.

DTR is active, and the DB setup is as per the JDI installation manual, the files JTECH, BUILDT & JEE were imported with no issue, but mss & ess are giving this error.

on the DB side : we use SQL server 2000, i verified the server memory ( >500MB), max no of user connections = 0, but i could not find the setting for max no of concurrent row locks [upon googling it, i could not find the master property whic controls this, only a command level parameter in SQL syntax]

thank you for your support, hope to resolve this asap.

with respect,

amit

[Former Member · Accepted solution]

Hi Amit,

JTECH, BUILDT & JEE do not contain any sources so they do not trigger this problem.

I'm not an export on SQL server, but isn't there also a setting in the JDBC service that determines the maximum number of connections from the engine to the database?

I'd really check with the logviewer (standalone or within Visual Admin): merge the default trace and the application log of DTR and check if there is some error message or stacktrace that gives additional information about the cause of this problem. I'm pretty sure there is an error even if it is hard to find using the logviewer....

Regards,

Marc

[Former Member · Accepted solution]

Hi Marc,

thank you for the assistance.

my max DB connections in the JDBC driver is 80.

with respect,

amit

here is the defaultTrace error message which keeps recurring :

#1.5#000F1FF8D9F4001C0000258700002FE8000425421F937704#1166865834700#com.sap.engine.services.dbpool##com.sap.engine.services.dbpool######48d551d0909811dbb684000f1ff8d9f4#Thread[CBS Resource Orc,10,SAPEngine_System_Thread[impl:5]_Group]##0#0#Error#1#/System/Server#Java#dbpool_0300##ResourceException in method ConnectionFactoryImpl.getConnection(): com.sap.engine.services.connector.exceptions.BaseResourceException: Cannot get connection for 120 seconds. Possible reasons: 1) Connections are cached within SystemThread(can be any server service or any code invoked within SystemThread in the SAP J2EE Engine), 2) The pool size of adapter "SAPEPDDB" is not enough according to the current load of the system or 3) The specified time to wait for connection is not enough according to the pool size and current load of the system. In case 1) the solution is to check for cached connections using the Connector Service list-conns command, in case 2) to increase the size of the pool and in case 3) to increase the time to wait for connection property. In case of application thread, there is an automatic mechanism which detects unclosed connections and unfinished transactions.#3#ResourceException#ConnectionFactoryImpl.getConnection()#com.sap.engine.services.connector.exceptions.BaseResourceException: Cannot get connection for 120 seconds. Possible reasons: 1) Connections are cached within SystemThread(can be any server service or any code invoked within SystemThread in the SAP J2EE Engine), 2) The pool size of adapter "SAPEPDDB" is not enough according to the current load of the system or 3) The specified time to wait for connection is not enough according to the pool size and current load of the system. In case 1) the solution is to check for cached connections using the Connector Service list-conns command, in case 2) to increase the size of the pool and in case 3) to increase the time to wait for connection property. In case of application thread, there is an automatic mechanism which detects unclosed connections and unfinished transactions.#

#1.5#000F1FF8D9F4001C0000258800002FE8000425421F94B5A2#1166865834778#com.sap.engine.services.dbpool##com.sap.engine.services.dbpool######48d551d0909811dbb684000f1ff8d9f4#Thread[CBS Resource Orc,10,SAPEngine_System_Thread[impl:5]_Group]##0#0#Error#1#/System/Audit#Java###Exception #1#com.sap.engine.services.dbpool.exceptions.BaseSQLException: ResourceException in method ConnectionFactoryImpl.getConnection(): com.sap.engine.services.connector.exceptions.BaseResourceException: Cannot get connection for 120 seconds. Possible reasons: 1) Connections are cached within SystemThread(can be any server service or any code invoked within SystemThread in the SAP J2EE Engine), 2) The pool size of adapter "SAPEPDDB" is not enough according to the current load of the system or 3) The specified time to wait for connection is not enough according to the pool size and current load of the system. In case 1) the solution is to check for cached connections using the Connector Service list-conns command, in case 2) to increase the size of the pool and in case 3) to increase the time to wait for connection property. In case of application thread, there is an automatic mechanism which detects unclosed connections and unfinished transactions.

at com.sap.engine.services.dbpool.cci.ConnectionFactoryImpl.getConnection(ConnectionFactoryImpl.java:59)

at com.sap.tc.cbs.dbapi.fw.CbsDbPool.getConnection(CbsDbPool.java:73)

at com.sap.tc.cbs.server.db.impl.DBAccess.getConnection(DBAccess.java:99)

at com.sap.tc.cbs.server.rt.impl.ResourceOrc.act(ResourceOrc.java:182)

at com.sap.tc.cbs.server.rt.impl.CourteousTimer.run(CourteousTimer.java:104)

at java.lang.Thread.run(Thread.java:534)

Caused by: com.sap.engine.services.connector.exceptions.BaseResourceException: Cannot get connection for 120 seconds. Possible reasons: 1) Connections are cached within SystemThread(can be any server service or any code invoked within SystemThread in the SAP J2EE Engine), 2) The pool size of adapter "SAPEPDDB" is not enough according to the current load of the system or 3) The specified time to wait for connection is not enough according to the pool size and current load of the system. In case 1) the solution is to check for cached connections using the Connector Service list-conns command, in case 2) to increase the size of the pool and in case 3) to increase the time to wait for connection property. In case of application thread, there is an automatic mechanism which detects unclosed connections and unfinished transactions.

at com.sap.engine.services.connector.jca.ConnectionHashSet.match(ConnectionHashSet.java:272)

at com.sap.engine.services.connector.jca.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)

at com.sap.engine.services.dbpool.cci.ConnectionFactoryImpl.getConnection(ConnectionFactoryImpl.java:51)

... 5 more

#

[Former Member · Accepted solution]

Hi Amit,

no idea about this, sorry. 120 seconds seems long enough so that timeouts should not be the cause and 80 connections sounds sensible to me. Most installations I saw used 100, but I rarely saw them all used up.

Is the database on the same machine? Is there perhaps some machine specific limit? (some kind of firewall, processes not getting enough file descriptors, etc.). Just guessing here, though.

BTW: the stacktrace indicates that this is a CBS issue, the original problem should come from DTR (i.e. some com.sap.dtr.* or com.tssap.dtr.* package in the stacktrace).

Regards,

Marc

[Former Member · Accepted solution]

Hi Marc,

Yup 80 seems fine 2 me 2.

pretty much stumped as well.

the db is on the same machine.

sap should get access to the server today hopefully [system admin unreachable last evening]

will update the thread when they tell me why

)

amit

[Former Member]

Hello,

the increase in physical RAM on the box from 2 GB to 3.5 GB seems to have worked.

The ESS source has been successfully imported into Dev, now i am importing into Cons.

with respect,

amit

[Former Member]

Hi Tim,

how are you doing ?

we still face this problem, but your post definitely helped. thank you.

MSS has gone thru.

SAP believes that it is a space issue on the installation drive for ESS to be imported.

Hopefully we will have a resolution soon.

with respect,

amit

[Former Member]

Amit,

Check your DTR Logs. There are some background tasks that run and sometimes if they aren't successful they hang. So when the CMS needs to connect to the DTR to, for example, extract the source, it can't because the DTR isn't responding, no it hangs too naturally...until it times out.

After reviewing the logs, I was able to determine that there was an error with the NameReservationsTask. The error was that it could not connect to the the SLD because in the Name Server Configuration of the DTR

(http://<host>:<httpPort>/dtr/system-tools/administration/NameServerConfiguration),we didn't put a Fully Qualified Domain Name, so it would just hang because it could not connect. More details could be found in this thread, have a look and see if it applies to you

Unfortunately though, we aren't able remove the task... and are stuck on that part, so I'm hoping to see if this is the same issue you are experiencing and if so, you could oblige us with a solution.

[Former Member]

Hi Tim,

how r u doing ?

happy new year 2 u.

thx for the thread, was helpful to learn about the queues.

in my case the name server is fine.

all the routine checks have not yeilded any results.

SAP should have a look at my system mid next week.

[would have been sooner, but it's holiday time and no one's around ...]

i will definitely update the thread once a solution is found

with respect,

amit

[Former Member]

Amit,

Just to update, the issue for us turned out to be addressed by

SAP NOTE: 985821 DTR Import Hangs on MSSQL Database

Not saying this will help you, but it did help us.

Cheers,

-Tim

[Former Member]

We have this same issue... greatly appreciated if anyone knows how to resolve it.