J2EE server become unresponsive - MaxDB error

[Former Member]

Hi All,

We have a j2ee server (Netweaver 7.0) with MaxDB. It often becomes unresponsive when we try to access any url like portal or DI. But the server instance shows green.

When I searched log files (default trace) i found the following error.

#1.5 #0013726696BF0089005F394E00001A240004633EF5BC2335#1235021968703#com.sap.caf.eu.gp.model.pfw.notification.NotificationQueueProcessor##com.sap.caf.eu.gp.model.pfw.notification.NotificationQueueProcessor#Guest#0##n/a##5039da54fdf011dd80c30013726696bf#SAPEngine_Application_Thread[impl:3]_31##0#0#Error#1#/System/Server#Plain###ERROR_ACQUIRE_CONNECTION

com.sap.caf.eu.gp.base.exception.EngineException: ERROR_ACQUIRE_CONNECTION

* at com.sap.caf.eu.gp.base.db.ConnectionPoolJ2EE.getConnection(ConnectionPoolJ2EE.java:92)*

* at com.sap.caf.eu.gp.base.db.TransactionController.openTransaction(TransactionController.java:159)*

* at com.sap.caf.eu.gp.model.queue.QueueFactory.readQueueItemIdentifier(QueueFactory.java:1786)*

* at com.sap.caf.eu.gp.model.pfw.notification.NotificationQueueFactory.getNextQueueItemID(NotificationQueueFactory.java:362)*

* at com.sap.caf.eu.gp.model.pfw.notification.NotificationQueueProcessor.run(NotificationQueueProcessor.java:152)*

* at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)*

* at java.security.AccessController.doPrivileged(Native Method)*

* at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:102)*

* at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:172)*

Caused by: com.sap.engine.services.dbpool.exceptions.BaseSQLException: ResourceException in method ConnectionFactoryImpl.getConnection(): com.sap.engine.services.dbpool.exceptions.BaseResourceException: SQLException thrown by the physical connection: com.sap.dbtech.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: Cannot connect to jdbc:sapdb://APRINS05/EP5 [Receive of connect failed.].

* at com.sap.engine.services.dbpool.cci.ConnectionFactoryImpl.getConnection(ConnectionFactoryImpl.java:59)*

* at com.sap.caf.eu.gp.base.db.ConnectionPoolJ2EE.getConnection(ConnectionPoolJ2EE.java:89)*

* ... 8 more*

Caused by: com.sap.engine.services.dbpool.exceptions.BaseResourceException: SQLException thrown by the physical connection: com.sap.dbtech.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: Cannot connect to jdbc:sapdb://APRINS05/EP5 [Receive of connect failed.].

* at com.sap.engine.services.dbpool.spi.ManagedConnectionFactoryImpl.createManagedConnection(ManagedConnectionFactoryImpl.java:193)*

* at com.sap.engine.services.connector.jca.ConnectionHashSet.match(ConnectionHashSet.java:320)*

* at com.sap.engine.services.connector.jca.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:188)*

* at com.sap.engine.services.dbpool.cci.ConnectionFactoryImpl.getConnection(ConnectionFactoryImpl.java:51)*

* ... 9 more*

Caused by: com.sap.dbtech.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: Cannot connect to jdbc:sapdb://XXXXXX/EP5 [Receive of connect failed.].

* at com.sap.dbtech.jdbc.DriverSapDB.connect(DriverSapDB.java:188)*

* at com.sap.sql.jdbc.NativeConnectionFactory.createNativeConnection(NativeConnectionFactory.java:215)*

* at com.sap.sql.connect.OpenSQLDataSourceImpl.createPooledConnection(OpenSQLDataSourceImpl.java:608)*

* at com.sap.sql.connect.OpenSQLDataSourceImpl.getPooledConnection(OpenSQLDataSourceImpl.java:285)*

* at com.sap.engine.services.dbpool.spi.ManagedConnectionFactoryImpl.createManagedConnection(ManagedConnectionFactoryImpl.java:125)*

* ... 12 more*

So i decided that there is some problem with the database and gone through the +knldiag+ log file and found the following

2009-02-19 00:43:20 0x1534 13808 RTE System page cache released no free blocks

2009-02-19 00:43:20 0x1534 13810 RTE System page cache still holding 31293 pages in 2 splitted blocks

2009-02-19 00:43:20 0x1534 ERR 54008 MEMORY TransContext T098 : could not allocate

2009-02-19 00:43:20 0x1534 ERR 54008 MEMORY memory

2009-02-19 00:43:20 0x1534 ERR 54008 MEMORY required : 134217744

2009-02-19 00:43:20 0x1534 ERR 54008 MEMORY allocated : 389120

2009-02-19 00:43:20 0x1534 ERR 54008 MEMORY supplement : 134221824

2009-02-19 00:43:21 0x1534 ERR 54008 MEMORY limit : -1

2009-02-19 00:43:21 0x1534 ERR 54008 MEMORY free blocks size 24 : 4

2009-02-19 00:43:21 0x1534 ERR 54008 MEMORY free blocks size 184 : 1

2009-02-19 00:43:21 0x1534 54000 MEMORY free blocks size 1208 : 1

2009-02-19 00:43:21 0x1534 54000 MEMORY free blocks size 1480 : 1

2009-02-19 00:43:21 0x1534 54000 MEMORY free blocks size 6600 : 1

2009-02-19 00:43:21 0x1534 54000 MEMORY free blocks size 11792 : 1

2009-02-19 00:43:21 0x1534 ERR 18793 EXCEPT EXCEPTION:'Access violation' (0xc0000005), The program code at IP:0x93abfe attempted to write to/from address:0

Can anyone help me to solve this error.

Regards,

Tamil K

[Former Member]

Hi,

Possibly you could increase the Connection Pool on ur j2ee.

Please check for details on the below SAP note, which speaks on this issue.

Note 1168546 - NotificationQueueProcessor cycles fast on no connections

Hope this helps you.

Regards,

Prem

[Benny]

Are you sure you database does run at all?

Regards,

Benny

[Former Member]

Hi Benny,

By that time my DB was stopped. Here is the another part of knldiag log which clearly says DB was stopped.

2009-02-19 00:43:40 0x834 ERR 19999 BTRACE Mini dump file 'KnlMini.dmp' written!

2009-02-19 00:43:40 0x1534 ERR 7 Messages Begin of dump of registered messages

2009-02-19 00:43:40 0x1534 ERR 8 Messages End of the message list registry dump

2009-02-19 00:43:40 0x1534 19820 EXCEPT calling ak dump for T98

2009-02-19 00:43:40 0x1534 19684 TASKING Tracewriter termination timeout: 1200 sec

2009-02-19 00:43:40 0x1154 20000 Trace Start flush kernel trace

2009-02-19 00:43:40 0x1154 19617 VOLUMEIO Single I/O attach, 'knltrace', UKT:1

2009-02-19 00:43:50 0x1154 20001 Trace Stop flush kernel trace

2009-02-19 00:43:50 0x1154 20002 Trace Start flush kernel dump

2009-02-19 00:46:09 0x1154 20003 Trace Stop flush kernel dump

2009-02-19 00:46:09 0x1154 19619 TASKING Releasing tracewriter

2009-02-19 00:46:09 0x1534 19890 DIAGHIST Backup of diagnostic files will be forced at next restart

2009-02-19 00:46:09 0x1534 19600 VERSION 'Kernel 7.6.00 Build 035-123-139-084'

2009-02-19 00:46:09 0x1534 19654 DBSTATE SERVERDB 'EP5' has stopped

After this I manually restarted the DB and then server instance. But for that time the server runs fine and after some time again it becomes unresponsive.

Some times as you asked the DB will also be running in services but the server become unresponsive. During that time we can find the same log (given earlier) in the default trace. Again i have to restart the DB service and server to get response.

Regards,

Tamil K

Edited by: Tamil Venthan on Feb 19, 2009 11:31 AM