FAIL: process hdbindexserver HDB Indexserver not running

[former_member183087]

Hello Experts

When I am trying to start the HDB system, I am getting the below error:

StartService

OK

OK

Starting instance using: /usr/sap/HDB/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 00 -function StartWait 2700 2

14.05.2013 17:24:47

Start

OK

14.05.2013 17:26:32

StartWait

FAIL: process hdbindexserver HDB Indexserver not running

Can someone point me to any clue here ?

Thanks

Sri

[simon25]

Hi,

There are multiple causes for this type of service startup issue. You really need to investigate the specific service trace like the indexserver from the point of startup: "Starting hdbindexserver <revision number>" and follow the sequence of events until an exception or error is encountered. A lot of the time it comes down to persistence or directories like in these scenarios :

3109802 - HS - process hdbindexserver HDB Indexserver not running

2352487 - Indexserver failing to start due to missing log segment(s).

2356465 - Error: the log directory file is missing when HANA startup

Kind Regards

Simon

[nir_benami2]

Hi Vijay

You can run the following command to check the disk space: df -h

The location of the log files is:

/hana/shared/HDB/HDB02/backup/log/DB_HDB

Do not remove all, you should at least keep the files from the last 7 days.

Greetings

Nir

[nir_benami2]

Dear All

Here is what I did:

HDB stop (successful)

HDB start (FAIL: process hdbdaemon HDB Daemon not running)

Checked the file system: it was full from the log backup, I removed old logs, free disk space dropped to 28% from 100% full.

HDB start (successful)

Greetings

[former_member757024]

can i know where did you found the older logs and removed?

[former_member565891]

Hi Dear,

I face the same problem, so could you please share the solution with me?

Regards,

[Former Member]

The indexserver_alert_hanaserver.trc shows:

[25181]{0}[-794253] 2013-05-14 19:00:23.304935 e Basis

FaultProtectionImpl.cpp(00964) : SIGNAL 11 (SIGSEGV) caught, thread: 2785[thr=25181]: Assign, addr: 0xfffffffffffe00c8, time: 2013-05-14 19:00:23 000 Local

Instance HDB/00, OS Linux hanaserver 2.6.32.27-0.2-default #1 SMP 2010-12-29 15:03:02 +0100 x86_64

followed by register dump and a pointer to full core dump

Call stack of crashing context:

1: 0x00007f3075b34693 in ptime::VarGroup::slot() const+0x60 at SegMap.h:143 (libhdbrskernel.so)

2: 0x00007f3075b34055 in ptime::VarGroup::destroy(ptime::Transaction&)+0x21 at VarGroup.cc:12 (libhdbrskernel.so)

3: 0x00007f307579c7ce in ptime::Catalog::reclaim_row_table_space_for_restart(ptime::Transaction&, int, ltt_adp::vector<ptime::ClassPtr<ptime::FieldInfo>, ltt::integral_constant<bool, true> >&, ltt_adp::vector<ptime::ClassPtr<ptime::FieldInfo>, ltt::integral_constant<bool, true> >&, ltt_adp::vector<ptime::ClassPtr<ptime::FieldInfo>, ltt::integral_constant<bool, true> >&, ltt_adp::vector<ptime::ClassPtr<ptime::FieldInfo>, ltt::integral_constant<bool, true> >&, unsigned long)+0x1aa at catalog.cc:7109 (libhdbrskernel.so)

4: 0x00007f3075d27e36 in ptime::QueryExecutor::post_drop_in_memory_table(ptime::Transaction&, ptime::ClassPtr<ptime::TableInfo>, int, bool, bool)+0x222 at qx_table.cc:2303 (libhdbrskernel.so)

5: 0x00007f3075d2cf43 in ptime::QueryExecutor::post_drop_table(ptime::Transaction&, ptime::ClassPtr<ptime::TableInfo>, bool)+0xd0 at qx_table.cc:2140 (libhdbrskernel.so)

6: 0x00007f307676e6c3 in ptime::MemoryMgr::cleanupDeletedTables()+0x300 at mm_recovery.cc:871 (libhdbrskernel.so)

7: 0x00007f30764a35c5 in ptime::Master::recover_memorydb_after_tm_init()+0x91 at ptime_master_start.cc:733 (libhdbrskernel.so)

8: 0x00007f30764a3edd in ptime::Master::poststartup()+0x39 at ptime_master_start.cc:166 (libhdbrskernel.so)

9: 0x00007f3076568be5 in ptime::DriverManager::poststartup()+0x11 at DriverManager.cc:812 (libhdbrskernel.so)

10: 0x00007f308ca0309e in PersistenceController::startup(PersistenceLayer::PERSISTENCE_MODE, NameServer::ServiceStartInfo*, bool, TREX_ERROR::TRexError*, DataAccess::TablePreloadWriteCallback*, DataAccess::TablePreloadReadCallback*)+0x3ea at PersistenceController.cpp:442 (libhdblogger.so)

11: 0x00000000004c1ac0 in TRexAPI::TREXIndexServer::assign(NameServer::ServiceStartInfo&, bool, TREX_ERROR::TRexError&)+0x130 at TREXIndexServer.cpp:414 (hdbindexserver)

12: 0x000000000054510d in TRexAPI::AssignThread::run(void*)+0x39 at TREXIndexServer.cpp:278 (hdbindexserver)

13: 0x00007f308d4e56fa in TrexThreads::PoolThread::run()+0xac6 at PoolThread.cpp:258 (libhdbbasement.so)

14: 0x00007f308d4e6ea8 in TrexThreads::PoolThread::run(void*&)+0x14 at PoolThread.cpp:105 (libhdbbasement.so)

15: 0x00007f30806e9bf2 in Execution::Thread::staticMainImp(void**)+0x6a0 at Thread.cpp:455 (libhdbbasis.so)

16: 0x00007f30806e9ddd in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:526 (libhdbbasis.so)

[3077]{0}[-796302] 2013-05-14 19:08:03.222764 e Basis        FaultProtectionImpl.cpp(00964) : SIGNAL 11 (SIGSEGV) caught, thread: 2785[thr=3077]: Assign, addr: 0xfffffffffffe00c8, time: 2013-05-14 19:08:03 000 Local

In /var/log/messages we get:

May 14 19:45:19 hanaserver SAPHDB_00[6889]: Unable to open trace file sapstartsrv.log. (Error 11 Resource temporarily unavailable) [/bas/802_REL/src/ext/agent/ntserv/ntservsserver.cpp 3372]

[swapan_saha]

Hi  Venkatesh,

Please check out whether Rev 52 (just released) have the same issue or not. For additional information about SAP HANA One Rev 52, please check out .

Thanks,

Swapan

[Former Member]

Hi Swapan,

Thanks for the reply. We will look into Rev 52. But more importantly, we wanted to bring back our Hana one instance - rev48. Are you suggesting that we can migrate our data and objects to the new instance without restarting the rev 48 instance ?

regards,

Venkatesh

[swapan_saha]

Hi Venkatesh,

Please follow the section on “Upgrading your instance to HANA Rev 52” in  .

You will be able to migrade schema and content using HANA export and import utilities from Rev 48 to Rev 52.

Thanks,

Swapan

[Former Member]

Hi Swapan,

I looked at the document you sent and it looks like I will need to export from the old instance before importing. We cannot bring up the old system to export the schema and contents. That is the reason we are trying to bring up the old one first, unless you have a work around. Please treat this as urgent as this is a productive instance and it has been down for more than 48 hours.

Please let me know if you need more information around the dump.

regards,

Venkatesh

[Former Member]

Hi Swapan,

Thanks for the help. We were able to recover our database and port it onto Rev 52. But when I go to the Hana Studio I get a message that the License is not valid. How do I obtain a new license key for Rev 52 ?

regards

Venkatesh

[swapan_saha]

Venkatesh,

Backup from system 1 and recovery into system 2 is not supported in HANA One yet which has been documented in the document we mentioned. We recommended export/import utilities which will not export contents (schema, data, models) and then you can import them in a second system.

Thanks,

Swapan

[Former Member]

Hi Swapan,

You are right. We were able to move our content with the import export functionality. It is fine now. Thanks for all your help.

regards,

Venkatesh

[former_member198774]

Hi All ,

I am also facing the same issue while start the HANA Database we have getting error like as

"fail : process hdbindexserver HDB indexserver not runing".

Please attached trace file and provide a resolution.

HANA LOGS.rar - Google Drive

Thanks,

Vijay.

[Former Member]

Hi Sri,

It seems like HANA DB did not shut down correctly.

I would suggest trying to shut down HANA DB one more time by running the following command as hdbadm user

shell >> HDB stop

And then try to start HANA DB by running

shell >> HDB start

Let us know if this solves the problem.

Thanks,

Mudit

[former_member183087]

Hi Mudit

Yes, I have tried these steps and during the start I am getting the error. Can you please point me to some logs where I can check the error ?

Thanks

Sri

[Former Member]

Hi Sri,

Log and trace files of interest are under /usr/sap/HDB/HDB00/hanaserver/trace

Meanwhile, I am also looking at the issue.

Thanks,

Mudit

[former_member183087]

Thanks, I am also searching for some hints in the logs located in the trace directory.