cancel
Showing results for 
Search instead for 
Did you mean: 

HANA DB will not start

Former Member
0 Kudos

Hi,

We have installed HDB (version: 1.00.82.00.394270) on which SAP ERP 6.0 EHP7 is installed. The systems were running fine, however, after a scheduled reboot of the HANA server, HDB will not start. (Prior to this we have ran a test-run of client-copy which failed, if this information is in any way relevant.)

Except nameserver, preprocessor and compileserver, all other HDB processes just stay in 'Initializing' phase. On checking the trace files, noticed errors related to memory. However, I can see memory resources available.

Any help on troubleshooting further is much appreciated.

If i am posting on wrong group, then please move this to the correct group.

Regards,

Naga

Trace (indexserver_hostname---.trc) looks as below:

---

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515081 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 5682560B, flags 0x0

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 5682560 byte.

Current callstack:

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515308 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 17402784B, flags 0x0

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 17402784 byte.

Current callstack:

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516637 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/malloc/libhdbrskernel.so, size 16933920B, flags 0x0

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 16933920 byte.

Current callstack:

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517927 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 16735968B, flags 0x0

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 16735968 byte.

Current callstack:

1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : Information about shrink at 10.11.2014 13:12:47 000 Mon:

Reason for shrink: Precharge for small block allocation. User size: 256

ShrinkCaller

1: 0x00007f6011b151c8 in ResourceManager::StackFrames::fillCallstack()+0x64 at ResourceContainer.cpp:596 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587659 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 48B, flags 0x0

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 48 byte.

Current callstack:

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 2: 0x00007f6011b15316 in ResourceManager::ShrinkInformation::updateShrinkInformation(char const*, unsigned long)+0x62 at ResourceContainer.cpp:1220 (libhdbbasis.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 3: 0x00007f6011b0a048 in ResourceManager::reportShrink(ResourceManager::ShrinkInformation&, char const*, unsigned long)+0x64 at ResourceContainer.cpp:1299 (libhdbbasis.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616679 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 6234496B, flags 0x0

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 6234496 byte.

Current callstack:

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616845 w Memory PoolAllocator.cpp(01059) : Out of memory for Pool/IndexRebuildAllocator, size 6143776B, flags 0x0

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : OUT OF MEMORY occurred.

Failed to allocate 6143776 byte.

Current callstack:

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 1: 0x00007f6011aed9cd in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x279 at PoolAllocator.cpp:1069 (libhdbbasis.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 4: 0x00007f6011b0f933 in ResourceManager::ResourceContainerImpl::shrink(unsigned long, unsigned long, char const*, unsigned long, unsigned long&, bool)+0x5f0 at ResourceContainer.cpp:3404 (libhdbbasis.so)

5: 0x00007f6011b19034 in ResourceManager::ResourceContainerImpl::compactMemory(MemoryManager::MemoryCompactor::CompactionInfoBlock&, unsigned long, unsigned long, char const*, unsigned long)+0x50 at ResourceContainer.cpp:2465 (libhdbbasis.so)

6: 0x00007f6011aa82b2 in MemoryManager::CompactionThread::compactCompactors(unsigned long, unsigned long, unsigned long&, MemoryManager::CompactionTasks*, char const*, unsigned long)+0x5c0 at CompactionTrigger.cpp:515 (libhdbbasis.so)

7: 0x00007f6011aa904e in MemoryManager::CompactionThread::doCompaction(unsigned long, unsigned long, bool, bool, char const*, int, char const*, unsigned long)+0x6fa at CompactionTrigger.cpp:619 (libhdbbasis.so)

8: 0x00007f6011aba94e in MemoryManager::GlobalMemoryHandler::callCompaction(MemoryManager::ProcessSlot&, unsigned long, unsigned long, MemoryManager::RequestorInfo const&, unsigned long, MemoryManager::AllocatorConfigurationSPI&, char const*, unsigned long)+0x26a at GlobalMemoryHandler.cpp:4068 (libhdbbasis.so)

9: 0x00007f6011abb3ac in MemoryManager::GlobalMemoryHandler::requestCompactions(MemoryManager::RequestorInfo const&, bool, MemoryManager::PMInfo&, char const*, unsigned long)+0x138 at GlobalMemoryHandler.cpp:3850 (libhdbbasis.so)

10: 0x00007f6011abc668 in MemoryManager::GlobalMemoryHandler::pmInterProcessProvideMemory(MemoryManager::PMInfo&, MemoryManager::RequestorInfo const&, char const*, unsigned long)+0x1c4 at GlobalMemoryHandler.cpp:4900 (libhdbbasis.so)

11: 0x00007f6011abcead in MemoryManager::GlobalMemoryHandler::provideMemory(unsigned long, unsigned long, bool, MemoryManager::AllocatorConfigurationSPI&, bool, char const*, unsigned long)+0x639 at GlobalMemoryHandler.cpp:3297 (libhdbbasis.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 12: 0x00007f6011a72cf6 in MemoryManager::AllocatorConfigurationSPI::provideMemory(unsigned long, unsigned long, bool, char const*, unsigned long)+0x22 at AllocatorConfigurationImpl.cpp:2223 (libhdbbasis.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 13: 0x00007f6011aced94 in MemoryManager::MemoryPool::allocateAligned(unsigned long, unsigned long, unsigned short, ltt::allocator_statistics&, bool&, bool, void const*)+0x550 at MemoryPool.cpp:1631 (libhdbbasis.so)

14: 0x00007f6011aed30f in MemoryManager::PoolAllocator::allocateAlignedNoThrowImpl(unsigned long, unsigned long, void const*)+0x6b at PoolAllocator.cpp:1113 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 2: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 15: 0x00007f600405c841 in ptime::cpb::Allocator::allocateLeaf(ptime::cpb::NodeReservior&, ptime::m_Ref_Any&)+0x30 at pcc_MonitorAlloc.h:107 (libhdbrskernel.so)

16: 0x00007f600403f091 in ptime::cpb::Leaf::create(ptime::cpb::NodeReservior&, ptime::m_Ref<ptime::cpb::Allocator> const&, bool, bool, ptime::m_Ref<ptime::cpb::Leaf>&)+0x20 at cpb_leaf.cc:1014 (libhdbrskernel.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004198e7e in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x3ca at vector.hpp:883 (libhdbrskernel.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 17: 0x00007f6004055017 in ptime::cpb::Tree::loadLeaves(ptime::cpb::Txn&, ptime::cpb::LoadEntry const*, unsigned long, float, bool, ptime::cpb::Stat*)+0x163 at cpb_tree.cc:2236 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

6: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 7: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f60040dfb59 in ptime::IndexRebuildParallelScan::run(Execution::JobObject&)+0x475 at memory.hpp:769 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:00.516678 e Memory ReportMemoryProblems.cpp(00733) : 8: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

9: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 3: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 18: 0x00007f600404c425 in ptime::cpb::Tree::load(ptime::cpb::LoadEntry const*, unsigned long, float, ptime::Transaction*)+0x121 at cpb_tree.cc:5218 (libhdbrskernel.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23020]{-1}[9/-1] 2014-11-10 13:34:00.571121 w ResMan ResourceContainer.cpp(01300) : 19: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

20: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

21: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

22: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

23: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22160453632, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132971, FSL=0

Total allocated memory= 47460979568b (44.20gb)

Total used memory = 39839253850b (37.10gb)

Sum AB = 29826617344

Sum Used = 22204891626

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

5: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23017]{-1}[9/-1] 2014-11-10 13:34:00.587757 e Memory ReportMemoryProblems.cpp(00733) : 7: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

8: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : Information about shrink at 10.11.2014 13:12:47 000 Mon:

Reason for shrink: Precharge for big block allocation. User size: 6410816

ShrinkCaller

1: 0x00007f6011b151c8 in ResourceManager::StackFrames::fillCallstack()+0x64 at ResourceContainer.cpp:596 (libhdbbasis.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 2: 0x00007f6011b15316 in ResourceManager::ShrinkInformation::updateShrinkInformation(char const*, unsigned long)+0x62 at ResourceContainer.cpp:1220 (libhdbbasis.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 7: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

8: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 3: 0x00007f6011b0a048 in ResourceManager::reportShrink(ResourceManager::ShrinkInformation&, char const*, unsigned long)+0x64 at ResourceContainer.cpp:1299 (libhdbbasis.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23014]{-1}[9/-1] 2014-11-10 13:34:00.616722 e Memory ReportMemoryProblems.cpp(00733) : 9: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

10: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 4: 0x00007f6011b0f933 in ResourceManager::ResourceContainerImpl::shrink(unsigned long, unsigned long, char const*, unsigned long, unsigned long&, bool)+0x5f0 at ResourceContainer.cpp:3404 (libhdbbasis.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 5: 0x00007f6011b19034 in ResourceManager::ResourceContainerImpl::compactMemory(MemoryManager::MemoryCompactor::CompactionInfoBlock&, unsigned long, unsigned long, char const*, unsigned long)+0x50 at ResourceContainer.cpp:2465 (libhdbbasis.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 7: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 8: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

9: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 6: 0x00007f6011aa82b2 in MemoryManager::CompactionThread::compactCompactors(unsigned long, unsigned long, unsigned long&, MemoryManager::CompactionTasks*, char const*, unsigned long)+0x5c0 at CompactionTrigger.cpp:515 (libhdbbasis.so)

7: 0x00007f6011aa904e in MemoryManager::CompactionThread::doCompaction(unsigned long, unsigned long, bool, bool, char const*, int, char const*, unsigned long)+0x6fa at CompactionTrigger.cpp:619 (libhdbbasis.so)

[23013]{-1}[9/-1] 2014-11-10 13:34:00.515131 e Memory ReportMemoryProblems.cpp(00733) : 10: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 8: 0x00007f6011aba94e in MemoryManager::GlobalMemoryHandler::callCompaction(MemoryManager::ProcessSlot&, unsigned long, unsigned long, MemoryManager::RequestorInfo const&, unsigned long, MemoryManager::AllocatorConfigurationSPI&, char const*, unsigned long)+0x26a at GlobalMemoryHandler.cpp:4068 (libhdbbasis.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 4: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 9: 0x00007f6011abb3ac in MemoryManager::GlobalMemoryHandler::requestCompactions(MemoryManager::RequestorInfo const&, bool, MemoryManager::PMInfo&, char const*, unsigned long)+0x138 at GlobalMemoryHandler.cpp:3850 (libhdbbasis.so)

10: 0x00007f6011abc668 in MemoryManager::GlobalMemoryHandler::pmInterProcessProvideMemory(MemoryManager::PMInfo&, MemoryManager::RequestorInfo const&, char const*, unsigned long)+0x1c4 at GlobalMemoryHandler.cpp:4900 (libhdbbasis.so)

11: 0x00007f6011abcead in MemoryManager::GlobalMemoryHandler::provideMemory(unsigned long, unsigned long, bool, MemoryManager::AllocatorConfigurationSPI&, bool, char const*, unsigned long)+0x639 at GlobalMemoryHandler.cpp:3297 (libhdbbasis.so)

12: 0x00007f6011a72cf6 in MemoryManager::AllocatorConfigurationSPI::provideMemory(unsigned long, unsigned long, bool, char const*, unsigned long)+0x22 at AllocatorConfigurationImpl.cpp:2223 (libhdbbasis.so)

13: 0x00007f6011acd305 in MemoryManager::MemoryPool::allocateBigOrHugeBlock(unsigned long, unsigned short, ltt::allocator_statistics&, void const*, bool, Synchronization::LockHandle<Synchronization::Mutex, false>&, bool)+0x2c1 at MemoryPool.cpp:1453 (libhdbbasis.so)

14: 0x00007f6011acdfba in MemoryManager::MemoryPool::allocate(unsigned long, unsigned short, ltt::allocator_statistics&, bool&, bool, void const*)+0x3a6 at MemoryPool.cpp:1358 (libhdbbasis.so)

15: 0x00007f6011aed7bf in MemoryManager::PoolAllocator::allocateNoThrowImpl(unsigned long, void const*)+0x6b at PoolAllocator.cpp:1041 (libhdbbasis.so)

16: 0x00007f60048915cf in ltt::allocator::allocate(unsigned long)+0x2b at memory.cpp:61 (libhdbrskernel.so)

[23026]{-1}[9/-1] 2014-11-10 13:34:00.616924 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

7: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

8: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

9: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

10: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 17: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 5: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 18: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

[23021]{-1}[9/-1] 2014-11-10 13:34:00.517949 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

7: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

8: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

9: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

10: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 6: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

7: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

8: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

9: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 19: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

[23313]{-1}[9/-1] 2014-11-10 13:34:00.515398 e Memory ReportMemoryProblems.cpp(00733) : 10: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

Memory consumption information of last failing ProvideMemory, PM-INX=1481:

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22148618180, UA=0, U=18032645107, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132811, FSL=0

Total allocated memory= 47449144116b (44.19gb)

Total used memory = 39839253690b (37.10gb)

Sum AB = 29814781892

Sum Used = 22204891466

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18032645107b (16.79gb)

2: Pool 17523085064b (16.31gb)

3: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1173029000b (1.09gb)

6: Pool/malloc 897550376b (855.97mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 103658576b (98.85mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13722998392b (12.78gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 704795464b (672.14mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 103658576b (98.85mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23015]{-1}[9/-1] 2014-11-10 13:34:01.610575 w ResMan ResourceContainer.cpp(01300) : 20: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

21: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

22: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

23: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

24: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

IPMM short info:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb), SHARED_MEMORY = 9937798000b (9.25gb), CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22152437760, UA=0, U=18015710283, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132971, FSL=0

Total allocated memory= 47452963696b (44.19gb)

Total used memory = 39822319026b (37.08gb)

Sum AB = 29818601472

Sum Used = 22187956802

Heap memory fragmentation: 16% (this value may be high if defragmentation does not help solving the current memory request)

Top allocators (ordered descending by inclusive_size_in_use).

1: / 18024378666b (16.78gb)

2: Pool 17514817448b (16.31gb)

3: Pool/IndexRebuildAllocator 13705916504b (12.76gb)

4: Pool/PersistenceManager 1180462240b (1.09gb)

5: Pool/RowEngine 1175630152b (1.09gb)

6: Pool/malloc 903798216b (861.92mb)

7: Pool/RowEngine/LockTable 805307776b (768mb)

8: Pool/malloc/libhdbrskernel.so 711043304b (678.10mb)

9: Pool/PersistenceManager/PersistentSpace(0) 505951328b (482.51mb)

10: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

11: AllocateOnlyAllocator-unlimited 411556888b (392.49mb)

12: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

13: Pool/RowEngine/Internal 236112384b (225.17mb)

14: Pool/Statistics 186062256b (177.44mb)

15: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

16: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1> 182452224b (174mb)

17: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

18: Pool/ResourceContainer 167960168b (160.17mb)

19: Pool/RowEngine/CpbTree 106259728b (101.33mb)

20: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

Top allocators (ordered descending by exclusive_size_in_use).

1: Pool/IndexRebuildAllocator 13705916504b (12.76gb)

2: Pool/RowEngine/LockTable 805307776b (768mb)

3: Pool/malloc/libhdbrskernel.so 711043304b (678.10mb)

4: Pool/PersistenceManager/ContainerFileIDMapping 436248880b (416.03mb)

5: Pool/PersistenceManager/PersistentSpace(0)/RowStoreLPA 343408992b (327.50mb)

6: Pool/RowEngine/Internal 236112384b (225.17mb)

7: Pool/Statistics 186062256b (177.44mb)

8: AllocateOnlyAllocator-unlimited/FLA-UL<3145728,1>/MemoryMapLevel2Blocks 182452224b (174mb)

9: Pool/PersistenceManager/ContainerNameDirectory 168904504b (161.07mb)

10: Pool/ResourceContainer 167960168b (160.17mb)

11: Pool/RowEngine/CpbTree 106259728b (101.33mb)

12: Pool/malloc/libhdbexpression.so 90575440b (86.37mb)

13: Pool/malloc/libhdbbasement.so 90332152b (86.14mb)

14: Pool/PersistenceManager/PersistentSpace(0)/DefaultLPA/Page 86376448b (82.37mb)

15: AllocateOnlyAllocator-unlimited/ReserveForUndoAndCleanupExec 84029440b (80.13mb)

16: AllocateOnlyAllocator-unlimited/ReserveForOnlineCleanup 84029440b (80.13mb)

17: Pool/TransactionManager 75505912b (72mb)

18: Pool/PersistenceManager/PersistentSpace(0)/DynamicConverter/ConvPage 65798144b (62.75mb)

19: StackAllocator 54747136b (52.21mb)

20: AllocateOnlyAllocator-limited/ResourceHeader 43100000b (41.10mb)

[23022]{-1}[9/-1] 2014-11-10 13:34:09.642159 e Memory ReportMemoryProblems.cpp(00793) : Finished writing oom runtime dump!

[22990]{-1}[-1/-1] 2014-11-10 13:35:20.817921 f PersistenceLayer PersistenceController.cpp(00604) : startup failed exception 1: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16999648; $name$=libhdbrskernel.so; $type$=pool; $inuse_count$=8795; $allocated_size$=698308840

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004198e7e in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x3ca at vector.hpp:883 (libhdbrskernel.so)

3: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

4: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

5: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

6: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

7: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 2: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16933920; $name$=libhdbrskernel.so; $type$=pool; $inuse_count$=8796; $allocated_size$=704795464

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004198e7e in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x3ca at vector.hpp:883 (libhdbrskernel.so)

3: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

4: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

5: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

6: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

7: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 3: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16735968; $name$=IndexRebuildAllocator; $type$=pool; $inuse_count$=187277331; $allocated_size$=13476567560

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

3: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

4: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

5: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

6: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

7: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

8: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 4: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=17402784; $name$=IndexRebuildAllocator; $type$=pool; $inuse_count$=187255143; $allocated_size$=13475922008

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

3: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

4: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

5: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

6: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

7: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

8: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

[22990]{-1}[-1/-1] 2014-11-10 13:35:27.760722 i Logger BackupHandlerImpl.cpp(00321) : Shutting down log backup, 0 log backup(s) pending

[22990]{-1}[-1/-1] 2014-11-10 13:36:18.366386 e assign TREXIndexServer.cpp(00690) : assign failed with persistence startup error. exception 1: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16999648; $name$=libhdbrskernel.so; $type$=pool; $inuse_count$=8795; $allocated_size$=698308840

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004198e7e in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x3ca at vector.hpp:883 (libhdbrskernel.so)

3: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

4: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

5: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

6: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

7: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 2: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16933920; $name$=libhdbrskernel.so; $type$=pool; $inuse_count$=8796; $allocated_size$=704795464

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004198e7e in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x3ca at vector.hpp:883 (libhdbrskernel.so)

3: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

4: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

5: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

6: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

7: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 3: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=16735968; $name$=IndexRebuildAllocator; $type$=pool; $inuse_count$=187277331; $allocated_size$=13476567560

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

3: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

4: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

5: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

6: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

7: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

8: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

exception 4: no.1000002 (ltt/impl/memory.cpp:64)

Allocation failed ; $size$=17402784; $name$=IndexRebuildAllocator; $type$=pool; $inuse_count$=187255143; $allocated_size$=13475922008

exception throw location:

1: 0x00007f600408501d in _ZN3ltt6vectorIN5ptime3cpb9LoadEntryEE16allocateAndCopy_IPS3_EES6_mT_S7_.isra.3117+0x29 at memory.hpp:769 (libhdbrskernel.so)

2: 0x00007f6004197ec2 in ltt::vector<ptime::cpb::LoadEntry>::reserve_(unsigned long)+0x30 at vector.hpp:1061 (libhdbrskernel.so)

3: 0x00007f6004198d9d in ptime::_MergeSortMergeNode<ptime::cpb::LoadEntry>::run(Execution::JobObject&)+0x2e9 at vector.hpp:1156 (libhdbrskernel.so)

4: 0x00007f60119a6702 in Execution::JobObjectImpl::run(Execution::JobWorker*)+0x650 at JobExecutorImpl.cpp:496 (libhdbbasis.so)

5: 0x00007f60119bbffb in Execution::JobWorker::runJob(ltt::smartptr_handle<Execution::JobObjectForHandle>&)+0x347 at JobExecutorThreads.cpp:341 (libhdbbasis.so)

6: 0x00007f60119bd154 in Execution::JobWorker::run(void*&)+0x180 at JobExecutorThreads.cpp:505 (libhdbbasis.so)

7: 0x00007f60119e0ccf in Execution::Thread::staticMainImp(void**)+0x98b at Thread.cpp:475 (libhdbbasis.so)

8: 0x00007f60119e121d in Execution::Thread::staticMain(void*)+0x39 at Thread.cpp:545 (libhdbbasis.so)

stopping service...

[22896]{-1}[-1/-1] 2014-11-10 13:36:26.928748 i Service_Shutdown TrexService.cpp(05738) : Preparing for shutting service down

[23307]{-1}[-1/-1] 2014-11-10 13:36:26.955984 i assign TREXIndexServer.cpp(02622) : unassign from volume 4

[23307]{-1}[-1/-1] 2014-11-10 13:36:26.956379 i Service_Shutdown TREXIndexServer.cpp(02625) : Preparing to shutdown

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.021999 i Service_Shutdown TREXIndexServer.cpp(02653) : Stopping LOBGarbageCollectorThread

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.022945 i Service_Shutdown TREXIndexServer.cpp(02669) : Stopping EPM

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.023262 i Service_Shutdown TREXIndexServer.cpp(02673) : Stopping PlanningEngine

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.023683 i Service_Shutdown TREXIndexServer.cpp(02678) : Stopping SQL session service

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.027715 i Service_Shutdown TREXIndexServer.cpp(02712) : Stopping TRexApiSystem

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.027876 i Service_Shutdown TREXIndexServer.cpp(02716) : Stopping joinengine

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.030573 i Service_Shutdown TREXIndexServer.cpp(02720) : Stopping executor

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.041105 i Service_Shutdown TREXIndexServer.cpp(02724) : Stopping DebuggingBackend

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.042443 i Service_Shutdown TREXIndexServer.cpp(02728) : Stopping CalculationEngineManager

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.042561 i Service_Shutdown TREXIndexServer.cpp(02732) : Stopping CacheMgr

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.504567 i Service_Shutdown TREXIndexServer.cpp(02739) : Stopping GraphEngineMgr

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.505762 i Service_Shutdown TREXIndexServer.cpp(02743) : Stopping Invalidator

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.511577 i Service_Shutdown TREXIndexServer.cpp(02747) : Stopping RClient

[23307]{-1}[-1/-1] 2014-11-10 13:36:27.513017 i Service_Shutdown TREXIndexServer.cpp(02751) : Stopping SAMLFactory

[23307]{-1}[-1/-1] 2014-11-10 13:36:33.497299 i Service_Shutdown TREXIndexServer.cpp(02764) : Stopping udivmgr

[23307]{-1}[-1/-1] 2014-11-10 13:36:33.498113 i Service_Shutdown TREXIndexServer.cpp(02770) : Closing trace files

[23307]{-1}[-1/-1] 2014-11-10 13:36:33.498243 i Service_Shutdown TREXIndexServer.cpp(02774) : Stopping persistenceLayer

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.502969 i Service_Shutdown TrexService.cpp(05797) : Disabling signal handler

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.503059 i Service_Shutdown TrexService.cpp(05809) : Stopping self watchdog

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.505699 i Service_Shutdown TrexService.cpp(05821) : Stopping request dispatcher

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.505739 i Service_Shutdown TrexService.cpp(05828) : Stopping responder

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.522670 i Service_Shutdown TrexService.cpp(05835) : Stopping channel waiter

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.523456 i Service_Shutdown TrexService.cpp(05840) : Shutting service down

[22896]{-1}[-1/-1] 2014-11-10 13:36:33.974254 i Service_Shutdown TrexService.cpp(05845) : Stopping threads

---

Accepted Solutions (0)

Answers (1)

Answers (1)

lbreddemann
Active Contributor
0 Kudos

Dude... seriously?

Do you expect anyone to sift through all of this?

Really, you do have a support contract (hopefully) with SAP. Just open a support incident and have support tell you, what went wrong here.

Clearly, there's something wrong with the memory allocation.

And rather obviously it happens during the index rebuild phase during startup.

And, yes, again obviously there seems to be a problem with the rather small global allocation limit (GAL) on this box:

GLOBAL_ALLOCATION_LIMIT (GAL) = 47449144116b (44.19gb),

SHARED_MEMORY = 9937798000b (9.25gb),

CODE_SIZE = 7696564224b (7.16gb)

PID=22775 (hdbnameserver), PAL=47583361843, AB=2303234048, UA=0, U=1602010759, FSL=0

PID=22864 (hdbcompileserve), PAL=47583361843, AB=988254208, UA=0, U=366041204, FSL=0

PID=22861 (hdbpreprocessor), PAL=47583361843, AB=968224768, UA=0, U=299537305, FSL=0

PID=22896 (hdbindexserver), PAL=47583361843, AB=22152437760, UA=0, U=18015710283, FSL=0

PID=22902 (hdbxsengine), PAL=47583361843, AB=1745149952, UA=0, U=984524280, FSL=0

PID=22899 (hdbstatisticsse), PAL=5368709120, AB=1661300736, UA=0, U=920132971, FSL=0

Total allocated memory= 47452963696b (44.19gb)

Total used memory = 39822319026b (37.08gb)

Sum AB = 29818601472

Sum Used = 22187956802

But if you didn't play around with the GAL parameters and this worked before, there might be another issue that needs attention from, guess whom, correct, the friendly support .

Please, next time, either post what is actually usable or don't post.

90% of the characters in this post are actual garbage.

Cheers,

Lars

Former Member
0 Kudos

Yeah, I should have used the attachment option instead of posting in the message.

This is already been raised with SAP, however, the friendly support is usually very late . So thought if someone might help me in the network.

Thanks for your answer though!

Naga

lbreddemann
Active Contributor
0 Kudos

...and there we go again with the old forum hobby of bashing the support.