high memory consumption on BWFFACTableCompression operation

[nicholas_chang]

Hi All,

We've a BPC/BW production system where during lite optimize, BWFFacTableCompression operation will be executed and this operation normally took up a significant memory (>150GB) and eventually hit an OOM.

Please bear with me as i'm not a BPC/BW consultant but a basis/ hana technical guy.

I'm just curious why such operation will consume such a high memory as i can't find any relevant notes, correction, bugs for this. We are running revision 102.02 and so far our client don't have a plan to update hdb revision yet.

And the number of table (B28/F*) record to be compressed is >1bn records, table size around 20GB++

From the HANA memory usage, i can see heap memory Pool/itab is spike up to 60-70GB every time BWFFacTableCompression operation is running. From the OOM trace (exclusive), the connection ID itself consume up to 150GB memory.

Couldn't run explain plan and plan visualize due to the SQL Text is a procedure call - CALL BW_FACT_TABLE_COMPRESSION (?,?,?,?,?)

I know temporary memory is required for intermediate result, sorting, join and etc, but just want to confirm is it normal for such a statement to take up >150GB memory on a >1bn records table besides the possibility of memory leak bugs.

Appreciate if someone can shed some light on this.

Thanks,

Nicholas Chang

[nicholas_chang]

anyone? any idea? probably from BW/ BPC side?

Cheers,

Nicholas

[former_member183326]

Can you show me the Top 5 allocators In the OOM?

[nicholas_chang]

Hi

Thanks for your reply. You can have a for the top 3 allocators in compositelimit_oom trace. As you can see, we've set statement memory to 150GB, and statement with connection ID itself had consume all the 150GB.

Top allocators (component, name, size). Ordered descending by inclusive_size_in_use.

1: System: Connection/316905/Statement/1361100166262858 150gb (161065356124b)

2: System: Connection/316905/Statement/1361100166262858/IMPLICIT 150gb (161065339356b)

3: System: Connection/316905/Statement/1361100166262858/Pool/RowEngine/QueryExecution 16.25kb (16640b)

Top allocators (component, name, size). Ordered descending by exclusive_size_in_use.

1: System: Connection/316905/Statement/1361100166262858/IMPLICIT 150gb (161065339356b)

2: System: Connection/316905/Statement/1361100166262858/Pool/RowEngine/QueryExecution 16.25kb (16640b)

3: System: Connection/316905/Statement/1361100166262858 128b

During this time, cpu utilization and number of thread is high too. Please look a the screenshot below where during the time where BWFFacTableCompression is ran.

any clue?

Thanks,

Nicholas Chang

[former_member182967]

Hi Nicholas,

Not sure if note 2108247 - Running BPC reports leads to memory leak at HANA side helps or not, maybe you can have a look.

Regards,

Ning

[nicholas_chang]

Hi Ning Tong,

Yup, i've went thru the note and it is not relevant.

Thanks,

Nicholas Chang