on 01-30-2013 7:42 PM
Hi,
what's the difference between the
load_balancing_func
and the
auto_merge_decision_func
if a Table is set to CREATE TABLE ... ENABLE AUTOMERGE or to ALTER TABLE...AUTO MERGE ?
According to
help.sap.com
the load_balancing_func checks the no of available tokens,
the system checks the number of requested / needed token.
Than, that could it be: no of available tokens >= requested / needed token and the (AUTO) MERGE can be triggerd.
What do we need an auto_merge_decision_func or memory_merge_descision_func or smart_merge_decsion_func for ?
Thank You !
Best Wishes
Martin
Hi Martin,
the auto_merge functionality and the decision functions are there, because performing a merge of a column table main store with the delta store to create a new, merged, read-optimized main store is a very costly operation.
It takes a lot of memory (eventually leading to the unload of other columns) and CPU cycles to do just that.
So, you don't want to do it every time you changed some data.
You don't want to do it just for the sake of having an 'optimal' read-optimized table.
What you want to do is, you want to perform the merge when all the effort actually pays off.
That is, when the read-access is so much quicker afterwards, that there really is a relative advantage in taking the costs of merging.
Now, what the criterium here?
And, assuming you have a criterium, what do you do when you got lots of columns that fulfill this? Perform the merge-jobs all at once?
The merge or no-merge decision is basically governed by the function you can but in auto_merge_decision.
smart_ - and memory_- merges are just different varieties of merges.
How many of the merges can be performed in parallel is decided via the load_balancing_function.
(obviously, ideal would be that HANA could know when there won't be a CPU/memory spike during the next n seconds and start the merge just then. This is, well, ideally... )
A word about the smart_merge: in contrast to the automerge (HANA checks automatically for the tables, that have automerge enabled, whether or not the merge conditions are fulfilled and eventually triggers the merge), the smart_merge allows for more control.
Imagine you load lots of data into your table:
HANAs automerge might decide will right during your data loading that now it's merging time and performs the merge.
Although this works fine, as the merge is lock-free, it uses resources without benefit, because you keep on loading data and the table will be as 'sub-optimal' as before the merge.
And this could happen several times during a data load.
As a better approach HANA allows for the users/application programmers to exclude tables from this automatic. For these tables, the application can indicate, when it may be a good point in time for the merge.
E.g. after the loading of data.
Now - beare with me a bit longer - imagine this application only really knows about that data was loaded but not how much compared to the current table content.
In that case, it might not be so clever to let the application actually pull the merge-trigger.
That's why there is a smart_merge_decision_function.
You can generally define conditions for when a merge of a table is reasonable and the application can tell HANA 'I think it's good to merge now'; but HANA will then check with the decision_function and either trigger the merge or leave it this time.
That's what we do in SAP BW on HANA with the highly managed data loading of DSOs and InfoCubes.
Ok, I tend to get into lecture mode at times... yet I do hope that you got the twist here and that your questions have been answered.
- Lars
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Lars,
in more general terms:
An Applications (or the Mergedog issues) a Merge request ( to the Mergedog).
The Merge Request is differtiated by mergetyps (AUTO, SMART, HARD,
MEMORY, CRITICAL, FORCED ).
The mergtype differentiation is dependend on Table-, MERGE- Statements
and Mergedog Settings.
The Load Balancing Function evaluates the available System Ressources
and how many Merge Requests and Merge Request Threads can be run in parallel.
The mergetypte ( AUTO, SMART, HARD ) specific Priority Function
assigns a Priority to the Merge Request (in a queue?).
The mergetypes (MEMORY, CRITICAL, FORCED) requets are executed at once,
withou of considerung the availablilty of system ressources.
The mergetypte ( AUTO, SMART, HARD ) specific Decision Function
evaluates (to TRUE / FALSE) if enough system ressources are availabe for the
specific request.
The mergetypes (MEMORY, CRITICAL, FORCED) requets are executed independetly
of available system ressources.
Thanky You.
Best Wishes
Martin
Table Locks seems to occure: HOw to ....
"With this double buffer concept, the table only needs to be
locked for a short duration the beginning when open transactions
are moved to delta2 and at the end of the process when the storages are “switched”.
"
User | Count |
---|---|
81 | |
24 | |
11 | |
9 | |
7 | |
5 | |
5 | |
5 | |
4 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.