on 09-17-2013 6:53 AM
Hi,
I need help in understanding how processing is distributed between various engines (Join Engine, OLAP Engine, Calculation Engine, SQL Engine). Lets assume that there are two Calculation Views (one graphical and one SQL based). Each having the same set of attribute and analytic views. The SQL based Calc View will have some SQL queries as well.
Pl help me in understanding how the processing will be distributed between (if there is any parallel processing) various engines in both these cases.
Also, in case of SQL based Calc Views, does the system process SQL statements in serial or there is some mechanism to parallelise the execution.
Besides, is there a mechanism to control this distribution.
Pl assist. Thanks.
Ok, this is again one of those "explain the world to me"-questions.
Obviously I cannot do this in a discussion thread.
But I can start with some rough explanations.
First of all: do your homework!
There is already plenty of material available here on SCN (e.g. ) or on Welcome | SAP HANA .
Also: read the documentation. As bad as it sounds, but really: do it.
There's a lot of knowledge in it that cannot be found elsewhere.
Please make sure to educate yourself based on what you find there.
Now, to answer some of your questions:
Also, in case of SQL based Calc Views, does the system process SQL statements in serial or there is some mechanism to parallelise the execution.
Besides, is there a mechanism to control this distribution.
Statements and commands in a calculation view actually are nodes in a graph of stacked calculations.
Whenever data needs to "flow" from one node into another, this would be an edge between these nodes. Basically this is a functional dependency then.
Now, if you think of how the data flows in your SQLscript then you might see that some of your statements actually don't have any direct dependencies and no data "flows" from the one statement/command into the other.
In that case it's possible to run these statements at the very same time and still get the correct result, right?
That's what the calculation engine automatically does for you.
This only works of course, if the statements don't change anything. If they do change data, we must obey the order as otherwise we could mess up the data consistency.
So, yes, for read-only procedures, SQLScript commands are executed in parallel.
The way to control this parallelism is to decide whether you want to have it - or not
You can specify that a procedure should be a read/write procedure (by omitting the READS SQL DATA specification).
Alternatively, you can explicitly specify that a specific BEGIN END block should be executed sequentially (BEGIN SEQUENTIAL EXECUTION ... END).
Obviously, you don't want to manually control the degree of parallelism in your code.
When you write it, you just don't have the right information at hand to make a proper decision on the right (=optimal) degree of parallelism.
With this I think I gave you something to think about.
We do have a lot of other places where we execute things in parallel.
E.g. if you select multiple columns from one table the data for the columns will be read in parallel.
Same is true for partitions of a partitioned table.
When you aggregate data, this can be done in parallel, just as joins could be executed in parallel.
So even if you see just one SQL statement in your code, it doesn't mean that it would run sequentially.
In fact, even the smallest operations in SAP HANA (the plan operators/POPs) implement parallelism at their very core code.
- Lars
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Lars,
I recently found a discussion which says that there are 2 basic engines in HANA ( OLAP engine is same as CE engine and JOIN engine is same as the SQL engine)
http://www.saphana.com/message/6508#6508
Most places i also see that there are 3 engines. Would you please let me know if the above information is correct ?
Thanks in advance.
Regards
Trinoy
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
88 | |
10 | |
10 | |
9 | |
7 | |
7 | |
6 | |
5 | |
4 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.