Load all HANA data to HADOOP

[patrickbachmann]

Hi folks,

My HADOOP team is looking for a way to replicate all ECC data into HADOOP system using HANA as the middle man.  I've been doing some research and mostly I'm finding SCN posts on doing the opposite;  consuming aggregated or reduced HADOOP data into HANA.  To me that's how HADOOP should be leveraged but I'm trying to see what others are doing.  My questions;

1) Is anybody doing this (loading all HANA data into HADOOP on daily basis)?  Is this really best practice and is there any good documentation on this approach?

2) Does anybody know much about HANA Dynamic Tiering?  I was asked to look into this and as far as I can tell it seems to be intended for BW scale out system.  Our HANA system that contains SAP replicated data is a single node scale up system so I do not think this can be leveraged. 

3) I'm not clear on how delta mechanism would work.  Is it possible to somehow use log files in HANA and pull inserts/updates/deletes this way?

Thanks,

-Patrick

[Former Member · Accepted solution]

Hi Patrick,

1) Your scenario  from Hadoop to HANA   is interesting . I have seen until now all opposite directions now. Can you please share the business use case for the same so that we can think of  something else.

I  think even  SAP HANA Vora focus on the approach of moving on data from Hadoop to Hana and  not vice versa.

2) on HANA dynamic tiering , did you got a chance to look at below link:

https://hcp.sap.com/content/dam/website/saphana/en_us/Technology%20Documents/SPS09/SAP%20HANA%20SPS%...

Regards,

Rajesh

[patrickbachmann · Accepted solution]

Thanks for this great link Rajesh, I had not yet seen this one.  My interpretation of this is that it's only available as of SPS9 is that correct?  As far as use cases, nobody has really given me a good use case exactly yet as I have asked the same question.  Even reading about Dynamic Tiering I can see how it can save memory but I do not see how it's any huge benefit to the HADOOP team in getting their daily extracts into HADOOP from HANA.

-Patrick

[Former Member · Accepted solution]

Hello All,

I am starting on a same task as mentioned above. Replicating/Moving data from SAP HANA to Hadoop.
I have short listed below ways. Kindly suggest or correct me if i am wrong.

1. SAP Data Services 4.2

2. SAP Replication Server

3. Scoop

4. Data Life-cycle Manager

Kindly suggest me which of the above listed method will be the most reliable, easy to deploy.
I am also open for new suggestions.

Regards,

Shekhar

[otto_frost4 · Accepted solution]

Hi,
I have the same issue - Replicating/Moving data from SAP HANA to Hadoop.
What solution did you end up with?

I found there is a BADI in the "sap replication and transformation server" that maybe can be used to replicate.

Hadoop WebHDFS requires SPNEGO Kerberos authentication which complicates things.