SPS09 and smart data integration and smart data quality: a question

[former_member185332]

In the link What's new in SPS09 i read the following:

"Moreover, smart data integration and smart data quality in SAP HANA SPS09 enables the provisioning filtering, transformation, cleansing and enrichment of data from multiple sources into SAP HANA eliminating the need for a separate ETL or replication stages. Pre-built adapters are available for common data sources such as IBM DB2, Oracle, Microsoft SQL Server, OData, Hadoop and Twitter, and an open SDK is also available to build new adapters"

What are these pre-built adapters? We now use SAP DS to provide data to HANA. is DS become obsolete as from SPS09 as ETL tool? Are these "Pre-built adapters" usable through Hana Studio?

[werner_daehn · Accepted solution]

Data Services and the Smart Data Integration features sound like similar things as both are meant to move data, but the targeted use case are different.

First and foremost, the Hana solution requires Hana. Hence DataServices is required for all other customers that are not using Hana or where Hana is just yet another database. Yes, SAP makes a lot of noise with Hana but not every database user in the world had yet migrated to Hana, hence DataServices is and will be the biggest selling product and does get the deserved attention.

The second difference is Hana is all about realtime. Where DataServices can do realtime, its unique selling point is batch processing and ease of use. The Hana solution does support batch as well for the initial load, the primary focus was the question "When the source sends a change row, how to transform this row according to the initial load rules so the target data is correct". In DataServices you have to build an initial load, you have to find out how to identify changes and build dataflows that handle the various changes. The intention of the Hana solution is to do as much as possible automatically.

Because of the realtime focus the Hana solution it does transactional consistent loading. One of the design differences with Data Services. There many parallel sessions are used at the expense of transactional control.Here the transactional control was key.

And the final difference, Hana requires access to remote data in various ways, not only batch and realtime. It should support Smart data Access, Calcviews etc.

So only if you use DataServices exclusively to load into Hana, only then you should consider the Hana SP9 options. That is the only overlap.

Agreed?

[Former Member · Accepted solution]

Thanks Werner.

Would it be true to comment that for an organization who moves completely to HANA SPS10 for transactional and reporting solution , DS can be removed from landscape as SDI covers all the use cases in such organizations.

Right now we have put HANA platform SPS10 as our core component for ERP and reporting in our target state but diffcult to figure out DS place as we dont see lot of data movement once things are completely moved to HANA Platform. I m talking about 3-4 years target state strategy.

[werner_daehn · Accepted solution]

Yes, for such a scenario the answer would be an unqualified "Yes". Except that we have some catch up to do in Hana before this solution has the same feature richness as Data Services. Most of the problems stem from the fact that Hana is a platform. Take workflows for example. In DataServices Workflows are standard, would not be much of a problem to add this functionality to Hana as well. But instead we want to use the Business Process Modeling feature of Hana and not an isolated ETL-only solution. Not only for development cost reasons but also because then you can build ETL jobs like: start dataflow1 -> let user review the result -> when finished continue with dataflow2.

There are solutions even today, using above to start tasks or using SQL Script to assemble workflows, we just want to get better.

Hence my advice would be to get started on the Hana to gain experience. If you have Data Services already, do not exchange the technology right away. There will be a migration path from the DS engine to the Hana engine later anyhow.