Best way to read data from SAP ECC.
Wanted to know the best possible method to pull data from ECC in below scenarios using DS 4.*
Volume of Data
Dirk Venken replied
The best possible method is not necessarily related to data volume but rather determined by functionality.
1/. When extracting from a single table, do that in a normal data flow. Note that there is a volume impact here: SAP installations have a time-out set for RFC calls (system-configurable typically 10, 30, 60 minutes); when data extraction takes longer than this limit, the DS job will fail. In that case, use an ABAP data flow.
2/. When extracting from a join of tables, also use an ABAP data flow. Although where-conditions are pushed down to the underlying database from a normal data flow, the joins are not, leading to abominable performance.
3/. If Business Content Extractors are being used in your environment, you can leverage them in DS for initial and incremental load. Use them in a normal data flow when used as such, in an ABAP data flow with additional where-conditions.
4/. You can use existing ECC functions returning tables as data flow sources, as well.
5/. And you can use the IDOC functionality if you're familiar with that technology.