sap bods full vs delta load
Can anybody help me with advantages and disadvantages of full and delta load and which one to choose in which case.
Venkata Ramana Paidi replied
Full load is the method every time truncating the target data and load the whole data.
If your source contains very less records and you no need to maintain the history then full load is the better.
The main disadvantage is performance. If we take transactional data , it may have millions of records . If we want to load full load , it takes lot of time . We cannot maintain the history of the source changes in the target as everything deleting the target data.
Delta Load :
Delta load we will use in the 2 scenarios .
1. Maintain the master data history .
2. Load the transaction data .
Maintain the master data history :
In the data warehousing if we want o maintain SCD2 ( slowly changing dimensions) , we will go with table comparison ,history preserving ,map operations and key generations . Slowly changing means changes occur once in while not frequently like address changes . As it is master data it contains less data only and changes are less . In this case we can compare the source and target tables and load the delta part and maintain the history.
This scenario will not workout for transactional delta as source having huge data and we will not going to maintain the history for transactional data.
Load the transaction data
If we want to load the delta of the transaction data , we will go with database level CDC . It works on log based or trigger based. Mostly data service with log base CDC techniques. Whenever changes occur in the transaction CDC will push the records into CDC database . We will read the data from CDC base and load into the target . In this scenario we will use map_cdc transform.
It is very helpful to save the loading time . Here some negative point is most of the databases creates CDC tables in the operation databases. Most of the client don't allow to create the tables in the production database. We need to go with Sybase replication server like that options.
I hope you understand the delta load and full load.
Thanks & Regards,