on 05-16-2014 3:15 PM
Hello Experts,
Have anyone worked on using a parallel processing profile in a data source.
We are extracting the data from planning area for backup and extracting to BW for reporting purposes. To speed up the process, we have an option to implement parallel processing for the datasource related to the planning area.
Does this really work without any discrepancies in the data extracted?
Please provide some pointer on the concept.
Thanks,
Rag
Hi Raj,
There will be no discrepancy in data quality as parallel processing profile is meant to extract data packages in parallel. Also there is another setting (image) which can be used to speed up the process and also reduce the number of Data packets. In our project we save more than 40% of process chain run time
Regards
JB
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi JB,
Appreciate your response. 2 questions from regarding this.
1. Could you please help me with the PP profile parameters used in your project which could save runtime. I tried as below.
No of parallel processes: 2
Block Size: 1000 to 5000 (testing different options)
2. When I try with above parameters in RSA3 extraction, I could not get the same number of records that are being extracted with PP profile and without PP profile.
Ideally in both cases RSA3 should be extracting same number of records in test mode right.
Please advise.
Thanks,
Raj
Hi JB,
Appreciate your help on below 2 questions.
1. Could you please help me with the PP profile parameters used in your project which could save runtime. I tried as below.
No of parallel processes: 2
Block Size: 1000 to 5000 (testing different options)
2. When I try with above parameters in RSA3 extraction, I could not get the same number of records that are being extracted with PP profile and without PP profile.
Ideally in both cases RSA3 should be extracting same number of records in test mode right.
Please advise.
Thanks,
Raj
Hi Raj,
In our project we are using 8 parallel processes with Block size 1000.
Regarding question 2: Ideally, yes both the iterations should result in same number of records, but data packets will be different based on Block Size. Please check whether you are referring to data packets or records.
Also if you have the check box marked in one of the iterations, the number of records may vary.
Regards,
JB
Hi JB,
Thanks for your reply.
I have tested executing an infopackage with given parameters of the PP profile. It completes much faster and the number of records are also matching (with and without PP profile).
But I am trying the extractor (RSA3 or test datasource), which is not showing me the exact number of records in both cases. Not sure if I can ignore this as long as my infopackage is able to extract same number of records.
Appreciate your inputs.
Thanks
Raj
User | Count |
---|---|
9 | |
4 | |
3 | |
3 | |
2 | |
2 | |
1 | |
1 | |
1 | |
1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.