File or JDBC when dealing with large amount of data

[Former Member]

Hi Experts,

We planning to integrate the third party system, we dealing with very large amount of data, what is the best option to integrate the third party using FILE or JDBC with stored procedure. They can do both options. please let me know what is best way to integrate in PI for architectural point of you and performance point of view.

Regards,

Rachana.

[naveen_chichili · Accepted solution]

Hi Rachana,

If you are trying to handle large amount of data you can go with FIlE adapter so that you can have an option to split the data in case of very large file.

Also JDBC adapter might have dependencies like locks, thread issues e.t.c

Thanks and Regards,

Naveen

[iaki_vila]

Hi Rachana,

With File adapter you can omit ESR development and the mapping if this is not mandatory, then you can improve the deficiency in PI.

On the other i've scenarios with a jdbc diary load about 5 - 10 Gb, the main problem is to have enough space in the PI tablespaces and the messages with more than 100-150 Mb in one message PI can throw exceptions if you don't have the SMICM parameters about the message size supporting this size.

Pay attention to your PI trace level and to have the jobs of Archive and Deletion running.

From my point of view, the file adapter is the best solution to move huge quantity of information between systems.

Regards.

[Former Member]

Hi  Inaki Vila,

If we send the real time data like maximum of 5MB, then which one we can prefer FILE or JDBC?

Thanks for your help.

Rachana.

[Former Member]

I would suggest to go for File adapter processing time will be better compare to JDBC. if you use JBDC in case of any data issue for single record whole message will be roll back.