cancel
Showing results for 
Search instead for 
Did you mean: 

Handling Large files in PI scenarios?

Former Member
0 Kudos

Hello,

We have lot of scenarios (almost 50) where we deal with file interfaces atleast in receiver or sender side. Some of them are just file transfers where we use AAE and some are where we have to do message mapping (sometimes very complex ones).

the interfaces work perfectly fine will a normal file which dont have much records but recently we started testing big files with over 1000 records and its taking a lot of time to process. It is also causing other messages which gets lined up in the same queue to wait in the queue for the amount of time it takes for the first message to process.

This must be a very practical scenario where PI has to process large files specially files coming from banks. What is the best way to handle its processing? Apart from having a better system hardware (we are currently in the test environment. Production environment will definetely be better) is there any technique which might help us improve the processing of large files without data loss and without interrupting other message?

Thanks,

Yash

Accepted Solutions (0)

Answers (1)

Answers (1)

former_member181962
Active Contributor
0 Kudos

This discussion might give you some direction:

Regards,

Ravi kanth Talagana

Former Member
0 Kudos

How does RECORDSET PER MESSAGE works? Suppose i have a file which has a header, 1000 line items and a trailer and then how will PI split the file?

Will it work in this case as in my message mapping we have certain validations on the header and footer depending upon the line item?

Yash

Former Member
0 Kudos

Hi Yash,

In your case you need to have a key field for header, detail and trailer.

Regards,

---Satish

Former Member
0 Kudos

How will it exactly work and configured? Can you please in details or point me to the right blog?

Former Member
0 Kudos

Hi Yash,

Check this blogs for the strcuture you are mentioning:

/people/shabarish.vijayakumar/blog/2006/02/27/content-conversion-the-key-field-problem

/people/shabarish.vijayakumar/blog/2005/08/17/nab-the-tab-file-adapter

Regards,

---Satish