Need to Process 2gb data Through PI

[former_member200386]

Dear Experts,

I am working on a scenario Proxy(BW) to JDBC when ever proxy program was executing

Where Input data is 2gb (2400000) records  will come to PI system. I Just want process 1000 records  at a time how can we do in PI end? this interface will run on Monthly basis.

i have one doubt i we  change the occurrence of operation mapping to 1.onbounded what will happen?

Thanks in advance,

Pavan

[baskar_gopalakrishnan2]

>I Just want process 1000 records  at a time how can we do in PI end?

Design logic at proxy coding side to push only limited records each time and have some sequencer at proxy side to trigger records at desired time interval. This way you would not load the jdbc receiver too much. Again inserting 1000 records using jdbc receiver adapter is not a big task.  You might also check whether db side tables are fully indexed.Mediate your messages many to complete 2gb records in smaller message size to get the performance.

[rajasekhar_reddy14]

Hi Pavan,

When you deal with this kind of interfaces remember below points

1)Splitting the pay load should be done at proxy level and never ever push 2 gb message from BW to PI because there are certain limiations processing hugr payload at one shot.

2)One you change logic at proxy level then use stored procedure design in PI-JDBC side to achieve the best performance.

Regards,

Raj

[former_member200386]

Thank you very much for your response Raja,

The problem we are facing here the data not able to reach the PI system it is got stuck in Queue only.

When i check in the smicm the execution time is set 10 mins for each message & 100 mb is max message size that we can process.

my ABAP programmer don't know how to spilt the payload at proxy level. i have one doubt what will happen if change the message interface occrrecnce 1.unbounded?