Help with JDBC Receiver performance issues with large number of messages...

[Former Member]

I have an XI scenario where SAP sends MATMAS (Materials) IDoc's to XI. XI then uses XSLT to map to a JDBC format to insert each Material into extrernal database (SQL Server) tables corresponding to each of the IDoc segments.

So for each material IDoc it will create 1 record in a table for mara, a couple for marc and mard and a couple for mbew, etc.

We want to run this interface every now and then with about 200,000 materials out of SAP.

The problem is that it takes about 24 hours to process just 24,000 materials and seems to run very slowly after that (1 Material every few seconds). Extrapolating this out means way over a week of processing ;-o

I don't think the performance issue is on the external SQL Server db because if we perform an upload to thge DB with the same data from files it is blindingly quick!

The performance bottleneck seems to be between the outbound queues on XI and the jdbc adapter. The SAP IDoc processing and the XSLT mappings occur quite quickly.

I have turned on PACKET processing in XI for the inbound and outbound EO queue's but it has made no difference as the XI IEngine processing is quick - its the jdbc adapter that is slow.

An issue that I can see is that there is a separate jdbc message going to the adapter for each Material, but I can't see how to batch them together (I thought the packet processing would do that?)

When you send IDoc's to XI in packets, the XI IDoc sender-adapter splits them up into individual messages which is a pain.

Do you think it would be wise to use a ccBPM to COLLECT all the incoming messages and send them to the jdbc adapter in batches of about 5MB or so..? I'm not sure if ccBPM can handle it though?

Any thoughts will be most appreciated.

Regards,

Jason.

[Former Member · Accepted solution]

HI Jason

Looks like performance hit is the requirement where your mapping is creating record by record. if you try to load the data normally it has not shown that right.

Collection of messages in BPM does help but BPM usage itself can hit performance again so take that as last option.

I think making changes to the mapping to execute 200,000 in a batch can solve this performance.

Share the mapping and how it is getting triggered to understand the performance issue.

Thanks

Gaurav

[Former Member · Accepted solution]

Thanks for the reply Gaurav...

The issue is that SAP is th sender system and it sends MATMAS Idocs to XI. Even if these are packed into 500 Idocs per package , the XI Sender Idoc adapter splits them into separat individual xi messages. Therefore the mapping takes place per message and not for the packet of Idocs.

Note however I have turned on packet processing in the XI inbound and outbound queue for this interface:

http://help.sap.com/saphelp_nw2004s/helpdata/en/ab/bdb13b00ae793be10000000a11402f/frameset.htm

I'm thinking the only available option si to use a ccBPM to group the xi messages together into larger packets to send to the jdbc adapter.

[Former Member · Accepted solution]

HI Jason

Did you already checked with these

Thanks

Gaurav

[Former Member · Accepted solution]

I think using enhancement pack 1 will be the way to go and we can retest with high volumes then...

Thanks.