Value Mapping performance with large data sets

[Former Member]

I've read some good stuff on SDN about how various people have implemented lookups in PI

- /people/jin.shin/blog/2008/02/15/sap-pi-71-mapping-enhancements-series-graphical-support-for-jdbc-and-rfc-lookups

- /people/alessandro.guarneri/blog/2006/03/27/sap-xi-lookup-api-the-killer

I have a couple of questions about the "Value Mapping" function in PI and I wondered if anyone knows this, or could point me to a SAP Note / SDN blog/wiki?

I need to know the behaviour of the system when we have large Value Mapping tables, specifically:

- is the ValueMapping an SQL table or a Java Hash Map? I.e. is there any performance concerns if we do lookup against a large Value Mapping table

- is the whole ValueMapping cached in RAM?

- if we update a record in the ValueMapping from ECC to PI via standard interface - as documented on SDN - what happens to the cache?

- if we have 120k items in the ValueMapping and we have +-700 real-time updates per day, does this mean we have 700 full cache refreshes?

If you have experience of hitting limits on ValueMappings could you please share? E.g. after the ValueMapping reached nnn items performance changed...

If you have any details on how this function works behind the scenes it would really be appreciated.

[Former Member]

bump...

Anyone?

[Former Member]

Derek,

I have the same question.

So as you are already one step ahead of me - you implemented the replication already - proceed with your good work and do a test run with 100.000 entries - than you will know !

If not - wait some days and I will deliver the answer. But actually I would prefere you do it

So please help yourself and me

Andreas

[Former Member]

Andreas,

Thanks for the reply. At this time we just don't have the people/tools available - much of the project resources are out-sourced and they are not contracted to start yet... (i.e. performance test tools are not available onsite)

I would like to collaborate so please do share any info you get from your efforts. As for my side, the client has decided the risk to the project is too high, and we will change our design.

We are going to keep small mapping tables in the the ValueMapping, and move large tables to SQL tables in PI ABAP. As we are only on PI 7.00 (without EhP1) we will have to create UDF to do the lookup in the mapping... but that's another story for another day.

[Former Member]

hello Derek,

Let me share my experience with value mapping.

In my project , we have implemented the Mass Value Mapping Replication Scenario, almost 2 yrs back.

Still it's running fine with minor issues..

We maintain the records in ECC Z table ( more than 10000 record at present)

whenever , any new addition/deletion in the Z table , we execute the standard value mapping replication interface and corresponding vlaues get updated in Runtime cache.

we could not find any performance issue with till now, but yes faced the below issues.

1) if duplicate record exists in Runtime cache , PI will send source value ( in PI 7.1 now u can error out or send defalut )

solution - wrote a custom java code which will fetch only single target value.

2) if you have multiple server node , sometimes the the VM program misses the cache...

solution: after applying SAP Note ( forgot the number) problem solved.

But we havn't found any performance issue till now..

let me know if you need more info....

regards

biplab

[Former Member]

Hi Derek,

If you are having PI 7.1 then you can maintain all the values in an external table and can send all the values as an array with one shot which may increase your performance to a little extent.

Else you can try value mapping replication. Check this help and blog on this:

/people/sreekanth.babu2/blog/2005/02/23/value-mapping-replication

http://help.sap.com/saphelp_nw04s/helpdata/en/2a/9d2891cc976549a9ad9f81e9b8db25/content.htm

Regards,

---Saish

[Former Member]

Hi Satish and thanks for the reply.

We have already implemented the replication using the patterns in your message. My question is more focused on the performance of the data once it is in PI i.e.:

- are there known limits to the performance of the lookup

- what is the affect of a single item refresh

Rgds,

Derek