Syndication of MDM records to Process Integration

[Muniyappan]

Hi,

I am a PI Consultant. i am integrating MDM 7.1 and WMB (WebSphere message broker) using PI 7.1.

The flow will be MDM to WMB.

MDM has table called City which has 7000 records.

Earlier MDM team has set the port manual and they have imported these many records.

After importing they set the port automatic. it started to syndicate the records.

For 7000 records we are getting 200000 messages in our PI System.

Can anyone tell me how we are getting this much messages for 7000 records?

Regards,

Muniyappan

[Former Member · Accepted solution]

Muniyappan,

As per my understanding, you are getting such amount of messages because of missing remote key mapping in syndication map. Could you let me know the following things:

1. How many remote system you have?
2. Did you map Source Remote Key field to destination field in syndication map?

Thx/ -Tarun

[Muniyappan · Accepted solution]

Hi Tarun,

1. i have two remote systems.
2. Source Remote Key field is mapped to destination field in the syndication map.

Regards,

Muniyappan.

[Former Member · Accepted solution]

Hi Muniyappan,

Possibly what is happening is records are getting changed many times in mdm and so many copies are sent out to PI.

Please maintain "Supress Unchanged record- SET" and "Supress records without Key - SET" in syndication map.This will make sure only changed records with valid keys for Remote system get syndicated.

It will be a gud idea to look into the changes happening in MDM,these appear to be mass changes as 7000 to 200000 message ratio is very high.

Also,you can ask MDM team to put Filter in the syndication map so that only required data gets synbdicated to the port.

Thanks,

Ravi

[Muniyappan · Accepted solution]

Hi Ravi,

Thanks for your reply.

MDM team put the syndication mode manual and imported the data many times.because of that 2 lakhs  of messages were present in the Ready folder.when they put the mode to automatic PI started to pick those 2 lahks.

sure i definitely ask mdm team to maintain "Supress Unchanged record- SET" and "Supress records without Key - SET" in syndication map.

can you please tell me what will happen when i do not give connection between MDM and PI and MDM syndicates the records?

i mean the records will be in ready folder for a long time otherwise records will be moved to Archive folder after some time.

Regards,

Muniyappan

[Former Member · Accepted solution]

Hi Muniyappan,

Kindly ensure you have a Filter set in syndication map.

This filter is defined by business use for example records created with Flag set as TRUE should be send to PI this can be set as Filter in the map.Thereby sending only valid data to PI.

If PI channels are closed the files will be sitting in Ready folder,and only if they are picked by PI the files move to Archive folder.This setting is governed by MDM Adapter in PI.

Untill the filters are imposed it will be worthwhile to either close the channels or set port to manual to avoid such a scenario.

Thanks,

Ravi

[Former Member · Accepted solution]

Hello Muniyappan,

Suppress unchanged records option controls the behaviour of the syndicator while syndicating the Records to the remote system.

Firstly let me explain you how this property works

Let us say you have Suppress Unchanged Records option set to Yes, then once a record is syndicated, the syndicator adds a Time Stamp to the remote system of the Map (This time stamp contains the Date and Time along with the field values of the record at the time or record syndication), and when the syndication is again triggered, the record's field values are compared with the already syndicated record's field values using the Time Stamp, this checks if any of the field value of the record has changed since last syndication, if all the Fields are the same, then the record will not be syndicated, and if any of the field value has been changed from the last syndication, then the complete record will be syndicated and the syndicator will again create a new time stamp for future reference .

Now let us say you have NOT set the suppress unchanged records option to yes, and the Port has been set to be automatic, then each time when the port triggers the syndication, the records will be syndicated to the Port's ready folder, this will create multiple files consisting of the same record.

Now, if you donot have a connection between MDM and PI, the syndicator will keep syndicating the records to the port's ready folder and will remain their since their is no PI to pick the files, and as the files on the port's ready folder increases (due to multiple syndication), it consumes memory, the system performance is affected and system hangs. Recommendation would be to set the Port as Manual until the MDM PI configuration is completed

And if you have a connection between MDM and PI, Then the PI will scan the port's ready folder periodically and pick up the files for processing if the ready folder has any files, if the file is successfully processed by PI, it will be moved to the archive folder, and if the file has any error, it will be moved to the Exception folder.

Hope this helps.

Rgds,

Prasad