IDoc to CSV on multiple folders based on field in IDoc

[Former Member]

HI Experts,

My scenario is IDoc to CSV and vice versa CSV to IDoc. Let us say IDoc to CSV as Outbound and CSV to IDoc is Inbound here. Please find the below requirements interface wise.

Outbound scenario: IDoc --> CSV

All the end users who has access to SAP system would be entering the ORDERS01 IDocs manually from different mines, a background job will be executed to fetch all the IDocs in a day and would be triggered to SAP PI 7.31 single stack. Based on the value of WERKS field in IDoc received all the IDocs relating to one mine [or same WERKS value] should be transformed as CSV file with each IDoc data as a single line item and should be posted to the corresponding folders that are created based on the name of the mine on receiver file system. I have done IDoc packaging on the SAP system side and settings on the channel to receive multiple IDocs at a time, FCC receiver and corresponding changes accordingly and the scenario is working like multiple IDocs to single folder only for example if 50 ORDERS01 IDocs are triggered from SAP then a CSV file with 50 line items is getting created and posting to a single folder.Now the requirement is in addition or modification to the above, suppose 70 ORDERS01 IDocs with different values are triggered from with different value to WERKS field and in PI those multiple IDocs with same WERKS value should be categorized as a group and should be transformed to one CSV file with each IDoc as single line item. Can you please suggest some standard ways like variable substitution to achieve this.

Inound scenario: CSV --> IDoc

The requirement is like the end users of file system will place the CSV files on specific Inbound different folders and PI system will poll them and all of them should be triggered as DELVRY07 to SAP R3 from SAP PO and currently the settings in such the interface is working like from single folder a file should be picked and based on the number of line items that many number of IDocs should be triggered to SAP R3.

Kindly suggest standard ways to achieve them.

Thanks,

Nithin.

[justin_santhanam · Accepted solution]

Hi Nithin,

The 10 different folders corresponds to 10 different WERKS?

- Let say today if you write the data into the folder, will it be archived/deleted before the next day job rans?

I think you can use Dynamic parameters to create the folder name (based on WERKS) and the file name with the Date , and create the file in append mode.

So, for example for a given day the below IDOCS get generated:

10 IDOCS for WERKS 1010

20 IDOCS for WERKS 2030

30 IDOCS for WERKS 3010

10 IDOCS for WERKS 3011

in the mapping program create the file name for WERKS 1010 as \\sharedfolder\\1010_Folder\1010_20151229.TXT

in the mapping program create the file name for WERKS 2030 as \\sharedfolder\\2030_Folder\2030_20151229.TXT

in the mapping program create the file name for WERKS 3010 as \\sharedfolder\\3010_Folder\3010_20151229.TXT

in the mapping program create the file name for WERKS 3011 as \\sharedfolder\\3011_Folder\3011_20151229.TXT

In the communication channel have the write mode as "append"

So for the given day the files for the corresponding mines gets accumulated.

Let me know if I missed any of the requirement or the solution won't meet ur needs.

Thanks!

[maheswarareddykonda · Accepted solution]

Hi Justin,

i am just curious to know about that solution, as per my understanding Nitin getting multiple Idocs with multiple WERKS(1010,2030,3010.....) Idocs

if that so, here challenge is , we need to group all Idocs depending WERK field and send to different folder using single channel.(even multiple channel also some what tricky)

if we write the UDF for dynamic folders , i guess that wont work since here multiple Idocs are cumming with different WERKS.

@nitin,

why dont you check with ABAPer to handle batch job depending on WERKS fields should be trigger from ECC, so that here you will get same type of WERKS IDOCs and you can place them in folder depending UDF.

[justin_santhanam · Accepted solution]

Hi Mahes,

I think I missed that part, which wasn't pretty clear to me. Anyways it's not a still big deal.

Scenario 1 - Get the IDOC's and split them in WERKS (multi mapping) and loop it back to PI  (  Thanks to Eng Swee Yeoh )

Scenario 2 - Set the File/Folder name in the dynamic config and proceed with the approach I have given in the above thread.

Thanks!

[Former Member]

Hi Praveen,

Thank u for your reply !

@Praveen : I tried that way earlier also  , I meam given parameters in FCC as:

var1	payload:MT_Sendtest,1,Record,1,Name,1

But I face error as below:

Transmitting the message to endpoint <local> using connection AFW failed, due to: com.sap.engine.interfaces.messaging.api.exception.MessagingException: com.sap.aii.adapter.file.configuration.DynamicConfigurationException: The Adapter Message Property Directory was configured as mandatory element, but was not supplied in the XI Message header.

Thanks,

Nithin.

[former_member182412]

Hi Nithin,

You need to disable adapter specific message attributes in your receiver file adapter because we are not creating the directory using dynamic configuration, we are using variable substitution.

Regards,

Praveen.

[Former Member]

Hi Praveen,

I am using Dynamic configuration to create folder name and disabled variable substitution but now the error is different:

Transmitting the message to endpoint <local> using connection AFW failed, due to: com.sap.engine.interfaces.messaging.api.exception.MessagingException: java.lang.Exception: Exception in XML Parser (format problem?):java.lang.Exception: Message processing failed in XML parser: java.lang.Exception: Consistency error: more fields found in XML structure than specified in conversion parameters! (Value abc), probably configuration error in file adapter (XML parser error).

Thanks,

Nithin.

[former_member182412]

Hi Nithin,

If you do multi mapping you cannot use dynamic configuration, you must use variable substitution and make sure the fields mentioned in content conversion and the fields in payload after the mapping are equal.

Regards,

Praveen.

[Former Member]

Hi Praveen,

I followed the same steps but in my case WERKS should populate to a field in target structure.Target structure in snippet shown below:

Error snippet:

Thanks,

Nithin.

[Former Member]

Hi Nithin

You can split the files based on WERK using graphical mapping.Only add a additional node in your target structure to hold the file name and folder name as mentioned in Praveen's blog.

I am proving you sample mapping. Hope this helps

Input XML:

expected output:

mapping:

same logic for subject and address.

[former_member182412]

Hi Nithin,

Your variable substitution config is wrong. You must give like this.

var1     : payload:MT_Sendtest,1,Record,1,Name,1

Regards,

Praveen.

[Former Member]

Hi Indrajit,

Thank you for your inputs!

But ID[sample mapping]  ~ WERKS [my requirement] should appear as line item and also hope u understand my receiver structure in CSV with field names under line item should appear as column names in CSV and also that names should appear only once.

Thanks,

Nithin.

[Former Member]

Hi Praveen,

My input structure is having below fields as inputs with the below XPaths:

/Zyyyy/IDOC/E1EDS01/ZORDRS/VBELN

/Zyyyy/IDOC/E1EDKA1/PARVW

/Zyyyy/IDOC/E1EDP01/E1EDPA1/PARVW

/Zyyyy/IDOC/E1EDS01/ZORDRS/BSTKD

/Zyyyy/IDOC/E1EDP01/WERKS

/Zyyyy/IDOC/E1EDP01/MATNR

/Zyyyy/IDOC/E1EDS01/ZORDRS/VRKME

/Zyyyy/IDOC/E1EDS01/ZORDRS/KBETR

May I know why did you sort MENGE, BELNR by key WERKS?

Will the same mapping logic works in my case also? I hope you should have used SplitByValue type as Value change?

Thanks,

Nithin.

[former_member182412]

Hi Nithin,

We need to group the IDocs based on plant that is why i sort by WERKS, your case also you need to sort by WERKS. But make sure the contexts and the mapping the concept is same the mapping may be small difference depends on your structure.

Regards,

Praveen.

[Former Member]

Hi Justin,

Bu using “append” option in receiver channel it can be appended as you suggested but even column names are getting appended which I do not require. I need only control data of IDoc needs to be appended as line item to receiver CSV file.

By the way you suggested even column names getting appended every time, for example Name, Organisation, City and Designation column names are even appending:

Whereas I require like in snippet shown below:

Kindly help me to achieve the above as shown in second screen shot.

Thanks,

Nithin.

[Former Member]

Hi Experts,

Can you pls help me to solve the issue.

Thanks!

[anupam_ghosh2]

Hi Nithin,

                  Use an adapter module in  receiver communication channel.

This custom adapter module will check the target directory before any file is appended. It will know which file name to check and the directory where to check. In case the file is existing it will delete the header line from the payload else will write the contents as it is.

Regards

Anupam

[yeeloon-khoo]

Hi Nithin,

Alternative way is use use Run OS command after message processing.

Every time line get appended is without header line, then run OS command every time to check if no header line, add it at the first line.

Regards,

Yee Loon

[former_member182412]

Hi Nithin,

You can achieve this by doing multi mapping as i suggested before.If you receive multiple IDocs in a single transaction from ERP you can do the mapping like below.

Mapping for StockOrder and OrderHeader like below, WERKS field context should be ORDERS05

Fields related to E1EDK01 normally header fields the mapping like below.

Field related to E1EDP01 normally item fields the mapping like below.

Output payload after mapping, For 3204 plant there are two records.

FCC:

Variable Substitution:

I took the same test payload in the below blog.

I am showing one of the output file that contain two IDocs for same plant which is 3204.

Regards,

Praveen.

[yeeloon-khoo]

Hi Nithin,

I support Justin's solution.

I'm helping to clear the the confusion, based on your questions using Justin's solution:

"How to achieve it with out multi mapping?"

Reply: Set to no Idoc packaging, mean set package size to 1 in partner profile. This mean idoc send one by one to PI. For each message processing, since is only one idoc one WERKS, one ASMA Folder + Filename, then no need 1:N multi mapping, no need splitting. This is stateless as well, no need BPM.

"how to merge the records in MM based on WERKS"

Reply: The merging is happen when each line get appended into CSV file with same name, after receiver file adapter created the file, based on ASMA Folder + Filename you set in MM.

"..and push the CSV with n number of line items where n number of IDocs of same WERKS value."

Reply: CSV file is get appended line by line until n-time with same WEKRS. It is not push single CSV file with n lines.

Check example posted by Justin earlier, thing should be cleared.

Regards,

Yee Loon

[Former Member]

Hi Justin,

How to achieve it with out multi mapping?

Thanks,Nithin.

[justin_santhanam]

Nithin,

There is only one Outbound and one Inbound Interface here. Correct?

Thnx

[Former Member]

Hi Justin,

Yes only one Outbound & Inbound interfaces here.

Thanks,

Nithin.

[Former Member]

Hi Raghu,

Will the grouping of IDocs as per WERKS value can also be done and form a CSV with multiple line items and post to a corresponding folder?

Thanks,Nithin.

[former_member186851]

Hello Nithin,

Try using Sort functions to get all the Werks under one context and then try creating target nodes based on change in value.

[anupam_ghosh2]

Hi Nithin,

                 Here are the steps you need to follow here

idoc---->csv

1. Collected idocs reach PI server.

2. There is a multi mapping program with 5 different message types in receiver end (inbound). The data type of all these message types will be same. Since we need to do multi-mapping we are having different message types.  This will help you later to recognise which values in file were wrong from message failures. there will be one outbound interface.

3. The data type for outbound interface  remains constant depending on idoc structure.

4. In multimapping use if -else condition to check "WERKS" create the target messages typ1,typ2....typ5.

5. In ICO in ID create only one  receiver component and five receiver communication channel one for each inbound message type. The different inbound components will automatically show up in receiver interfaces.

reference

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/90dcc6f4-0829-2d10-b0b2-c892473f1...

Csv---- Idoc

-----------------

This scenario is pretty simple compared to first. Here in mapping step create one idoc for each record in file and send to target destination.

Regards

Anupam

Message was edited by: Anupam Ghosh

[Former Member]

Hi Anupam,

I have question here regarding:

"2. There is a multi mapping program with 5 different message types in receiver end (inbound) "

On what basis have you decided number 5, I have 11 different folder so I need to create 11 different Message type right according to you?

Thanks,

Nithin.

[former_member182412]

Hi Nithin,

You can do this with single interface, using variable substitution you can do the dynamic folder, check below my blog regarding this. I did not merge the records based on same plant but you can do this in the mapping.

Regards,

Praveen.

[Former Member]

Hi Praveen,

Nice blog and my requirement is to determine the folder name based on first four characters of WERKS field so I have to choose UDF by ASMA, hope the above is possible even by ASMA too.

Also pls suggest how to merge the records in MM based on WERKS and push the CSV with n number of line items where n number of IDocs of same WERKS value.

Thanks,

Nithin.

[anupam_ghosh2]

Yes Nithin you need 11 different channels one for each folder. By mistake I mentioned 5 instead of 11.

[anupam_ghosh2]

Hi Nithin,

                   Your scenario should not use ASMA. If you are trying to determine directory name based on ASMA, in mapping runtime the last run will determine  the final directory.

Regards

Anupam

[justin_santhanam]

Anupam,

Do you think it's feasible to create 11 different channels for this? In future if they have added a new plant let say, we need to create another new channel. I don't think that's an optimal solution in my opinion.

Thanks!

[justin_santhanam]

Hi Nithin,

>>Also pls suggest how to merge the records in MM based on WERKS

Please let me know what I have been missing here. Why you need to merge the records in Mapping? You will be getting only one IDOC per transaction correct, as per the proposed solution?

Thanks!

[anupam_ghosh2]

Hi Justin,

                  In case you are thinking of ASMA, then we cannot apply here. In multi-mapping we are grouping the records and say we are assigning a directory name. The final run will decide which directory name PI finally assigns.

We are not adding sender communication channel which would poll and cause PI to slow down. We are adding receiver channels only. 

Regards

Anupam

[justin_santhanam]

I completely understand it's not sender channel ...I'm talking about the maintenance here, if the functional team adding new plant, then you have to be involved by adding a new channel here. Can you just promote your changes to Prod? Don't you think again testing needs to be involved, sign off etc... This is what I'm trying to avoid.

If you have followed the thread , there is no need for multi mapping here. As we are planning to have single IDOC per transaction. Again I don't know what's Nithin plan here is. I think we all are pulling him into multiple directions. As per the solution I have proposed no  need for multi mapping. But as per your solution we need multi mapping. So there was quite a confusion here ..

Thanks!

[anupam_ghosh2]

Well I suggested the solution keeping in mind that bundling of segments on basis of werks field is taking place in PI.

Regards

Anupam

[Former Member]

Hi Raghu

Thanks for reply! The receiver structure is same with all the fields but the requirement is as explained, kindly share some thoughts on the same.

Thnx,Nithin.

[former_member186851]

Hello Nithin,

As I told earlier

1.IDocs collected will reach PI.

2.Having one Interface with multiple operations and in the Interface determination split IDOCS to individual operations(Based on werks).

3.For Each plant configure one File channel to place the file.

Let me know if u have any face issues on this.

[Former Member]

Hi Raghu,

How to split multiple IDocs based on a field in ICO?

Thanks,

Nithin.

[former_member186851]

using conditions in Interface/Receiver determination Nithin.

[Former Member]

Hi Justin,

Thank you for your response!

Please take a look at the below blog and can you confirm whether you are receiving n number of IDocs in single transaction?

- Yes, the IDoc packaging is done for PI to receive n number of IDocs at a time.

Is it possible to change the collect IDoc packet size to 1 so that you receive only 1 IDoc and still achieve the same initial solution I proposed?

- Yes I have the access to ECC system and can change the partner profile, but I need to achieve the requirement as above explained like receiver is one physical system but files to be posted to already defined folders like ERP/SAP/DO/aaa, ERP/SAP/DO/bbb and so on where WERKS field occuring once per one IDoc [other WERKS field in ORDERS05 IDoc is ignored] and also WERKS will contains values like aaa01. aaa02, aaa03, ..etc so all IDocs with value containing aaa* should go to the folder ERP/SAP/DO/aaa and similarly if WERKS field contain bbb01, bbb02, ... etc and all the IDocs or single IDoc will transform to CSV and should go to ERP/SAP/DO/bbb folder.

Thanks,Nithin.

[justin_santhanam]

That's correct Nithin. You are having only one communication channel. As I mentioned before using dynamic configuration  determine where the file should land based on WERKS you create the folder/file name. Have u had a chance to look at the Dynamic configuration for mapping?

[Former Member]

Hi Justin,

The folders are already created on receiver side.

"Have u had a chance to look at the Dynamic configuration for mapping?"

Yes Dynamic config works when when one group[having same WERKS value] of IDocs triggered to PI and all the group of IDocs transforms to CSV but the challenge here is to achieve the grouping of IDocs based on WERKS value. Actually till now the sender R3 system settings are not made to trigger the IDocs so I am testing by WE19.

Thanks,

Nithin.

[Former Member]

Hi Justin,

Thanks for reply! Pls find the below answers.

How do you receive the message in PI? More than one ORDERS01 IDOC in single transaction correct?

Yes more IDocs are triggered in single transaction since collect IDocs option selected in partner profile, a background job will be executed once daily to collect all the IDocs only with one WERKS field per IDoc.

Each ORDERS01 IDOC can have more than one plant segment(WERKS), correct ?

No. only one WERKS field will be populated with a value in each IDoc.

Thanks,

Nithin.

[former_member186851]

Hello Nithin,

A background job is collecting all Idocs right?

once the IDOC is received in PI with multiple configure multiple Interfaces or receivers based on Werks,so that files will written based on each plant.

[justin_santhanam]

Nithin,

Sorry for asking so many questions. I wanted to make sure we are all on the same page, and it will help us to guide in the proper direction.

Please take a look at the below blog and can you confirm whether you are receiving n no.of IDOCS in single transaction? (just like in the example below they have 10 IDOC's in single transaction)

If yes, I was thinking instead of doing multi mapping (from n IDOCs to 1 IDOC) and sending it back to PI for our second scenario, is it possible to change the collect IDOC packet size to 1 so that you receive only one IDOC into PI and still achieve the same initial solution I proposed?

Let me know your thoughts!

Thnx.

[justin_santhanam]

Raghu,

What do you mean by configure multiple interfaces? Can u explain, please?

>>>>once the IDOC is received in PI with multiple configure multiple Interfaces

Thnx

[former_member186851]

Hello Justin,

One IDOC to Mutliple Interfaces based on Werks, Since his Receiver his same.

[maheswarareddykonda]

Hi Justin,

its great

what i learned from you is, if we stick just one we can not go further ..thanks again.

we can just make that partner profile trigger mode 1, then it becomes scenario very easy.

[Former Member]

Hi Mahesh,Justin,

Thank you very much for your reply and time!

@Justin, All : Hope you understand my requirement well. Let's speak about "Scenario 1 - Get the IDOC's and split them in WERKS (multi mapping) and loop it back to PI  (Multi-mapping with Dynamic Configuration - SOAP loopback approach  Thanks to Eng Swee Yeoh ) "  My question is why multi-mapping required here? How grouping of IDocs done here? since the receiver structure is same with same field names for all different WERKS field value but requirement here is same structure with same field names for receiver should get generated but step wise as below.

1)  As soon as the IDocs with different WERKS value are received to PI then it they should be grouped in to, say "m" number of groups for "m" different WERKS field value.

2) Let's say a group contains "n" number of IDocs data hence, that group should be transformed in to CSV with "n" number of line items and each CSV formed should be posted to the corresponding folder out of 10 folders based on the name of WERKS field value.

Can you please on how the above requirement can be achieved by approach of SOAP loop back suggested by Eng ? and also how to trigger many test IDocs with different WERKS field value from WE19 rather than triggering single or many IDocs[repeat factor] with same field values?

Thanks a ton,Nithin.

[justin_santhanam]

Hi Nithin,

Sorry I didn't understand your question. How do you receive the message in PI? More than one ORDERS01 IDOC in single transaction correct? Each ORDERS01 IDOC can have more than one plant segment(WERKS), correct ?

Thanks!

[Former Member]

Hi Anupam,Raghu,

Thanks for your reply. Totally there are 10 different folders in receiver system.

Thanks,Nithin.

[Former Member]

Hi Anupam,Experts,

Any other way other than NWBPM?

Thanks,Nithin.

[anupam_ghosh2]

Hi Nithin,

                 It is possible without nwbpm. Can you please let me know how many different folders you need in this scenario.

Regards

Anupam

[former_member186851]

Nithin,

Using ABAP report collect all IDOCS.

Then send to PI.

The folders are dyanmic and how many folders are in place?

[Former Member]

Hi Raghu, Thank you for useful inputs.

@ All: Pls let me know whether it is feasible through any ABAP report, actual process is a job scheduled in sender R3 system so that all IDocs are collected from the system and sent to PI. So my question is it possible from R3 side by ABAP to group the IDocs based on WERKS field and then collect them and trigger them to PI in one go? so that later PI can process them and create CSV file and post it to the corresponding folder by ASMA.

Thanks,Nithin.

[anupam_ghosh2]

Hi Nithin,

                 This is possible in ecc side to collecta and send idocs based on werks field.

This requires lot of development on abap side.so they might not agree.

Regards

Anupam

[Former Member]

Hi Raghu,

As of now on an average 100 IDocs are sent daily from which they have to be categorized based on WERKS field and then split to multiple files of based on different WERKS value should be sent to multiple folders accordingly. I mean say 100 IDocs have 10 different WERKS value, so they should be grouped into 10 different files with each file having line items formed from the IDoc data of same WERKS field and which are then to be posted to 10 different folders according to the WERKS value. We can do grouping of IDocs based on WERKS field using UDF I hope and after that pls let me know on the possible ways to split them? Also the challenge here is column names in every CSV[which are same as field names in XML file] should appear in every child file.

Thanks,Nithin.

[former_member186851]

Hello Nithin,

For a day if you wish to collect all Idocs you should use NWBPM.

Then you can split based on Werks @ mapping level.

[Former Member]

Hi Raghu,

I am working on PI 7.31 SP11 single stack, pls let me know whether it is feasible by UDF since the volume of data is less?

Thanks,Nithin.

[former_member186851]

Nithin,

You can do one more thing,How many IDocs you wish to collect?

Based on that you can do IDOC packaging,Once you get the collected Idocs in PI,you can do the split logic.

[Former Member]

Hi Experts,

As discussed above the requirement is to trigger the CSV with multiple line items to a folder according to the value in WERKS field hence in sender if a many IDocs with different WERKS value populated to the field and PI should group different IDocs based on WERKS field and should trigger CSV with line items [each CSV formed from multiple IDocs with same WERKS field] them to a different folders.Kindly suggest your ideas on the same

Thanks,Nithin.

[former_member186851]

Hello Nithin,

Is it a single or dual stack PI?

You should use BPM to solve this.(Collect pattern).