Loading text file in xMII

[former_member209705]

Hi Experts,

I have an requirement to load text file into xMII and process(reading fields from text file) it. I'm using Text

Loader to load the text file and am linking the StringContent property to input of flat file parser.

Now the output that am getting from flatfile parser is the content of my input file in one single row as string

without any spaces or delimiters.

  <?xml version="1.0" encoding="UTF-8" ?> 
- <Rowsets DateCreated="2008-04-11T13:51:32" EndDate="2008-04-11T13:51:32" StartDate="2008-04-11T13:51:32" Version="12.0.2 Build(88)">
- <Rowset>
- <Columns>
  <Column Description="LineNumber" MaxRange="0" MinRange="0" Name="LineNumber" SQLDataType="5" SourceColumn="LineNumber" /> 
  <Column Description="Line" MaxRange="0" MinRange="0" Name="Line" SQLDataType="1" SourceColumn="Line" /> 
  </Columns>
- <Row>
  <LineNumber>1</LineNumber> 
  <Line>500reels\200803\004770.0104770ReelLength263602StartTime1204379532ReadyTime1204384880NoCommentsFormationPictures180000000EdgePictures11000079315101SnapShots00000000000012759549562000956615295643800000000001FormationData44929070178000000000EdgeData63270500000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000AAAAAAAAAAAAAAAAAAAAAA8037</Line> 
  </Row>
  </Rowset>
  </Rowsets>

Am not able to read the file contents based on end of line or any other delimiters.

Please suggest me a way to load a text file without changing it from original format and also able to parse it.

my source file is in *.txt format and am using xMII 12.0.

Regards

Ravi shankar

[Former Member · Accepted solution]

Does the text file have fixed width fields? If not, forget about it.

If it does, use the string parsing functions to assign local variables to each defined element in the long string.

I had to do this a couple of times, most recently with 12MB files coming out of a VAX D system older than me. It actually processed very quickly.

[former_member209705 · Accepted solution]

Hi Ryan,

Thanks for your reply.

File contains dynamic field widths.

Please clear me, is getting a file through 'flat file parser' is the only way of using a text that is loaded?

Regards,

Ravi shankar

[agentry_src · Accepted solution]

Given your file sample, can you define which fields are variable length and which are fixed in length?

Does each variable length field stop at the next named field (or at the EOF)?

Are the field names consistent from file to file?

Depending on the answers, you may be able to sort it out. If you give me the specific answers, I will give it a try.

Thanks,

Mike

I love a challenge...

[Former Member · Accepted solution]

Hi Ravi,

basically this is easily done with xslt. The steps are:

1. LoadFile

2. CreateRows

3. Call xslt that does the parsing

4. Assign to a Doc

So all comes down to parsing with xslt. This is quite powerfull and should do the job.

Regards,

Matthias

[Former Member]

Ravi,

did you customize your link between ImageLoader and FlatFileParser as descriped above

(base64decode( Image_Loader_0.EncodedImage ) to Flat_File_Parser_0.Input)? Looks like,

that you forgot this.

Pedro

[Former Member]

Ravi,

yes this will work also with *.txt, when your file really have a carriage return per line.

Make sure that you are using the right delimiter "," or ";".

Let me know.

Pedro

[former_member209705]

Pedro,

The output which am getting is...

[INFO ]: [B@17f035e 

I still dont understand, how a image loader can encode a text file and give the output to flat file parser.

Ravi shankar

[Former Member]

Hi,

solution for loading CSV Files from external location could be the following:

1. Sequence_0; ImageLoader, Configure URL like "C:/temp/MyFile.csv"

2. Sequence_1; FlatFileParser, Link Expression base64decode( Image_Loader_0.EncodedImage ) to Flat_File_Parser_0.Input

3. Sequence_2; Repeater, Configure "Flat_File_Parser_0.Output{/Rowsets/Rowset/Row}"

4. Sequence_3; StringListToXmlParser, Link Expression Repeater_0.Output{/Row/Line} to String_List_To_Xml_Parser_0.Input

5. Sequence_3; StringListToXmlParser, Link Expression "YourDelimiter" to String_List_To_Xml_Parser_0.Delimiter

6. Sequence_4; Tracer, Link Expression String_List_To_Xml_Parser_0.Output{/Rowsets/Rowset/Row[1]/Item} & " - " &

String_List_To_Xml_Parser_0.Output{/Rowsets/Rowset/Row[2]/Item} to Tracer_0.Message

Point 6. is only for showing the result.

Hope this help.

Pedro

[former_member209705]

Hi,

Pedro my source file is .txt file. Will this method also works for text file?

I tried it with text file(comma seperated), no success....

Ravi Shankar

Edited by: Ravi Shankar on Apr 22, 2008 10:57 AM

[Former Member]

Not really...

I just found, if you import the file to workbench at first, the text loader works fine in this case and "end of line" characters are still in the text string.

[Former Member]

Hi,

I have the same problem.

I tried to parse CSV file (comma separated value) in MII 12.0.3.

The file located on the file system and wasn’t imported to MII workbench. By using of text_loader action were deleting all “end of line” character.

Therefore Flat File Parser doesn’t detect the end of line and I get back just one row with <line> with complete content of CSV file.

I think, the problem in my case is on text loader and not on file parser.

[former_member209705]

Hi,

Exactly, but did you arrive at any feasible solution?

Ravi shankar

[agentry_src]

If the location of each piece of data is fixed and of fixed length, you could parse it by using the various string functions:

stringlength, stringleft, stringright, stringpart, etc.

It does look like you have some regularity of data after the initial 30 or so characters.

So where you have regular names like ReelLength, StartTime, ReadyTime, etc.

You could do a stringindexof ([YourLineOfData], "ReadyTime") and it would return the location in your line of data.

That number plus stringlength of "ReadyTime" gives you the start point of the actual ReadyTime data. The length of the data field might be variable, but you can then do stringindexof the next fieldname and figure the data length that way.

If you don't have delimiters, fixed fields or standard fieldnames, then I suspect you are out of luck.

I would check back with the input file and see if it could be delimited in some way or restructured.

Good luck,

Mike

[former_member209705]

Hi Appleby,

Right now am using the functions stringindexof and stringpart function to extract the fixed fields. But there are fields in the file that is dynamic and their width and number of appearance may differ from file to file. And yes there are no delimiters in file.

Is there any other options to read this file? am really running out of ideas.

Or is there any way in which we could get the file in the same format as my source text file.

Regards,

Ravi shankar

[Former Member]

Ravi,

the Flat File Parser is meant to get the input from a CSV file (comma separated value). Say, you input file is like this:

col1,col2,col3

text,123,44.6

then the parser creates a MII-xml file like

<Rowset>

<Row>

<LineNumber>1</LineNumber>

<Line>Col1,Col2,Col3</Line>

</Row>

<Row>

<LineNumber>2</LineNumber>

<Line>123,Text,334.5</Line>

</Row>

</Rowset>

Now you can use a for loop to run through the lines and string functions to split the comma values. I don't think there is an easier way.

Regards

Michael