Loading CSV file automatically

[freekmenger]

A question about the best/fastest way to go here; I have some CSV files on a separate server that need to be loaded 'automatically' by SAP HANA.

I know that you can load CSV files through FTP, but that wouldn't meet my requirements. I need a program that can load the CSV data 'on call' instead of an FTP transfer that I need to run myself.

I've tried:

1. XSJS => Calling SQLscript directly, line by line. Not a good idea, my CSV files are about a 100.000 records, so I'm getting timeouts when loading. The recommendation is to push as much SQL processing to the SQLscript/procedures. The other disadvantage is that I can't get batch processing to work, I get the error: 'Bulk insert not enabled'.

2. XSJS => Creating an SQLscript procedure and passing the whole string (or parts of it) to the procedure. This doesn't work as I'm getting an error: The requested unit size(17168574) should be smaller than 1048576 (which is the the size of a megabyte?). I should be able to get this working though, but it seems very cumbersome doing it this way.

Just so I know I'm on the right track... would there be a better way to transfer a CSV from an external server to SAP HANA? (programatically)

[Former Member · Accepted solution]

Hi Freek,

I think that you can use BODS tool to upload them automatically, but you need to share folder on your file servers firstly.

Hope this can help you.

Regards,

Jerry

[freekmenger · Accepted solution]

Hi Jerry,

Thanks for the reply. You are correct, but I'm looking for something that does not rely on using an extra tool though. Would be nice if I can just use the SAP HANA functionality, it should be possible...

[Former Member · Accepted solution]

HI Freek,

http://scn.sap.com/thread/2072330

http://www.youtube.com/watch?v=4B55DrzFyIM

http://www.youtube.com/watch?v=MlElWeBSkHs

http://scn.sap.com/docs/DOC-27960

http://wiki.sdn.sap.com/wiki/display/inmemory/Importing+CSV+files+into+SAP+HANA

[Former Member · Accepted solution]

Hi Freek,

As my knowledge, I think that it maybe has no better way to implement it in HANA functionality so far, if you do not use external ETL tool.

If you want to implement it to use SAP HANA functionality, as your mention is using XSJS, if possible, you can split your corresponding data file to smaller data file.

Hope this can help you.

Regards,

Jerry