Hadoop to SAP IQ through FIFO Pipe
There are currently four existing methods for integration between SAP IQ and Hadoop:
- Data Federation
- Query Federation
- Client-side Federation
This archive file contains a document that describes yet another ETL method: push data from the Hadoop file system (HDFS) to a FIFO pipe, and then load the data into a global temporary table in IQ using the LOAD TABLE command.
In tests performed on a virtual machine (VM) with modest configurations, it took approximately 2 seconds to load 100,000 rows with 2 columns worth of data from HDFS to IQ. The zip file also contains the scripts and data that implement the approach.