IQ log .iqmsg file did not update until stop server

[Former Member]

Hi,

We use SAP IQ16 SP8 for data warehouse. We found that the .iqmsg did not change anything since IQ server started.

It only bulk update all activities when we stop the server.

This make us very difficult to find out what's happening.

Cause we are try tuning our ETL.

Best regards,

Chupong

[c_baker · Accepted solution]

IQ continually adds to the .iqmsg log.

In Linux/Unix, you can 'tail -f' the file to watch entries add to it.

For Windows, the file is locked exclusively until the IQ server quits.  It is still growing, but you cannot see it.

There may be ways to see the file in Windows as it grows, using some OS features (I think you can play wtih pipes, the user starting the service and powershell, but do not know specifics).

What platform are you on?

Chris

[c_baker · Accepted solution]

You do not really need to tune the ETL, just make sure you are using the IQ Bulk load (LOAD TABLE) capability.

Anything else will use row-by-row inserts, which are very slow in IQ, no matter how you try tuning.

You can use RLV but, again, that is not the correct solution and will not be as fast as bulk loads.

Chris

[Former Member · Accepted solution]

Hi Chris

It's IQ16 on Windows 2012r2 server. So how can I see the IQ log without stop it?

We are changing ETL to use bulk load but we found from ipmsg that, in a job, records was insert 1 by 1 (still can not find out why). So we want to confirm that it doing bulk load for real.

Another question, we found that bulk load will use only single core CPU.

Can we improve it? We use SAP Data Services 4.2 SP6 with target table bulk load options.

Thanks

Chupong

[c_baker · Accepted solution]

IQ uses all cores to load records whether row-by-row or bulk.  There is probably another reason why you are seeing this.  It could be anything from the way the disk is configured (threads are waiting for disk access) or you could have the ETL process (which is probably single threaded) running on the same machine.

In short, the graph doesn't tell anything.

Along with the manuals, I would suggest reviewing the following docs:

http://scn.sap.com/docs/DOC-46166

http://scn.sap.com/docs/DOC-39896

Chris

[markmumy · Accepted solution]

What ETL tool are you using?  In their bulk loader, is it actually calling the IQ bulk loader?

As far as the IQMSG is concerned, there are tools that can see the file as it is changing.  It's been a while, but I seem to remember using Cygwin 'tail' (specifically tail -f) to see the changes to the tail.

You may want to grab the open source Cygwin tools that give you more Linux like commands and control.

Mark