Memory issues & jobs hanging

[Former Member]

We are staring to expeience jobs "hanging", meaning they get stuck on a data flow. The hanging happens on different data flows and tables during our nightly loads. Here is a sample of the last two Trace statements before it hung.

(11.7) 09-30-08 08:40:26 (8128:5284) DATAFLOW: Cache statistics determined that data flow <TableXYZ> uses <0> caches with a total size of <0> bytes. This is less than(or equal to) the virtual memory

<1606418432> bytes available for caches. Statistics is switching the cache type to IN MEMORY. (11.7) 09-30-08 08:40:26 (8128:5284) DATAFLOW: Data flow <TableXYZ> using IN MEMORY Cache.

Any ideas what we are doing wrong?

[Former Member]

I tired this if I open the view in SQL server it gives arithmectic overflow error, but for me the Job is failing with error "error message for operation <SQLFreeStmt>: <>.", though the error message is not about arithmetic overflow, the job does fails with error

is this the only source table or there is join? I just tried with a simple DF where the view is source and writes to a XML target

[Former Member]

There are views on the source database that match one underlying table each with one additioanl column that converts a binary to an integer. The column in the view that does the conversion experiences the artithmetic overflow.

My client is most concerned that the job hung and no error code was returned in the DI job to terminate. The job hung overnight until I killed it the next morning. Besides the issue of the specific eror condition and its handling, is there any way to force the DI job to terminate if it takes too long?

And also a quick mention why we are doing the conversion from binary to begin with. The source system/application that we are using uses "timestamps" (these are NOT a datatime as noted in another post but rather ninary numbers.) The "timestamp" datatype is not recognized by DI and in fact the columns are hidden when you setup the tables in the Datastore.

[werner_daehn]

If I recall correctly, what I did to read SQL Server timestamp data into DI is editing the datastore and check the flag "import unsupported datatypes as varchar" and then re-import the table.

No, in DI there is no watchdogtimer that fires after a time lap of inactivity.

Are you sure it is that? Not a database lock preventing DI to read the data??

[Former Member]

Problem with DI not recognizing an arithmetic overflow error message from SQL Server and hanging.

[scott_broadway]

Hi Richard,

Can you explain this error in a little more detail? If you feel this is a product error, please open an incident on the SAP Service Marketplace.

Thanks,

~Scott

[Former Member]

The data flow was accessing a view from a SQL server 2000 db as a soiurce. Opening the view from SQl resulted in an arithmetic overflow error. When the DI data flow encountered this condition it just "froze", i.e. no procssing. It stayed in that condition for a few hours until I aborted it in the morning.

I would hoped DI would have trapped the error and stopped.

Also, the workflow where the data flo exists has 27 other DFs that are not connected so there was not a way to even determine, at least from DI, what DF or table.view it was hung.

[Former Member]

Also any suggestion on how to force a time out. I posted the ?

[werner_daehn]

Interesting.... So you are saying I should create a table with two columns, INT1 and INT2, one row with the value 10 and 0. And then build a dataflow reading from it with the select INT1/INT2 from table?

Or is it that I have to multiply two very large integers to get an INT overflow?