Processing of large number of records using JDBC Sender Channel

[Former Member]

Hi experts,

We have a JDBC-File scenario where in the tables contain about 500K records on an average.

I used multimapping to generate a flat file for every 10K. The problem is ..when I start the JDBC Sender CC, the memory goes up and the J2EE engine restarts. In the Sender CC, I gave the Disconnect from the database option too. The query is SELECT * from TABLE and the Update statement is <TEST>. Please help me out how to solve this.

Regards.

[prateek]

How big is your polling interval? If it is short, then it is better to uncheck option "Disconnect from the database". What is the purpose of using multimapping here?

Regards,

Prateek

[Former Member]

My polling interval is 86400 secs..which is one day. The purpose of multimapping is to generate a file for every 50K records...I am using message split using node functions. Also I am controlling the CCs externally for start and stop operations.

Regards.

[Former Member]

Hi,

We are facing a similar issue here. We have a select query as SELECT * FROM TABLE.

I have the read the note 1253826. I want to know if there is a way to limit the maximum number of rows to be processed in the sender CC. For example, if I the TABLE has 10k rows, I want to process only 1000 rows at one time with the same query.

Any help would be greatly appreciated.

Regards.

[Former Member]

Hi

Use the below query,

// Oracle

SELECT Statement : select colname from tblname where rownum<=1000 ;

// MSSQL

select top 1000 colname from tblname

Regards

Ramg

[Former Member]

Hey Ram,

Thanks for the reply. But what about the remaining 9000?

I don't have an Update statement because there is no flag in the table.

Is there a way that I can read 1000 rows 10 times using a query?

Regards

[former_member206760]

u need to have a status field in the table where intial status will be say N for unprocesses records..

u just have to select say top 1000 records where status = N using a select query like select top 1000 where status = N

and then in update statement wrtie a query to update the status to P for the top 1000 records selected in the above qry