Can NCo 3.0 retrieve a million records?

[Former Member]

Folks,

I have a custom ABAP module that sometimes returns million+ records. With NCo 2.0, this was not directly possible as the connector would try to get all the records at once in memory. Fortunately, NCo 2.0 supported a mechanism to get the data as callback (using DESTINATION CALLBACK in my ABAP code). Thus, my ABAP code can keep sending the data on my callback until all the records are exhausted.

Now, I am moving my .NET code from NCo 2.0 to NCo 3.0. As I understand, there is no support for "DESTINATION BACK" in NCo 3.0. This is not an issue for me as long as the new NCo 3.0 mechanism can handle returning million+ records. This can happen only if the connector streams records and not try to build the whole resulting table in memory.

So, does NCo 3.0 try to get the records as a network stream or fill the whole structure of records in memory as NCo 2.0 used to do?

Thank you in advance for your help.

Regards,

Peter

[MarkusTolksdorf]

Hello Peter,

NCo 3.0 is not offering an interface that would allow the streaming of result tables. Hence, the complete result is allocated in memory.

However, the architecture of NCo 3.0 is now completely different compared to NCo 2.0. Therefore, it is no longer necessary to allocate the memory for the data represented in a table in a single memory block, but it is distributed across the heap. Moreover, the NCo 3.0 has its own RFC layer with the consequence that there is no longer the need of an additional intermediate representation of the data in a separate RFC library. Last but not least, NCo 3.0 also supports the 64bit variants of the .NET framework, thus it is possible to allocate much larger data blocks like before.

Summary: You should give it a try, whether NCo 3.0 works in your use case. I would guess there is a good chance.

Best regards,

Markus