Expensive Architecture (Data Access Extension)

[Former Member]

(Lumira Desktop version 1.19 used)

In External DataSource option, there are back to back two calls made with preview and refresh modes at the time of data acquisition. It makes it architecturally expensive.

Imagine a web service which returns decent about of data (i.e. data intensive service). Single step of data acquisition in Lumira calls web service twice. First time when we press “Next” button on new dataset screen with 300 rows and preview mode and next time normal refresh mode. If we look at it from service publisher view point, then we have one unnecessary call is being made to server as a refresh mode at data acquisition step. High probability of not being acceptable by the source/customer.

Logically speaking it should be one call when we acquire data set and next subsequent calls as normal manually done refresh.

I don’t know the real reason behind SAP making it this way. Probably, it does the data sampling to determine the data types of columns with few rows and when “Next” button is pressed and it considers the complete data set on redundant refresh call later on “Create” Button.

There is a workaround through which we could avoid two calls but I would like to know the reason of its current working model.

I have already put-in an idea about this. If you agree then vote for the idea.

Expensive Architecture (Data Access Extension) : View Idea

Best Regards,

Mayank

[Former Member · Accepted solution]

I doubt the right way is to consolidate the calls, an efficient extension would never fetch the entire dataset from the source in PREVIEW mode.

But your argument makes sense at least for webservices which support ETag headers. They are used to remove the need for redundant calls when data hasn't changed on the server since the last request.

Currently, we can save the latest data to a csv file locally and read it from that file whenever the server returns a 304 Not Modified in subsequent calls. It'll be great if the SDK/Lumira always caches the latest set of data for a connection and allows developers to use the cached data with a simple API call instead of fetching again from the source.

[Former Member]

Most likely in DAE,  web Service to integrate with will exist. Cases where a new service is created for DAE, keeping the DAE friendly integration in mind would not be many.

Practically speaking we would be integrating with an old existing web service which just returns a data set in XML or JSON format.

In the very first step Lumira calls the extension with mode preview and size 300 and on the same step it makes another call with refresh mode.

once we acquired data in the first call then what is the need of making refresh call on the same step.

May be I am missing out something, I would like to know it in detail.

I liked DAE developer guide very much. but it is quite on this logic behind two calls made to extension the very first step, probably left to consultants to figure out.

[Henry_Banks]

Hi,

interesting feedback, however i'm not personally familiar enough with SDK Data Access Extensions to comment. perhaps Former Member   will be along to comment later.

I had a look at the Lumira DAE SDK guide http://help.sap.com/businessobject/product_guides/vi01/en/lum_120_dae_dev_en.pdf

and it does explicitly call out "preview, edit, and refresh" as 3 distinct phases where the datasource is hit.  (there's a neat flow diagram on page 10 in the above link)

It might also be useful if you fleshed out the thinking/solution behind your "workaround which avoids two calls"

Thanks & regards,

H

[Former Member]

Thanks Henry.

I would detail the solution which I used while working on an DAE Integration for a customer.

Looking forward to hear from the expert.

[Former Member]

Hope to hear something from product team on this. Thanks.