cancel
Showing results for 
Search instead for 
Did you mean: 

Query from a huge data source

Former Member
0 Kudos

I am currently working on the query on the BO through a Java application. Following the sample procedure, I can perform a query to extract data from the built-in Universe [e-fashion] using the JDK-provided web services: QueryService and ReportEngine.

However, I encountered some problems when doing the same thing in a universe that contains million rows of data [let say Unvierse A]. I am not sure how the differences between the 2 universes affect the result.

I have tried several times like setting different query objects and query condition but the results still outputs error. There are some error messages I commonly found when I tried to execute the query from the Universe A.



DataProviderInformation  boDataProviderInfo = boQuerySrv.createDataProvider(boUniverseUID, boQuerySpec, boRetProp);
			
DocumentInformation docInfo = objReportEngine.getDocumentInformation(boQueryDocRef, null, actions, null, retBOData);

org.apache.axis2.AxisFault: Read timed out

org.apache.axis2.AxisFault: Read timed out

When I tried doing the same query job in QaaWS / Webi, the report result looks fine. Am I missing some setting in Java ?

Any help is greatly appreciated.

-

Here are my findings on the Universe Specification:

(com.businessobjects.datasource.DataSourceSpecification)

[e-fashion]

>> getAdvancedConditionSupported() - true

>> getBothOperatorSupported() - true

>> getCombinedQuerySupported() - true

>> getConditionObjectValueSupported() - true

>> getDuplicateRowSupported() - true

>> getEditQueryAllowed() - true

>> getIsNullOperatorSupported() - true

>> getMaxInListNumber() - 999

>> getMaxRetrievalTime() - 300

>> getMaxRowsRetrieved() - 90000

>> getMissingRowSupported() - false

>> getName() - eFashion

>> getNotIsNullOperatorSupported() - true

>> getPercentageRankConditionSupported() - false

>> getRankConditionSupported() - false

>> getSortOnAnyObjectSupported() - true

>> getViewSQLSupported() - true

[Universe A]

>> getAdvancedConditionSupported() - false

>> getBothOperatorSupported() - false

>> getCombinedQuerySupported() - false

>> getConditionObjectValueSupported() - false

>> getDuplicateRowSupported() - false

>> getEditQueryAllowed() - true

>> getIsNullOperatorSupported() - false

>> getMaxInListNumber() - -1

>> getMaxRetrievalTime() - 600

>> getMaxRowsRetrieved() - 5000

>> getMissingRowSupported() - false

>> getNotIsNullOperatorSupported() - false

>> getPercentageRankConditionSupported() - false

>> getRankConditionSupported() - false

>> getSortOnAnyObjectSupported() - true

>> getViewSQLSupported() - false

Accepted Solutions (0)

Answers (3)

Answers (3)

Former Member
0 Kudos

Hi Ted, hi Thomas,

I am facing a similar issue running BO XI 3.1 and trying to retrieve Desktop Intelligence report data using the Webservices API.

Retrieving a single report works fine, but when I try to put some load on the server and request reports with 4 parallel threads, I get the "Read timeout error" when calling getDocumentInformation(repID, null, actions, null, boRetrieveData). The actions array just contains the FillPrompts instance.

BTW, the threads use different logins and in consequence different connections/sessions.

The timeout occurs after 30secs, so simple reports with no prompts are created without error. Increasing the timeout on the connection as suggested did not help.

I think the issue is related to Axis2. I don't know how to set the timeout on the Axis client via the BO API.

Thomas, did you open an incident at SAP support? Any outcome?

I will file the issue separately, just to make sure it's not overlooked

Regards,

con

Former Member
0 Kudos

Hi Ted

This is Thomas.

Actually I have added the timeout-setup codes before the web services get started, like this:


// Setup the Session object
Connection boConnection  = new Connection(sessConnURL);
boConnection.setTimeOut;
Session boSession = new Session(boConnection);
boSession.getConnection.setTimeOut;

...
...

// Setup the QueryService object
QueryService boQuerySrv = null;
if  (strBOQuerySrvURL.length > 0)
{	
	boQuerySrv = QueryService.getInstance(boSession,strBOQuerySrvURL;
	boQuerySrv.getConnection.setTimeOut;
}

...
...

boQuery.setQueryResultArray(boQueryResultObjects);
boQuerySpec.setQueryBase(boQuery);	

ReturnProperties  boRetProp = ReturnProperties.Factory.newInstance();
boRetProp.setIncludeQuerySpecification(false);
DataProviderInformation  boDataProviderInfo = boQuerySrv.createDataProvider(boUniverseUID, boQuerySpec, boRetProp);
String boQueryDocRef = boDataProviderInfo.getDocumentReference();

...
...


// Setup the ReportEngine object
ReportEngine objReportEngine = null;
if  (strBOQuerySrvURL.length > 0)
{	
	objReportEngine = ReportEngine.getInstance(boSession, astrURLs[0]);
	objReportEngine.getConnection.setTimeOut;
}

RetrieveData retBOData = RetrieveData.Factory.newInstance();
RetrieveView retBOView = RetrieveView.Factory.newInstance();
retBOData.setRetrieveView(retBOView);
Action[] actions = new Action[1];
actions[0] = Refresh.Factory.newInstance();
DocumentInformation boDocInfo = objReportEngine.getDocumentInformation( boQueryDocRef, null, actions, null, retBOData);
CharacterView myBOView = (CharacterView)boDocInfo.getView();
String docContents = myBOView.getContent();

...
...

(I am wondering why I cannot add "()" to part of my code here, it appears "POST to /edit.jspa not supported" every time when I try to attach my original codes. Please be noted that each Time Out is set 500 'seconds')

Am I still missing something? Please advise as I am quite new to this.

Many Thanks.

ted_ueda
Active Contributor
0 Kudos

In your code snippet:


boConnection.setTimeOut;
Session boSession = new Session(boConnection);
boSession.getConnection.setTimeOut;

I don't see the new value you've set the timeout to...

Sincerely,

Ted Ueda

Former Member
0 Kudos

Hi Ted

I don't know why when I posted the thread with my original codes from my program, the forum returns an error.

Please be understood that each of the TimeOut setting at Session, QueryService and ReportEngine is configured as followed <Please ignore the syntax error> :

int setTimeOutPara = 500;

getConnection<>.setTimeOutset<TimeOutPara>;

However, the Timeout error remains in the program.

ted_ueda
Active Contributor
0 Kudos

If it's still getting the Axis2 socket error, then something more interesting is going on.

You may want to open a SAP Incident for this issue.

Sincerely,

Ted Ueda

ted_ueda
Active Contributor
0 Kudos

How long does it take for the "Read timed out" to appear - is it more than 20 minutes?

Are you returning millions of rows in the query in your code?

Are you returning millions of rows in the QaaWS?

Sincerely,

Ted Ueda

Former Member
0 Kudos

Hi Ted

Thanks for your help.

1. The query takes me around 1-2 minutes and it appears the axis fault error.

2 & 3. I have tried to build the query in the QaaWS and Webi, and it returns less than 50 rows of data after some time (about a few minutes). I just constructed the same query in consumer APIs and expected the same query result obtained, but it failed.

ted_ueda
Active Contributor
0 Kudos

Connect.setTimeOut to modify the Axis2 socket timeout.

Sincerely,

Ted Ueda

Former Member
0 Kudos

Hi Ted,

I have tried the following to set the getConnection().setTimeOut codes to the instances of Session, QueryService and ReportEngine objects to 1000, but it appears another connection timeout and the program halts after about 2 minutes:


org.apache.jasper.JasperException: Exception in JSP: /query_dcs.jsp:115

112: 	boEnterpriseCredential.setAuthType(authType);
113: 	
114: 	// Login to the Enterprise System and retrieve the SessionInfo
115: 	SessionInfo boSessionInfo = boSession.login(boEnterpriseCredential);
116: 	String SterSess = boSessionInfo.getSerializedSession();
117: 	String SessToken = boSessionInfo.getDefaultToken();
118: %>


Stacktrace:
	org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:451)
	org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:361)
	org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:329)
	org.apache.jasper.servlet.JspServlet.service(JspServlet.java:265)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


root cause 

org.apache.axis2.AxisFault: Connection timed out: connect
	org.apache.axis2.AxisFault.makeFault(AxisFault.java:417)
	org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:195)
	org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
	org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:327)
	org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:206)
	org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:396)
	org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:374)
	org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:211)
	org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
	com.businessobjects.dsws.session.SessionStub.login(Unknown Source)
	com.businessobjects.dsws.session.Session.login(Unknown Source)
	org.apache.jsp.query_005fdcs_jsp._jspService(query_005fdcs_jsp.java:209)
	org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
	org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:331)
	org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:329)
	org.apache.jasper.servlet.JspServlet.service(JspServlet.java:265)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


root cause 

java.net.ConnectException: Connection timed out: connect
	java.net.PlainSocketImpl.socketConnect(Native Method)
	java.net.PlainSocketImpl.doConnect(Unknown Source)
	java.net.PlainSocketImpl.connectToAddress(Unknown Source)
	java.net.PlainSocketImpl.connect(Unknown Source)
	java.net.SocksSocketImpl.connect(Unknown Source)
	java.net.Socket.connect(Unknown Source)
	sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
	sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	java.lang.reflect.Method.invoke(Unknown Source)
	org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
	org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:124)
	org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
	org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1321)
	org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
	org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
	org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
	org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
	org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:520)
	org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:191)
	org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
	org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:327)
	org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:206)
	org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:396)
	org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:374)
	org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:211)
	org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
	com.businessobjects.dsws.session.SessionStub.login(Unknown Source)
	com.businessobjects.dsws.session.Session.login(Unknown Source)
	org.apache.jsp.query_005fdcs_jsp._jspService(query_005fdcs_jsp.java:209)
	org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
	org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:331)
	org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:329)
	org.apache.jasper.servlet.JspServlet.service(JspServlet.java:265)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


Any clues to this ?

Thanks

ted_ueda
Active Contributor
0 Kudos

The stack trace still indicates an Axis socket timeout.

You need to set the timeout right at the beginning - I don't see you set it in your code snippet.

If you are setting it right at the beginning, then it's a more interesting issue, so I'd recommend, if you have a support contract with SAP Business Objects, opening a SAP Incident.

Sincerely,

Ted Ueda