MII locking up and Java applet performance

[Former Member]

Our MII web application is locking up intermittently. When this occurs, it must be unlocked by performing the equivalent of Ctrl-Alt-Delete. The issue is unpredictable but happens often enough to seriously frustrate users.

For reasons below I believe this is a performance problem with our Java applets, but this is difficult to prove.

Resolutions we’ve considered are listed below. Comments would be appreciated.

Symptoms of current problem:

• When people use MII on our server farm, occasionally it will lock up. (See “system landscape” below for server farm details.)
• This has happened more and more frequently, as more and more users have started using the server farm
• Happens often enough to be very frustrating to users
• Happens seldom enough that we cannot reproduce it on demand
• We can’t identify a specific trigger within the MII web application
• When this happens, the user can “unlock” the lock-up by triggering a process we created that kills that user’s IE and Java processes on the server farm.
• Lest you think I am leaving you no clues whatsoever, see below for a similar previous problem we were able to resolve.

Previous problem Nov 2012: iBrowser applet function causing lock-ups

• Symptoms:
  • We had an .irpt page in MII that periodically locked up (requiring Ctrl-Alt-Delete). It couldn't be reproduced by users that were VPN'ed into our system, but users on site could lock up MII by clicking on a certain button frequently.
  • When it locked up, you could unlock it by killing either the user's IE process. You could also unlock it by just killing java.exe. So clearly a Java issue.
• Cause: Eventually this was traced to an iBrowser applet function that got called when you clicked on a button.
  • My coworker tells me this was linked to a bug with that applet function, though I haven't confirmed this.
• Resolution: We removed our usage of that applet function.
• Similarity to current issue: similar, but this was happening much more frequently than our current issue, it could be reproduced, and we were able to isolate the cause.

Possible resolutions we’ve considered:

• Replace some of our applets with AJAX calls (starting with the most frequently-used and easily-replaced applets)
• Buy more servers for the server farm

Our system landscape:

• Our client computers have the following profile:
  • Almost all Windows 7
  • IE 8 is the only browser used
  • Java version 6 Update 18
  • Some use their locally installed IE, but most do not. Most use the server farm’s instance of IE to view MII.
• Server farm
  • Most users get to MII not by opening their local IE, but by executing a local .RDP file to open up a remote instance of IE which runs as a published application on the server farm
  • The published application is done via Remote Desktop technology. (It looks similar to a local IE instance; it does not look like a full-fledged Remote Desktop Connection with a visible desktop, but the technology is basically the same.)
    Windows Server 2008 R2
  • IE 8 is the only browser used
  • Java version 6 Update 18
  • 7 servers are in the farm
  • We bought this server farm specifically for MII to comply with our company’s Java update policies and to address past issues we were having
  • Traffic is load-balanced via Remote Desktop Connection Broker
  • Looks like we currently have up to 15 users per server, but we are looking to expand this.
• MII:
  • Version 12.2.4
  • Running on a Linux server
• PCo:
  • Version 2.3 (2.302.1631.1246)
  • Running on a Windows 2008 R2 server

[Former Member · Accepted solution]

Hi Greg,

without having red everything carefully, this reminds me of a similar behaviour. I see the root cause in the fact that the default settings of MII (Netweaver) are such that it is not scalable at all. I have asked SAP to integrate the settings tested by Salvatore Castro as default settings in MII but it was neglected by SAP. So, please try to configure the settings of MII such that it is capable of normal user load.

Please see the following document to proceed with the settings:

http://scn.sap.com/docs/DOC-1834

Some of the settings are different on 12.2. Please see my add-on paper to the document above below.

To remind everyone: These settings are an absolute must for every MII deployment!

And I ask SAP again: Please make this settings default for every MII installation!

Netweaver MII Performance Settings

Table of Contents

• 1 Netweaver / MII Performance Settings
  • 1.1 Motivation
  • 1.2 Parameters to change with "ConfigTool"
  • 1.3 Additional parameter to place in NW profile

1 Netweaver / MII Performance Settings

  

1.1 Motivation

The settings mentioned in this howto are necessary for a standard MII production environment. This is because Netweaver's default settings permit only a few amount of parallel HTTP requests. If the number of parallel requests exeeds 6 (really parallel with a delay of a couple of ms) blockage appears on client side delaying page / data reloads massively (30s - 60s and more is common). Parallel requests of this kind can happen easily from different browser windows or data reload of multiple graphics on a HTML page which load there data independently. The settings described here increase all necessary thread pool, connection, requests and queue limits to mitigate that problem.

1.2 Parameters to change with "ConfigTool"

The parameters listed here are related to NW version 7.1 Patch 6 and MII 12.2.2.5. Parameters and settings described here are an excerpt from document The Performance of SAP MII on Intel Xeon Hardware Analysis Summary. Parameters have been adapted to NW 7.1 Patch 6. Some of them are not needed anymore because they are built dynamically and some of them have not been set because they are configured by default installation.

Start ConfigTool with running a Windows command window as Administrator. CD into directory and start configtool.bat. Full qualified path:

<drive>:/usr/sap/<SID>/JXX/j2ee/configtool/configtool.bat 

The following table lists the parameters to change to the values given in column "Custom Value". "Path (Instance)" specifies the path of the parameter in the tree view of the config tool.

 

Path (Instance)	Parameter	Default	Custom Value	Remarks
Managers -> ApplicationThreadManager	MaxThreadCount	100	1000
Managers -> ApplicationThreadManager	PercentageOfParallelismAllowed	30	50
Managers -> ThreadManager	MaxThreadCount	100	1000
Services -> cross	maxServiceThreads	5	5
Services -> dbpool	sysDS.initialConnections	1	10
Services -> dbpool	sysDS.maximumConnections	20 max (($link{../../../kernel/ApplicationThread/properties#MaxThreadCount} min (round(200/$link{${SYSTEM_INFO}#NumberOfNodes})))-10)	20 max (($link{../../../kernel/ApplicationThread/properties#MaxThreadCount} min (round(200/$link{${SYSTEM_INFO}#NumberOfNodes})))-10)	Dynamically calculated from Application MaxThreadCount.
Services -> http	FCAServerThreadCount	5	50
Services -> IIOP	parallelRequests	10	1000
Services -> IIOP	requestQueueSize	100	1000
Services -> jmsconnector	maxConnections	100	500
Services -> jmx	NotificationQueueMaxThreads	3	30
Services -> jmx	NotificationQueueThreadThreshold	30	100
Services -> p4	parallelRequests	10	1000
Services -> p4	requestQueueSize	100	1000
Services -> security	sessions_size	200	1000
Services -> servlet_jsp	ProductionMode	true	true	Value True is already defaulted.

Detailed information about each of these parameters can be found in the online: NetWeaver Help Documentation

1.3 Additional parameter to place in NW profile

The following parameters has to be added to the Netweaver profile. Path to profile:

<drive>:\usr\sap\<SID>\SYS\profile\<SID>_<JXX>_<Server> 

Note: The file has no extension.
This is a plain text file so you simply need to open it in any text editor; before modifying this file be sure to first backup this file. The following parameters were modified/added to the profile:

#----------------------------------------------------------------------- # Number of usable threads #----------------------------------------------------------------------- icm/min_threads = 100 icm/max_threads = 1000 #----------------------------------------------------------------------- # Increase the size of the MPI memory area #----------------------------------------------------------------------- mpi/total_size_MB = 500 #----------------------------------------------------------------------- # Number of maximum connections, length of the wait queue #----------------------------------------------------------------------- icm/max_conn = 2500 icm/req_queue_len = 2500 #----------------------------------------------------------------------- 

These parameters are referenced in the SAP NetWeaver CE Help documentation on the ICM for systems with an anticipated “High Load” as described here: http://help.sap.com/saphelp_nwce10/helpdata/en/56/2e453cabf4ef6fe10000000a114084/content.htm

[Former Member · Accepted solution]

This looks like it could be very helpful. I will start reading this today and ask our Basis team to read this as well.

Thank you!

[former_member185280 · Accepted solution]

Also if you are in the config tool you can go to File-->Change System Template and from there you can select some sap provided configuration templates based on your needs. You will probably still have to tweak but templates like CE_Complete_Stack_Production_full etc will have most of the above already set.

Regards,
Christian

[Former Member · Accepted solution]

Matthias, would you recommend any change to these values if we are running MII on a Linux rather than a Windows server?

I'll ask our Basis person to reply with further details about the server.

[Former Member · Accepted solution]

Hi Greg,

sure, it is recommended for every OS as this are mainly settings for Netweaver. MII only uses the Java Stack and as NW is an Java AppServer, the OS does not matter as the settings are application settings.

Cheers,

Matt

[Former Member · Accepted solution]

Hi Christian,

thanks for the info. Was not aware of this system templates. Nice. Will try this in a next deployment.

Cheers,
Matt

[Former Member · Accepted solution]

Just for the sake of reference, I'll say that the above configuration changes did not get rid of our problem. They didn't hurt, but didn't achieve the benefit we'd been hoping for. I appreciate the advice all the same, and we are continuing to use the suggested settings.

[Former Member · Accepted solution]

Hi Gregory,

I am sad to hear that. So now I looked a bit closer at your description.

It obviously is a problem on the client as you correctly concluded. There are lots debug possibilities that could be used to analyse the problem.

Also the Java version seems to be quite old. Did you have the chance to try a later Java Plugin?

Just give me a ping via Email so that I have your contacts.

Cheers,
Matt

[Former Member · Accepted solution]

I am surprised to hear you suggest the use of a recent Java version. In my experience, the applets (iGrids, etc) generated by MII 12.2.4 are unsigned and therefore way too problematic to be used by more recent versions of Java with added security restrictions. That is why, when our consultants led the development of our MII application 2 years ago, they suggested the use of Java 6 Update 18 specifically for that reason.

If I am wrong about any of this, please let me know.

I'm not seeing your email address. If you want to email me, my address is X at thomsonreuters.com, where X = gregory.boettcher.

[salvatore_castro · Accepted solution]

The applets are now signed properly and should work for all JRE environments.

Sam

[Former Member · Accepted solution]

Hi Sam,

nice to hear!

Cheers,
Matt

[Former Member · Accepted solution]

Sam, how can we take advantage of this?

As the latest version of MII seems to be 15.0, I imagine we have to upgrade to that, correct?

Do we need to change our <applet> references in our existing HTML? If so, can you point to any documentation?

Thanks for the reply, this is good news.

Greg

[salvatore_castro · Accepted solution]

The signing of the applets is outlined here in this SAP Note 1955567 and 1941458 both show the update is backported all of the way back to 12.1

The SAP Note 1927805 also has a specific fix for 12.2 SP7

Hope this helps,

Sam

[salvatore_castro · Accepted solution]

As for upgrading to HTML5 there is a completely different way that you define objects on the page as per the HTML 5 spec.  Everything is managed on the page as a <div> object that you define objects for using JavaScript and tell them to display in the various <div> objects that you've defined.  In MII 15.0 there is also a drag and drop builder environment that will allow you to quickly define a web page and also see how the objects are defined.  This will greatly improve your ability to construct pages and also migrate over to HTML5.

Sam