ESP Licensing Troubleshooting
The purpose of this page is to give a quick overview of ESP licensing, common ESP licensing errors, and how to troubleshoot them.
SySAM 2.0 licensing is used to license most SAP-Sybase products including Event Stream Processor (ESP). Many of the errors that can be encountered are very common, and can be addressed in a similar manner across all environments.
How Does it Work?
When a customer checks out a license from the Sybase Product Download Center (SPDC) or from SAP SMP, the license should be placed under the installation directory for ESP within the $SYBASE/SYSAM-2_0/licenses directory as this is where the product will check for proper licensing. Use the %SYBASE% variable on Windows instead of the $SYBASE, which is for UNIX and Linux only.
During license check out, the license must be defined as the Served or Un-served model. A Served license will require a SySAM license server to be running at all times in order to serve out licensing to the ESP platform. If the license server is down, it will not be able to be served out to ESP. On the other hand, the Un-served model does not require a license server to be served to the ESP platform. It will act as a flat file that the ESP environment can read and pick up to license the platform immediately. In most situations, it is recommended to choose Un-served to remove the license server as a point of failure for the ESP platform.
ESP will be configured during installation to use a specific license type. By default, this will normally be set to the CPU (CP) license type. If no changes are made during installation, the ESP platform will always request licenses with a CP license type and ignore all other license types. ESP can be reconfigured to check for a different license type by editing a file called esp_license.prop located under $ESP_HOME/sysam (%ESP_HOME% on Windows). Always back the file up first! This file should only be edited by programs that will not add hidden characters such as vi or Notepad as any hidden characters added may corrupt the file. It will contain a line stating LT= meaning the current license type is equal to a specific value. For the default CP license, it will be set to LT=CP. Valid ESP license type values are CP, DT (development and testing), EV (evaluation), and SF (standby CPU).
To determine the license type of the file currently in use, if it has been forgotten, the license file must be examined. All license files will have a .lic extension and should be located under $SYBASE/SYSAM-2_0/licenses if the files were put in the proper place. Normally, the license file will contain a line at the top that has been commented out. This line will state something similar to “CP License for Sybase Event Stream Processor Windows x64 64-bit” indicating the file is a CP license type. If this line has been removed for any reason, the license type can be determined by looking below the INCREMENT line in the file for a line starting with VENDOR_STRING. This line will contain an LT= value similar to the esp_license.prop file to indicate the license type. Do not edit the license file information as this will almost always corrupt the file and make it unusable!
Where is the Log Located?
All ESP projects will create two logs the first time the project runs under the ESP workspace. These logs will be appended with each subsequent run of the project. The workspace location will be selected when creating the project and can be changed optionally. Due to the workspace being located in any place the user chooses, it will be referred to as just workspace in this document.
Once the project is created a folder will be created with the name of the project within the workspace directory that will contain the files comprising the project excluding the log files. On the first run of the project, a new folder will be created under the workspace still using the project name in the format of default.project.0. This new folder will contain basic files to run the project and the two log files called esp_server.log and stdstreams.log.
The esp_server.log file will never contain any meaningful license messages beyond “License check started” and “License check succeeded” or “License check failed.” This log should only be used for troubleshooting non-license-based errors. For license-based errors, examine the stdstreams.log file instead.
Common License Errors
A list of some of the commonly encountered license errors and solutions can be found below. This list does not constitute a complete overview of every possible issue; however, it should assist in resolving most issues. ESP may fail to start with a vague error message when it cannot obtain a license similar to the following:
Error message text -
The Cluster has encountered a project error:
Check the Cluster policies for this project.
Cluster error message:
Invalid action:[FAILURE:Application wait for status is started, but application current status is not started]
The stdstreams.log file should always be consulted to gain a clearer picture of the situation. If there are no licensing errors present, contact support to discuss other possible causes. Otherwise, see the possible errors below that may relate to the issue encountered.
- Graced licensing. The log reports the following messages:
- 2013/05/08 16:36:13 Warning (131254): Checked out graced license for 1 ESP_CORE (2012.1119) will expire Thu May 30 23:17:42 2013.
- 2013/05/08 16:36:13 Error (131302): Failed to obtain license(s) for ESP_CORE feature from license file(s) or server(s).
- 2013/05/08 16:36:13 Warning (131275): WARNING: Sybase Event Stream Processor Engine functionality that requires the ESP_CORE license will be disabled on Thu May 30 23:17:42 2013, unless a suitable ESP_CORE license is obtained before that date.
Cause: No license is available to be checked out by ESP; however, the product is still within its 30 day grace period allowing it to check out a graced license. This grace license is specifically provided to allow users time to obtain a valid license to put in place while still being able to run a functioning product. The warning will not cause ESP to stop working until the graced license expiry date indicated as May 30 2013 in the final message.
Resolution: If any other license error messages exist, refer to those first to resolve those issues as they may resolve the graced license issue. Otherwise, obtain a new functioning license from SPDC, SAP SMP, or contact your sales representative for assistance if no licenses are available.
- Expired licensing. The log reports the following messages:
- 2013/07/01 17:32:56 Error (131302): Failed to obtain license(s) for ESP_CORE feature from license file(s) or server(s).
- Error: the license has expired
Cause: These error messages indicate that the license currently in use is expired and will no longer function. The message can be caused by the grace period for ESP running out because no license was obtained or because the environment was using an evaluation license.
Resolution: Obtain a new functioning license from SPDC, SAP SMP, or contact your sales representative for assistance if no licenses are available.
- Lack of licensing. The log reports the following messages:
- 2013/07/01 18:41:43 Error (131302): Failed to obtain license(s) for ESP_CORE feature from license file(s) or server(s).
- 2013/07/01 18:41:43 Error (131304): Insufficient ESP_CORE CPU licenses. Sybase Event Stream Processor Engine requires 2 ESP_CORE ("CP") CPU licenses for use on this machine but only 1 could be obtained. Check whether additional licenses are available at the Sybase Product Download Center.
Cause: Not enough licenses are currently available for the environment even though 1 proper license was found. The number of licenses available must be equal to the number of CPU cores on the host. This error message states that 2 are required indicating 2 CPU cores are on the host. Checking the number of CPU cores on the box without the error message can be done via $SYBASE/SYSAM-2_0/bin/cpuinfo -v. Refer to the message that is similar to “Detected 4 logical processor(s), 2 core(s), in 1 chip(s)” to determine how many cores are present. In the case of this message, 2 cores are present on this host. Logical processors and chips should be ignored.
Resolution: Generate further CP licenses or licenses of the proper type from the SPDC or SAP SMP to cover all CPU cores on the host. If no further licensing is available, contact your sales representative for further assistance.
- Configured incorrectly. The log reports the following messages:
- 2013/07/01 18:56:35 Error (131292): Licenses exist for Sybase Event Stream Processor Engine, but a license containing the configured (LT=CP) attributes could not be obtained. Verify that Sybase Event Stream Processor Engine is configured to use the correct type of license, and either reconfigure; or generate and deploy the desired license from the Sybase Product Download Center.
Cause: Licenses were found that could be checked out by ESP; however, the product is configured to check out a different license type. According to the error, ESP is currently setup to check out the CP license type.
Resolution: View the current license file under $SYBASE/SYSAM-2_0/licenses to determine what type of license is currently in use by looking at the LT= argument on the VENDOR_STRING line just beneath the INCREMENT line. Reconfigure ESP to check out this license type by editing the esp_license.prop file located at $ESP_HOME/sysam. Always backup the file first! Change the LT= value in the esp_license.prop file to reflect the license file and restart ESP.
- Invalid host. The log reports the following messages:
- 2013/07/01 19:13:21 Error (131081): Invalid host. The hostid of this system does not match the hostid specified in the license file. Unserved licenses can only be used on the host for which the license was generated. If the license is intended for use on this machine then you need to check-in the license and regenerate it using the host id for this machine.
- 2013/07/01 19:13:21 Error (0): The hostid of this system does not match the hostid
- 2013/07/01 19:13:21 Error (0): specified in the license file.
Cause: The license was generated for a different host ID than the host ID of the current host. The license quantity and type can be right; however, the license cannot check out due to being for another host.
Resolution: Check in the current license and check it back out for the proper host ID through the SPDC or SAP SMP. If the host ID for the current host is unknown, it can be acquired via $SYBASE/SYSAM-2_0/bin/lmutil lmhostid.
- Corrupt license. The log reports the following messages:
- 2013/07/01 19:29:26 Error (131080): Invalid (inconsistent) license key.
- 2013/07/01 19:29:26 Error (0): The license key and data for the feature do not match.
Cause: The license has become corrupt somehow. This can be due to editing the license file, selecting the “Download All Licenses for Host” button on the SPDC, copying over the license text into a file and accidentally adding hidden characters, or any number of other possibilities.
Resolution: Download the license file from the SPDC or SAP SMP again to retrieve a clean file without corruption. Avoid copying the text over to a file, editing the license, or selecting “Download All Licenses for Host” on SPDC in the future. The “Download All Licenses for Host” button on SPDC has been known to corrupt licensing on multiple occasions and should never be used under any circumstances.
- License older than product. The log reports the following messages:
- 00:00:00000:00000:2011/10/05 15:23:28.75 kernel SySAM: Application version > License version: 2011.0616 > 2011.01260
Note: This message is taken from ASE; however, it should be similar to what is seen in ESP. Currently no old license available to retrieve a test message.
Cause: Every license has a date based version that will be compared to the build date of ESP. This license date can be found in the form of YYYY.MMDD0 on the INCREMENT line in a license file. An example from a license would look like 2014.08150 meaning the version of the license is August 15 2014 with the 0 dropped as an extraneous number. If ESP’s build date is greater than the date based version in the license, ESP will be unable to check out the license. If the date based version in the license is greater than ESP’s build date, the license will be checked out without any issues.
Resolution: Upgrade the license via the SPDC or SAP SMP. This can be accomplished either by selecting the “Upgrade” button that will appear next to the license or by checking the license in and back out as a new license. If license renewals are not up-to-date, the date based version will not be able to be upgraded. Please contact your sales representative for further assistance if the version cannot be upgraded through either of these methods.
- License server down. The log reports the following messages:
- 2013/07/01 19:59:08 Error (131087): Cannot connect to license server system.
- 2013/07/01 19:59:08 Error (0): The license server manager (lmgrd) has not been started yet,
- 2013/07/01 19:59:08 Error (0): the wrong port@host or license file is being used, or the
- 2013/07/01 19:59:08 Error (0): port or hostname in the license file has been changed.
Cause: A Served mode license is currently being used, and ESP cannot contact the license server. This can be due to the license server not being started, or the port or host of the license server being different than what ESP expects.
Resolution: The license server will need to be started via the command line. License servers should never be started through Windows Services as this will often break the server. To start the license server, run it via $SYBASE/SYSAM-2_0/bin/sysam start. If the license server has moved to a new host or port, edit the license pointer file contained within $SYBASE/SYSAM-2_0/licenses to point to the proper host and port. This file will look identical to a license file in name; however, it only contains information pointing to the host name and port number of the license server.
Refer to the SySAM documentation.