HANA HLM fails

[marcus_bhme]

Hi all,

I'm running in an annoying error during upgrading my HANA from 68 to 70.

The HLM troughts the following:

Error Message: Execution of 'Deploys SAP Host Agent configurations using adm credentials' failed.

Detailed Message: Could not deploy Host Agent configurations [/usr/sap/hlm_bootstraps/SID/HLM/operations.d/chmod-file.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-studio.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-client-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-studio.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/change-newdb-studio-repository-installation-path-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/newdb-studio-repository-postupdate-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-client.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-studio-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-sedm-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/find-java-executable.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/updatehostagent-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/find-hdbeuspack.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-client-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-load-controller.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-hdb-plugin-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/exists-path.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-hostagent-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-hdb-plugin.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-load-controller-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-client.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/newdb-studio-repository-postupdate.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-hdbstudio-deliveryunit.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-studio-repository-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-studio.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/change-newdb-client-installation-path-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-server.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-load-controller-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/updatehostagent.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/exists-path-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-studio-repository.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-load-controller.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/find-java-home-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-client-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-studio-repository-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-studio-repository-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-sedm.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/find-java-home.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-installation-number.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-server-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-hdb-plugin.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-studio-repository.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-server-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-installation-number-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-server.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-client.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/find-java-executable-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-hdb-plugin-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-sda-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-sda-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-hostagent.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/install-newdb-studio-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/detect-newdb-studio-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-client-v2.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/gen-hdbstudio-deliveryunit.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/uninstall-newdb-client.conf, /usr/sap/hlm_bootstraps/SID/HLM/operations.d/update-newdb-studio-repository.conf] on host <servername> with user name SIDadm and signature /usr/sap/hlm_bootstraps/SID/HLM/SIGNATURE.SMF.

Verification of manifest /usr/sap/hostctrl/work/SIGNATURE.SMF failed.

Help: http://help.sap.com/hana

I already tried to upgrade HLM via Studio (when it works once) from 1.0.6.1 to the latest version 1.0.7.2 - which if I believe the value on the upper right corner in the HLM window its 1.7.2 now - but an error occured during the setup.

If I try to upgrade manually I got

/sapmnt/SID/HLM_template/hdbinst

SAP HANA Lifecycle Manager installation kit detected.

SAP HANA Database Installation Manager - SAP HANA HLM Installation 1.50.00.000000
*********************************************************************************

Options:
  SAP HANA system ID | Description
  ---------------------------------------------------------
  SID                | SAP HANA Database SID 1.00.68.384084
Enter SAP HANA system ID [SID]:
Root user password (SSH connectivity):
Root user SSH key passphrase (SSH connectivity):
Checking installation...
Preparing package "SAP HANA lifecycle manager"...
Installing SAP HANA Lifecycle Manager to /sapmnt/SID/HLM...
Installing package 'SAP HANA lifecycle manager' ...

saphostexec is already running (pid=146884). Stopping...-> Start /usr/sap/hostctrl/exe/saphostexec pf=/usr/sap/hostctrl/exe/host_profile <-
start hostcontrol using profile /usr/sap/hostctrl/exe/host_profile
saphostexec stopped
10:33:00 28.01.2014     LOG: Using PerfDir (DIR_PERF) = /usr/sap/tmp
  Installation failed
    error installing
      Cannot install
        <servername>: Executing post-installation checks failed. Installation will not proceed.
        Details: /sapmnt/SID/trans/hlm_install_1390901573/<servername>/checks/IsRunningCheck.sh failed to execute Details:

IsRunningCheck.sh: SAP host agent is not up and running.

Log file written to '/var/tmp/hdb_HLM_2014-01-28_10.32.42/hdbinst_HLM.log'.

I also upgraded the SAP Host Agent to the latest version without any error messages but the same errors occurs.

I'm realy running out of ideas. Does anyone solved a similar situation on the own server or has an idea how to get rid of the issue?

Thanks for comments.

Marcus

[HayBouten · Accepted solution]

Hi Marcus,

I had the same problem. What worked for me was I un-installed HLM and then re-installed HLM from the installation DVD.

1. Stop HLM by running /usr/sap/hlm_bootstraps/<SID>/HLM/stop-hlm.sh
2. Check that all HLM processes are stopped: ps -ef | grep java
3. rename /usr/sap/hlm_bootstraps/<SID> to /usr/sap/hlm_bootstraps/<SID>.old
4. rename /hana/shared/<SID>/HLM to /hana/shared/<SID>/HLM.old
5. Download HLM SPS7 patch 0 from the service.sap.com. Unpack it in a temp. subdirectory.
6. As root user run ./hdbinst to install HLM again.

Now HLM should work again. Test it with a HTML5 compatible browser via the url: https://<FQDN>:1129/lsml/HLM/<SID>/ui/?sid=<SID>

[marcus_bhme · Accepted solution]

Hi Hay,

I had already scripted that refresh of HLM with removing and re-installing the HLM.

The most important hint here was "HLM SPS7 patch 0". I used the latest patch directly which failed all the time. Now even the installation of HLM went fine without any additional parameters to skip anything.

Thanks to Hay and Sunny!

Regards,

Marcus

[former_member194780 · Accepted solution]

Hi Hay,

I tried exact the same procedure you have mentioned.. HLM was successfully installed as per logs

The HLM folder doesnt get created in /hana/shared/ but in /usr/sap/<sid>/HLM ...

I tried accessing the URL mentioned but it says "Webpage not available, unable to load" .

Please help

-Avinash

[marcus_bhme · Accepted solution]

Hi Avinash,

The HLM webpage is kind of outdated. To perform upgrades use the hdblcm from commandline which worked out for me without any trouble anymore... (instead of the instable web-based version).

So to upgrade your system

1) go to the SAP Market Place and download the latest/required HANA Installation

2) upload to the server (if not downloaded directly) and extract... foldername should look like 51049967 (SP9 Rev 97)

3) change directory (in context of root ) into 51049967/DATA_UNITS/HDB_LCM_LINUX_X86_64

4) start ./hdblcm

Follow the instructions on the screen and the upgrade will work smoother.

Or you use the webinterface but here you need to grant an application role (adhoc the name not in mind) to a HANA user to connect with and the files needs to be uploaded to the server as well.

Link: https://<serverip>:1129/lmsl/HDBLCM/<SID>/index.html

Username: <sid>adm

hope this helps as well.

regards,

Marcus

[former_member194780 · Accepted solution]

Thanks alot Marcus

I have successfully installed HLM and it works from Studio.. I had to check the ports.. As you said HLM is now deprecated..

My current AWS HANA version is SP8..I am planning for an upgrade to SP10.. As you mentioned by you I checked for the hdblcm in my SP8 system but I am unable to find it.. Should I install hdblcm separately like how HLM was installed?

-Avinash

[marcus_bhme · Accepted solution]

Hi Avinash,

not sure where it's located on your current system but if you are going to SP10 and you're following the mentioned steps the HLM is part of the installation package as mentioned in step 3). So every upgrade sources provides there latest HLM version.

good luck!

Marcus

[HayBouten · Accepted solution]

You say you use SPS08, so that means that the old HLM tool isn't to be used anymore, because as of SPS08 the SAP HANA Lifecycle Management tools (hdblcm or hdblcmgui) should be used. As far as I know the Lifecycle manager (HLM) has in SPS08 reduced functionality.

The resident HDBLCM tool can be found in /hana/shared/<SID>/hdblcm

I do not know what sort of system administration you are allowed to perform on the AWS instances, so that is maybe the reason the hdblcm is missing.

-- Hay

[antonio_nunes1 · Accepted solution]

Hi Avinash,

I had what I think was a similar issue. After migrating an ERP system to HANA SPS09 the resident hdblcm was missing.

I run hdblcm manually as root from the installation media to make an SPS update and it also installed the (updated) resident hdblcm.

If an update is applicable to you this could solve the missing resident tools issue.

Best regards

Antonio Nunes

[sunny_pahuja2]

Hello,

Your first error is known error. Please refer to SAP note 1955642. For 2nd error, Please check whether hostagent is working or not.

Thanks,

Sunny

[marcus_bhme]

Hi Sunny,

I will check the note. Thanks.

Regarding the Host agent...

I see it running

server:~ # ps -ef | grep hostctrl

root      25512  11003  0 11:18 pts/3    00:00:00 grep hostctrl

root      27188      1  0 10:32 ?        00:00:00 /usr/sap/hostctrl/exe/saphostexec pf=/usr/sap/hostctrl/exe/host_profile

sapadm    27209      1  0 10:32 ?        00:00:00 /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile -D

root      27237      1  0 10:33 ?        00:00:03 /usr/sap/hostctrl/exe/saposcol -l -w60 pf=/usr/sap/hostctrl/exe/host_profile

server:~ # /usr/sap/hostctrl/exe/saphostexec -status

saphostexec running (pid = 27188)

sapstartsrv running (pid = 27209)

11:18:04 28.01.2014     LOG: Using PerfDir (DIR_PERF) = /usr/sap/tmp

saposcol running (pid = 27237)

Anything else to check if it works for real and if the config is right?

Regards,

Marcus

[sunny_pahuja2]

Could you please paste hdbinst_HLM.log?

Thanks,

Sunny

[marcus_bhme]

Of cause... here the last lines. If the full log is required let me know.

10:32:56.918 - INFO:       <server>: Execution of '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/HLMInstallationDirOwnership.sh' finished successfully. Installation will proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/HLMInstallationDirOwnership.sh executed successfully
10:33:00.025 - INFO:       <server>: Executing '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/RestartHostAgent.sh' finished successfully. Installation will proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/RestartHostAgent.sh executed successfully
10:33:00.025 - INFO:       <server>: Execution of '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/RestartHostAgent.sh' finished successfully. Installation will proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/exec/RestartHostAgent.sh executed successfully
10:33:00.025 - INFO:       <server>: Applying post-installation executable configurations finished successfully. Installation will proceed.
        Details: Execution of executable artifacts was successful.
10:33:00.034 - INFO:       <server>: Executing 'cp -f /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/HLM.lmsldesc /usr/sap/hostctrl/exe/descriptors.d' finished successfully. Installation will proceed.
        Details: chmod -R 644 /usr/sap/hostctrl/exe/descriptors.d/HLM.lmsldesc executed successfully
10:33:00.034 - INFO:       <server>: Replacing/Creating '/usr/sap/hostctrl/exe/descriptors.d/HLM.lmsldesc' with '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/HLM.lmsldesc' finished successfully. Installation will proceed.
        Details: The non-executable artifact '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/HLM.lmsldesc' was applied successfully as '/usr/sap/hostctrl/exe/descriptors.d/HLM.lmsldesc'
10:33:00.046 - INFO:       <server>: Executing 'cp -f /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/StartHLM.conf /usr/sap/hostctrl/exe/operations.d' finished successfully. Installation will proceed.
        Details: chmod -R 600 /usr/sap/hostctrl/exe/operations.d/StartHLM.conf executed successfully
10:33:00.046 - INFO:       <server>: Replacing/Creating '/usr/sap/hostctrl/exe/operations.d/StartHLM.conf' with '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/StartHLM.conf' finished successfully. Installation will proceed.
        Details: The non-executable artifact '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/StartHLM.conf' was applied successfully as '/usr/sap/hostctrl/exe/operations.d/StartHLM.conf'
10:33:00.059 - INFO:       <server>: Executing 'cp -f /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/list-hlm-bootstraps.conf /usr/sap/hostctrl/exe/operations.d' finished successfully. Installation will proceed.
        Details: chmod -R 600 /usr/sap/hostctrl/exe/operations.d/list-hlm-bootstraps.conf executed successfully
10:33:00.059 - INFO:       <server>: Replacing/Creating '/usr/sap/hostctrl/exe/operations.d/list-hlm-bootstraps.conf' with '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/list-hlm-bootstraps.conf' finished successfully. Installation will proceed.
        Details: The non-executable artifact '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/list-hlm-bootstraps.conf' was applied successfully as '/usr/sap/hostctrl/exe/operations.d/list-hlm-bootstraps.conf'
10:33:00.069 - INFO:       <server>: Executing 'cp -f /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/hlm_connectivity_<SID> /usr/sap/hlm_bootstraps' finished successfully. Installation will proceed.
        Details: chmod -R 664 /usr/sap/hlm_bootstraps/hlm_connectivity_<SID> executed successfully
10:33:00.069 - INFO:       <server>: Replacing/Creating '/usr/sap/hlm_bootstraps/hlm_connectivity_<SID>' with '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/hlm_connectivity_<SID>' finished successfully. Installation will proceed.
        Details: The non-executable artifact '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/configurations/conf/hlm_connectivity_<SID>' was applied successfully as '/usr/sap/hlm_bootstraps/hlm_connectivity_<SID>'
10:33:00.070 - INFO:       <server>: Applying post-installation non-executable configurations finished successfully. Installation will proceed.
        Details: Non-executable artifacts applied successfully.
10:33:00.070 - INFO:       All post-installation configurations were applied successfully on all hosts. finished successfully. Installation will proceed.
        Details: ok
10:33:00.148 - ERR :       <server>: Executing '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/checks/IsRunningCheck.sh' failed. Installation will not proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/checks/IsRunningCheck.sh failed to execute Details:

IsRunningCheck.sh: SAP host agent is not up and running.

10:33:00.148 - ERR :       <server>: Execution of '/sapmnt/<SID>/trans/hlm_install_1390901573/<server>/checks/IsRunningCheck.sh' failed. Installation will not proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/checks/IsRunningCheck.sh failed to execute Details:

IsRunningCheck.sh: SAP host agent is not up and running.

10:33:00.148 - ERR :       <server>: Executing post-installation checks failed. Installation will not proceed.
        Details: /sapmnt/<SID>/trans/hlm_install_1390901573/<server>/checks/IsRunningCheck.sh failed to execute Details:

IsRunningCheck.sh: SAP host agent is not up and running.

[sunny_pahuja2]

It looks like installer is not able to restart the host agent. You can skip restart of host agent by using skip_host_agent_restart with installer.

Thanks,

Sunny

[marcus_bhme]

Hi Sunny,

the installation looks fine now (no error messages occur).

After the installation I got the string "unknown" where usually the version number is on the upper right corner but after rebooting the server I have the 1.7.2 there again.

But sadly the first error msg still occurs even after the SAP Host Agent is upgraded to the latest version.

Regards,

Marcus

[sunny_pahuja2]

Hello,

Have you referred to the note that I shared with you earlier?

Thanks,

Sunny

[marcus_bhme]

Hi Sunny,

Yes I did it... I upgraded the SAP HOST Agent as well as the HLM (with the ignore skip_host_agent_restart switch). After installing both I restarted the Host Agend... still not working.

I also tried to reboot the whole system to be sure no old process stucked. Also still not working and getting the first error msg with

Error Message: Execution of 'Deploys SAP Host Agent configurations using adm credentials' failed.

Very annoying