on 05-19-2015 11:40 AM
Hello,
lately we have deployed the hostagent to all of our sap servers. Now I have some messages at /var/log/syslog/all at several (not all) Servers like
May 19 11:57:10 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=2000 euid=0 tty= ruser= rhost= user=sapadm
May 19 11:57:12 hges2022 sapuxuserchk: _rebind_proc
May 19 11:57:12 hges2022 sapuxuserchk: _rebind_proc
May 19 11:57:12 hges2022 sapuxuserchk: _rebind_proc
This appears several time per minute.
I have already found note http://service.sap.com/sap/support/notes/927637. I have installed the latest hostagent (207). I have removed sapususerchk from the direcories /usr/sap/<SID>/<Instance>/exe.
Im am using new sap kernel versions, for example 742 PL 101.
We are not using PAM-Authentication anywhere.
I can not find any reason, why this message appears at some servers, but not at others.
Do you have any idea what to do, to solve the problem causing this messages?
Best regards
Andreas
Hello,
after I could not solver this Problem here, I have opened a customer message. They have analyzed that wrong credentials of several agents (SOLMAN, SMD, LVM) seem to be the cause of the problems.
The pam was working fine.
Bets Regards
Andreas
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi
Simply run
saproot.sh <SID> as user root from the following exe locations
/usr/sap/SID/<INSTANCE>/exe
/sapmnt/<SID/exe
And the right permissions will be achieved for the file sapuxuserchk
Thanks ,
Manu
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hello,
Have a look with SAP Note 958253 - SUSE LINUX Enterprise Server 10: Installation notes
PAM configuration needed for "sapstartsrv" and "sapcontrol".
SLES 10 SP3 or later:
After having installed the Web Service Interface (which uses "sapstartsrv" and "sapcontrol") the following configuration stepts are required:
The sapstartsrv file within directory /etc/pam. d has to be created and has to contain the following lines:
#%PAM-1.0
auth requisite pam_unix_auth.so nullok
SLES 10, SLES 10 SP1 and SLES 10 SP2:
After having installed the Web Service Interface (which uses "sapstartsrv" and "sapcontrol"), it might not be possible to authenticate a local created user (i.e. a user entered into the "/etc/passwd" and "/etc/shadow" files), if the SLES 10 system was not installed using the "md5" encryption method as default encryption method (see also above under section "Installing SUSE LINUX Enterprise Server 10 (SLES 10)"). This is due to a known limitation in the PAM authentication module "pam_unix_auth.so" used by "sapstartsrv" and "sapcontrol" via the PAM configuration file "/etc/pam.d/sapstartsrv".
This sapstartsrv file within directory /etc/pam. d has to be created and has to contain the following lines:
#%PAM-1.0
auth requisite pam_unix_auth.so nullok
Unfortunately, this PAM module is only capable of authenticating passwords that have been encrypted with either the "DES" or "MD5" encryption method, but not "blowfish", which is the default in SLES 10.
To change the default used encryption method, do as follows:
Start "yast2" as the "root" user.
Select "Security and Users".
Select "Local Security".
If the "Security Settings" has "Customer Settings" selected, click on "Next" button, otherwise click on "Details...".
Under "Password Encryption Method", select either "MD5" (preferred method to make use of) or "DES".
Click on the "Next" button until the "Finish" button appears.
Click on the "Finish" button to save the changes.
In addition, you have to edit the variable "CRYPT_FILES" in the file "/etc/default/passwd", using your favorite editor, setting the assigned value to either "MD5" (preferred) or "DES" (possible, but is less secure). After having changed the default used encryption method as described above, you have to reencrypt the password for the user used by the calls of "sapstartsrv" and "sapcontrol" by issueing the command "passwd <username>" as "root" and entering the wanted password twice.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hello,
Here is a summary of all configurations and tests done and the actual situation:
I have modified /etc/Pam.d/sapstartsrv as descibed in note http://service.sap.com/sap/support/notes/1375863
Now this is the Content:
auth requisite pam_unix_auth.so nullok #set_secrpc
auth sufficient pam_unix2.so
auth required pam_unix_auth.so
account sufficient pam_unix2.so
account required pam_unix_acct.so
In the installation note of SLES 11 (http://service.sap.com/sap/support/notes/1310037) is described to configure compatible mode to Linux Kernel 2.6 for old sap kernels. We are far above (740 PL 101) the required kernel versions displayed in note http://service.sap.com/sap/support/notes/1629558
Further the Parameter service/protectedwebmethods is set to SDEFAULT as described in note http://service.sap.com/sap/support/notes/927637
In /usr/sap/hostctrl/exe/host_profile the following parameter is set:
service/admin_users = daaadm
In /sapmnt/ETC/Profile/DEFAULT.PFL the following parameter is set:
service/admin_users = sapadm
Users (all local):
etcadm: 1868
eqcadm: 1860
sapadm: 2000
I have restartet
sapstartsrv of ETC
sapstartsrv of EQC
DAA whole Instance with sapstartsrv
sap hostagent with sapstartsrv
All sapuxuserchk in instance executabel directories are deleted. The sap kernel should always use the sapuxuserchk from hostagent (/usr/sap/hostctrl/exe/sapuxuserchk).
-rwsr-x--- 1 root sapsys 720100 Apr 28 14:02 sapuxuserchk
saphostagent is of release 720 PL 207.
Checks:
/usr/sap/hostctrl/exe/sapcontrol -prot GSOAP_HTTP -nr 00 -queryuser -function AccessCheck Stop
User? sapadm
Password?
22.05.2015 09:29:04
AccessCheck
OK
Situation in /var/log/syslog/all:
May 21 22:42:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 21 22:42:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 21 22:42:06 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 21 22:42:07 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 21 22:42:08 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 21 22:42:09 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 21 22:42:12 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 21 22:42:12 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 21 22:42:14 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 21 22:42:14 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 21 23:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 00:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 01:05:05 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 02:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 03:05:58 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 04:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 05:05:05 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 06:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 07:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 08:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
May 22 09:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
As you can see:
Sometimes a day multiple messages with differen user= values appear.
Every hour exactly one message for uid=1868 and user=sapadm appears.
I think thats all.
Best Regards
Andreas
Hello,
All sapuxuserchk in instance executabel directories are deleted. The sap kernel should always use the sapuxuserchk from hostagent (/usr/sap/hostctrl/exe/sapuxuserchk).
-rwsr-x--- 1 root sapsys 720100 Apr 28 14:02 sapuxuserchk
Each instance would use its local sapuxuserchk.
Try copying it back to each local "exe" folder, of each instance. You would also have to set the permissions for each of them as you did for the hostagent folder.
-rwsr-x--- 1 root sapsys 720100 Apr 28 14:02 sapuxuserchk
Regards,
Isaías
Hi Andreas,
For me the issue could be as because of sshd_config file.From the same OS file
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
#UsePAM no
Any changes under the file could be the possible reasons of getting messages sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; under /var/log/...... location.
Regards,
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hello,
something new to think about.
This issue appears at productive system but not at quality assurance system and I am using same configurations and patches on LINUX, SAP Kernel, SAP Parameters service/admin_users and service/protectedwebmethods
I am still searching for differences of the Systems.
Best Regards
Andreas
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Andreas,
Because of the error message "sapuxuserchk: _rebind_proc" I assume you are using LDAP-authentication?
saphostagent installs the PAM config file /etc/pam.d/sapstartsrv which uses pam_unix_auth.so as default.
I assume you need to modify it or add LDAP authentication there too.
Helge
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Could you check with SAP Note 1063897 - sapstartsrv user authentication on HP-UX
Regards,
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
hi andreas,
Do check the oss note 1563660 - sapcontrol, user authorization issues (SUM).
This note is for SUM but can be helpful in your situation.
Also do verify /usr/sap/hostctrl/exe/host_profile add parameter service/admin_users' where in you can maintain the user ids for sidadm,sapadm.
Restart you saphostcontrol and the sapstartsrv.
Which can resolve the issue.
Regards,
Ram
Hi Andreas,
Apart from this you may want to refer the OSs note 1310037 - SUSE LINUX Enterprise Server 11: Installation notes
Refer point regarding uname26.conf as below:-
Configuration procedure for new SAP installations:
Simply start the SAP installation program sapinst with a prepended uname26 command.
For example: /usr/bin/uname26 ./sapinst
This call provides the Kernel 2.6 compatibility environment for the sapinst process and all of its child processes. Since the first SAP application server start is performed by a child process of sapinst, all SAP application server processes inherit the compatibility environment.
After the installation has been finished, either patch the SAP kernel to a version that is compatible with the Linux Kernel 3.0 (see SAP note #1629558) or continue with the configuration procedure for existing SAP systems.
Configuration procedure for existing SAP systems:
1. Make sure that all SAP related programs (SAP instances & sapstartsrv) are stopped
2. Make a list of all SAP users (Unix) on the system, that may start or initiate the start of a SAP program. These are typically all <sid>adm users.
3. Create a file under /etc/security called uname26.conf and enter all <sid>adm users (one user per line), e. g.
--- <snip> ---
nv1adm
nv2adm
nv3adm
--- <snap> ---
This file is read by the PAM library and provides Kernel 2.6 compatibility for user environments.
4. Backup and edit the SAP Initscript used for the automatic start of SAP instances /etc/init.d/sapinit. This step is NOT needed, if you do not have installed SAP host agent.
a) Replace the first line of the script
#!/bin/sh
with the line
#!/usr/bin/uname26 /bin/sh
b) Save the file. All child processes of uname26 (the shell used by sapinit and all of it childs) inherit the 2.6 compatibility environment.
- 5. Start the SAP instances and verify that all SAP programs have been started without errors
Let us know if this helped.
Regards,
Ram
Hello Ram,
thanks for this hint. I had already added daaadm to the host_profile and the sapadm to all DEFAULT-Profiles of the SAP Systems.
I have another sever with same entries in host_profile and DEFAULT-Profiles, where this configuration works fine and I do not get any messages.
I have restartet the hostagent, the DAA-Instance and the sapstartsrv of all sap Systems.
The Problem is not solved.
Regards
Andreas
Hello Ram,
the issue is not solved.
I am trying to understand the usage of Parameter service/admin_users.
- I think, I have to add user sapadm to any sap instance to control these instances through the hostagent.
- Further I have to add user daaadm to host_profile to control the hostagent by SAP Solution manager.
- To add the <sid>adm of all SAP Systems to the host_profile would allow them to control the host agent. So I think it is not nessessary.
Did I understand it right?
Best Regards
Andreas
Hi Andreas,
Yes your right, As the sapadm will allow SAP SLtools to get the status of the systems.
And as you have mentioned DAAADM/sidadm will allow to control the hostagent/smd agents to be checked and controlled.
And by sidadm, you can get the statuses, control over sapcontrol processing.
So will say give a try by adding up this user ids and do remove all sapstream file from /tmp directory before restarting it.
regards,
ram
Hello Ram,
I have seen, that the number of records in /var/log/syslog/all are less than yesterday. Now they appear only once an hour.
Next appearance should be in 5 minutes.
I have added <sid>adm to host_profile and have restartet the hostagent and the sapstartsrv. Let us see what happens.
Best Regards
Andreas
Hello Ram,
the message appeared again. But it is only one per hour.
Do you have any idea of the paramters of sapuxuserchk in the message?
... logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
uid 1868 is the uid of a <sid>adm.
Which information are uid and user?
Is one of it the calling user and the other the target?
Best regards
Andreas
Hello Ram,
all information I get are posted at the first post, but here again:
May 20 07:51:49 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 20 07:52:00 hges2022 sapuxuserchk: _rebind_proc
May 20 07:52:03 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
May 20 09:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
uid of etcadm: 1868
uid of eqcadm: 1860
uid of sapadm: 2000
There are no more information in the syslog despite many _rebind_proc entries.
At 7:51 CET, the same time like yesterday, many entries have appeared.
The entries with user=sapadm only appear every hour 5 minutes past full hour.
Best regards
Andreas
Hi Andreas,
Do you have the sapstartsrv process running with ectadm and eqcadm. if yes try to restart those.
Also can you please check below things in your environment and the other server if everything looks same.
1. Search for sapuxuserchk. Note: this search may take several minutes until it finishes and log will be created. Open the log and check the
permission.
csh -c "date;echo START;find / -name sapuxuserchk -exec ls -al {} \;|& grep -i sapuxuserchk;echo END;date" > sapuxuserchk.log
2. Set owner and permission of every sapuxuserchk found by the above command according to SAP Note 927637; you need to login with root
user to do so:
chown root:sapsys <path>/sapuxuserchk
chmod u+s,o-rwx <path>/sapuxuserchk or alternatively chmod 4750 <path>/sapuxuserchk
3. Afterwards check if every sapuxuserchk belongs to user root group sapsys with permission -rwsr-x---
Use the command:
find / -name sapuxuserchk -exec ls -al {} \; | & grep -i sapuxuserchk
4. Test if the authorization check is OK now:
sapcontrol -nr <inst. nr> -host <hostname> -user <sid>adm <password> -function AccessCheck <Webmethod>
<Webmethod> is the method what failed previously, e.g. OSExecute, Start, Stop. You can find it the corresponding log of EhP, Upgrade, ...
AccessCheck has to come back with following result:
AccessCheck
OK
Let us know the details.
Regards,
Ram
Hello Ram,
I have restarted the sapstartsrv after every change I did.
I have a sapuxuserchk in /usr/sap/hostctrl/exe with root:sapsys and 4750
I have a sapuxuserchk in every /sapmnt/<SID>/exe Directory with sidadm:sapsys and 755
I have no sapuxuserchk in /usr/sap/<SID>/DVEBMGSxx/exe
There are no other sapuxuserchk. As described with sentence
"As of 640 patch level 392, 700 patch level 330, 701 patch level 170, 710 patch
level 262, 711 patch level 149, 720 patch level 113, 800 patch level 46, 802
patch level 24, and 803 patch level 2, sapstartsrv also searches for
/usr/sap/hostctrl/exe/sapuxuserchk with an s-bit configuration. In many cases
(if an SAP Host Agent is installed), this renders the s-bit configuration
described above unnecessary, because the SAP Host Agent installation
automatically performs an s-bit configuration of
/usr/sap/hostctrl/exe/sapuxuserchk."
in note 927637.
Best Regards
Andreas
Hello Ram,
I did the check several times before with success. Now I have repeted it in several kombinations:
successful with user
sapadm for all other instances
etcadm for SAP System ETC
eqcadm for SAP System EQC
daaadm for DAA
not successful with user
etcadm for EQC and DAA
eqcadm for ETC and DAA
daaadm for ETC and EQC
The failed tests are not in /var/log/syslog/all.
When I test for Instance 99 (hostagent) I get the error "FAIL: Webservice port type not enabled". I do not think that this should succeed.
Best Regards
Andreas
Hi Ram/Andreas,
May 20 07:51:49 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 20 07:52:00 hges2022 sapuxuserchk: _rebind_proc
May 20 07:52:03 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1860 euid=0 tty= ruser= rhost= user=eqcadm
After analysis of this thread i reached on following suggestions from the content at
Unixlore.net - Linux and Unix Commandline tips, hacks and howtos
Suggest you to follow the mentioned link,for me the main suspect here is
tty= ruser= rhost=
pointing remote connections,if possible share OS file sshd_config from location /etc/ssh or check the value for PermitRootLogin,if it's yes & enabled ,then try to comment the row with a soft restart by using command /etc/init.d/sshd restart
Please correct me if 'm wrong anywhere .
Hope this will help you.
Good luck !!
Hello,
I interprete the part tty= ruser= rhost= that the login is from local server. Because of this, ssh and sshd configuration should not be relevant. Remote Root Login is disabled at all of our servers.
Do you know the meaning of uid=1860 and user=eqcadm?
Does the user 1860 tries to call something from user eqcadm or vice versa?
Regards
Andreas
Hello,
I know uid means user ID but at this appearance with the message in syslog/all and the usage of sapuxuser? I had several differnt appearances:
May 20 07:51:49 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=etcadm
May 20 09:05:04 hges2022 sapuxuserchk: pam_unix_auth(sapstartsrv:auth): authentication failure; logname= uid=1868 euid=0 tty= ruser= rhost= user=sapadm
At both appearances uid =1868 but user differs. So I want to know which meaning uid and user have in this special situation.
For excample uid is the "uid" of the calling and "user" is the target. So it would mean the user with uid 1868 tries to login to user sapadm.
I just want to understand the meaning of the output.
the sshd_config is similiar at all of our SAP Servers. We are monitoring this.
Best regards
Andreas
Hi andreas,
Yes the sapuxusercheck is using pam_unix_auth mechanism using UID=1868 ( user name sapadm) for this for local login.
Also for the resolution of this request you to check the pam configuration as mentioned by Gaurav in later replys and in OSs note 958253 - SUSE LINUX Enterprise Server 10: Installation notes
OSs note 1310037 - SUSE LINUX Enterprise Server 11: Installation notes
Depending up on your OS version.
as per that pam.d needs to be recreated for the sapstartsrv.
request you to go through the gaurav reply and the oss notes and update us. Correct me if i am wrong and its good learning to understand the sapsuxcheck and hostagent mechanism on os level.
Regards,
Ram
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.