SAP, OpenSSL, and Heartbleed

[Matt_Fraser]

I'm sure by now everyone has heard more than they wanted to about the latest vulnerability sweeping the Internet.  As far as I can tell, SAP NetWeaver systems don't have any built-in OpenSSL components and shouldn't be vulnerable, but I'm not 100% sure about this.  I'm a little surprised that I can't find any mention of Heartbleed in any of the forums today.

So, can anyone state with authority that I'm correct about this, and NetWeaver (ABAP and/or Java) systems are not inherently vulnerable to this flaw?

Thanks,

Matt

[Former Member]

By default SAP uses SAP Cryptographic Library (or CommonCryptoLib in most recent releases) for SSL so the question becomes does SAP's library share code with the vulnerable TLS implementation in OpenSSL.

[Matt_Fraser]

Yes, thank you, that's a better way of restating my question.

[Former Member]

I have only ever seen customers with internet facing scenarios use openSSL for internal POC (proof of concept) purposes. For real scenarios they have their own CA.

The others are self-signed or http even, if it is just browsing a catalogue without actually ordering anything.

I am not aware that SAP have any special discrimination against openSSL. Why should SAP do that, other than making recommendations which CA to use... (can't see that happening either, other than their own Trust Center..).

Cheers,

Julius

[Matt_Fraser]

So there isn't any code reuse from OpenSSL within sapcryptolib or common-cryptolib, then?  I wasn't thinking so much of the CA scenario, though I understand what you mean by that, but whether as an SSL-protected host there would be any vulnerabilities related to this.

[Former Member]

None that I am aware of. It is just an option.

Probably SAP will make some recommendations (I remember some when TSL got into trouble in 2011, partly caused by a "SAP engineer"), but I can't see them blocking openSSL or changing backward compatibility to cipher suites.

My 2 cents (not authoritative, but I did ping the grand master of encryption in SAP about this thread),

Julius

[Matt_Fraser]

Awesome, thanks for doing that.

[mvoros]

Hi Julius,

I think you are confused here what OpenSSL is. This issue has nothing directly related to CAs except that if you decide to re-issue your certs then you need to contact your CA.

OpenSSL is a library that implements SSL and TLS protocols. There was a bug in one extension called Heartbeat. Hence the name of bug is heartbleed. The bug allowed a malicious attacker to read 64kB of memory per connection. The bug is already fixed in version 1.0.1g. The problem with this issue is that just patching might not be enough.If somebody new about this issue before they could retrieve private keys for any systems that used vulnerable OpenSSL. Hence the safest resolution is to patch all systems that use OpenSSL and then re-issue all certificates. The decision if re-issuing of all certs is necessary is up to every customer.

Regarding original question. I am just speculating here. I doubt that SAP SSL implementation is using code from OpenSSL. Here is a quote from OpenSSL license.

* 2. Redistributions in binary form must reproduce the above copyright

* notice, this list of conditions and the following disclaimer in

* the documentation and/or other materials provided with the

* distribution.

I've never seen this with SAP SSL library. So unless SAP is breaking OpenSSL license we are safe. Another reason why I think we are safe is that this was quite a new extension. SAP is usually slow with rolling out new crypto (which is good). So I doubt that SAP SSL implementation supports this extension. Hence I think that ABAP AS and Java AS are safe but there might be other SAP products that use OpenSSL.

Cheers

[mvoros]

SAP employee discovered that bug. I would not call that "causing the issue" 🙂

Cheers

[Matt_Fraser]

Thanks, Martin.  Yes, this was precisely my question.  I have seen some mention of OpenSSL being credited as one of many third-party products used in ADS, though it wasn't clear in what fashion (since ADS would simply reuse the NetWeaver Java AS' security protocols, no?), so that did make me wonder.  I've also seen elsewhere that the Java AS includes a cryptography suite from IAIK.  So, that made me wonder whether if SAP did reuse some third-party code in sapcryptolib, would the copyright notice for same be buried in the code itself?

At the moment I'm feeling reasonably secure about this (as secure as one ever feels in the face of the unknown, anyway), as if this truly were a problem I'd have expected there to be a lot more chatter here about it, and/or there would have been an identified patch to common-cryptolib or sapcryptolib within the past day or so, which doesn't seem to have been the case.  Perhaps that patch will come out in the next couple days, but I'm thinking you're right, and OpenSSL's mistake isn't repeated here.

Regards,

Matt

[Former Member]

The resultant trouble was caused by something discovered by a SAP engineer, not the problem itself caused by SAP. That is what I meant.

Cheers,

Julius

[mvoros]

I would not rely on lack of chatter here. SAP world is special. Sadly, it's still common to think that SAP security == authorization. But it's getting better. People started asking questions about weak cipher suits. Actually, I guess that there might be some open source tool already published that allows you to test your servers. I would try it against SAP systems and see how it goes.

BTW what is ADS?

Cheers

[mvoros]

Just an update. I found this Test your server for Heartbleed (CVE-2014-0160) . I did execute it against external facing portal that has SAP web dispatcher in front of it. It says that the system is not affected by this issue. My guess is that this extension is not supported by SAP.

Cheers

[Former Member]

I think Matt meant Adobe Document Services.

[Former Member]

I am not aware that SAP has any imbedding of openSSL for ADS. Actually I can hardly imagine that because it is an internal server.

I have only ever encountered openSSL for (internal) POCs.

Lets wait for the guru (I pinged him).

Cheers,

Julius

[mvoros]

Based on this note 1633459 ADS uses OpenSSL. It still might not be vulnerable. I don't think it uses OpenSSL for TLS. I think it uses it as a crypto library for signing PDFs. But again, pure speculation. So I think it's SAP's turn.

Cheers

[Former Member]

Hello,

"I've never seen this with SAP SSL library. So unless SAP is breaking OpenSSL license we are safe"

Well, SAP may not use OpenSSL but it does not mean that OpenSSL is not used by a SAP connection.

In my company, I have an SRM system which is published on the internet with this architecture :

Apache Reverse proxy in DMZ1 --> SAP Web Dispatcher in DMZ2 --> SRM in company internal network.

The Apache reverse proxy does use OpenSSL. I tested the access with  http://filippo.io/Heartbleed and I don't have the vulnerability.

The irony is this is because our OpenSSL is too old to have the bug !

Regards,

Olivier

[allaine_tabilin]

SAP released a statement on the BO

2003582 - How does The Heartbleed Bug (OpenSSL vulnerability) affects SAP BusinessObjects Xi3.1 and ...

... Basically stating that the product is not impacted ...

• SAP BusinessObjects Enterprise is not subject to the heartbleed vulnerability.  In this release, an unaffected version of OpenSSL (version 0.9.8) is leveraged for steamworks integration only. 
• The regular BI Platform services use a RSA implementation of SSL for secure communication, which is also not affected.

Regards,

Allaine

[Matt_Fraser]

Yes, that's right, I meant Adobe Document Services.

[Matt_Fraser]

An even more comprehensive test, for all sorts of SSL configuration issues, not just Heartbleed vulnerability, is at Qualys SSL Labs - Projects / SSL Server Test.  However, neither one of these is easy (or possible, rather) to use for an internal-only system.  The mechanism used by filippio.io might be adaptable for internal use, but I'm not sure how to do that.

[Matt_Fraser]

Thanks.  It's good to see SAP is responding to something about this, although this still doesn't answer the question about sapcryptolib and common-cryptolib used in the core ABAP and J2EE products.  I suspect sapcryptolib is not based on OpenSSL, but I don't really know that.

[MichaelShea]

The Product Security Response team has also posted a statement:

<dead_link_removed_by_moderator>

Message was edited by: Julius von dem Bussche

[Former Member]

Hi Michael,

you linked to an SAP intranet site. Is this statement also available for public?

Regards,

Thomas

[MichaelShea]

Sorry my mistake!

Moderator! Please kill my post!

Looks like they are working on it is all that I can say.

[Former Member]

Allaine,

You left out an important part of Note 2003582, "Default Tomcat provided by SAP with SAP Business Intelligence products  is not affected by this issue, unless customers explicitly enable SSL using APR native tomcat library [emphasis added. It provides some links, then continues], SAP will provide updated Tomcat in the future patches". Patches wouldn't be required if there wasn't a problem.

Also, the Note neglects to mention that APR is the default engine if you enable SSL. See

http://tomcat.apache.org/tomcat-6.0-doc/ssl-howto.html#Edit_the_Tomcat_Configuration_File

E.g., our Tomcat6\conf\server.xml contains these lines:

<!-- APR library loader. Documentation at /docs/apr.html  -->

  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

Regards,

Sean

[Former Member]

Another good case for a SAP webdispatcher inbetween which is hardened and terminates the SSL. Server side SSL in dmz or server zone is not seen. That has always been SAP's recommendation.

If the dispatcher is ok (and is not on expected ssl port number anyway) then script kiddies and backends should be safe for a while...

Cheers,

Julius

[mvoros]

That might not always be feasible. PCI requires end to end encryption when you are are transferring card details. HIPAA might have similar requirements.

Also assuming that web dispatcher implementation of TLS is better than other implementation (e.g. OpenSSL) is wishful thinking unless somebody has done proper review. For example SAP SSL implementation is lacking perfect forward secrecy which would be great to have.

Cheers

[allaine_tabilin]

Hi Sean,

I didn't left out anything when I mentioned that statement (see screenshot).  Please note that at that time the latest version of the note was 3.  SAP had modified it since then and is continuing to make updates.  As you've seen, it's now version 6 and as you've read their statement, it now seems to imply that SAP products are impacted if Tomcat is involved depending on the setup.

Anyway, we'll see more on how this note progress as we'll probably see more updates

[Matt_Fraser]

Most websites today seem to be lacking Perfect () Forward Secrecy, unfortunately, probably because they opt for greater compatibility with older browsers.  I'd like to see that implemented much more widely everywhere.

[Matt_Fraser]

Using a Python script found at heartbleed-masstest/ssltest.py at master · musalbas/heartbleed-masstest · GitHub I was able to test for vulnerability in our internal SRM and ECC systems, as well as our web dispatcher, and all came up clean.  However, the script is not intelligent enough to work with systems that use TCP ports other than 443 for SSL (even if you modify line 125 where it seems to hard-code 443 into the call), and other than our dispatcher our portal systems all use a mix of the SAP defaults of 50101 or 50001, depending on instance number.  I'm no Python expert -- not even a novice -- so I'm not sure why changing line 125 didn't make this work.  Perhaps one of you will have more luck.

Anyway, at this time I feel confident that ABAP systems (and web dispatchers) are not vulnerable.  I still don't have proof positive on J2EE systems yet.  It seems like it would be easier to debug the Python script than to modify a test portal to use TCP 443 instead of 50101, but for me the latter at least is something I know how to do!  It just feels a bit like the proverbial bus to cross the street, if you know what I mean.

[mvoros]

Hi,

I just tested that script and it seems to be working fine. I simply changed port number from 443 to something else and it works. E.g.

s.connect((domain, 8300))

Unfortunately, I do not have access to any Java AS with SSL at this moment.

Cheers

[Matt_Fraser]

Odd.  When I changed the port, it told me my servers had "No SSL," which I knew to be incorrect.  I'll give it another shot.

[Former Member]

Self respecting customers will have a SAP webdispatcher between the internet component and the backend port. I think that is foxing your attempts based on expected https ports.

Frank Koehntopp also blogged about it already, so I assume that SAP feels comfortable for the moment.

The plot thickens... 🙂

Julius

[Former Member]

From a risk perspective, you can still have a lot more "success" with external SAP ITS, partner connections and webdynpros/webguis than heartbleed IMO.

Cheers,

Julius

[koehntopp]

I would like to point out that "testing" scripts like these on servers other than your own might be a legal offense in some countries (I know such laws exist in the UK and germany).

I'm not a lawyer, but I'd rather be careful than sorry. Just sayin...

[Matt_Fraser]

True.  In my case, I am trying to test internal systems for which I am the system administrator.  If there's anyone with a right to test them, it's me.  I have also been involving our internal network security team, although they are not specifically SAP-oriented, but more Windows Server, router/firewall, and general Internet exposure oriented.  However, you make a very good point, and I am not advocating that anyone go out and test other organizations' SAP systems (unless they have contracted you to do so).

[StephanGesell]

Please have a look at this security notification: https://service.sap.com/~sapidb/011000358700000308332014E/

It says: At the current state of investigations we have no indications that SAP NetWeaver and SAP HANA are affected.

[Matt_Fraser]

Thanks, Stephan, that's what we've been looking for!

[Matt_Fraser]

By uncommenting the various 'print' messages in the script, and ultimately using the original script, unmodified, which allows interactive testing with any tcp port of one host at a time, I figured out a bit about why I was getting the erroneous 'No SSL' messages.

The full output when scanning an https dispatcher port on a J2EE server (v 7.01) returns:

Connecting...

Sending Client Hello...

Waiting for Server Hello...

... received message: type = 22, ver = 0302, length = 1782

Unexpected EOF receiving record header - server closed connection

Server closed connection without sending Server Hello.

In the dispatcher default trace a warning message appears:

Cannot get input and output streams from socket.  Connection is not initialized.

[EXCEPTION]

java.io.EOFException: Connection closed by remote host.

     at iaik.security.ssl.Utils.a(Unknown Source)

     ...

So, the dispatcher doesn't seem to handle the initialization handshake of the test script.  That in itself is not an indication of vulnerability or lack thereof.  However, it reinforces that J2EE is using the IAIK suite and not OpenSSL, which reinforces the statement Stephan highlighted currently found at the top of http://service.sap.com/securitynotes, in which SAP states they currently have no indication of being vulnerable in NetWeaver.

I'm going to assume we're ok until and unless we see otherwise in a message from SAP.  Thanks everyone who weighed in on this discussion.

Best regards,

Matt

[john_studdert]

And thanks to you for starting the thread Matt, it's been very informative and helpful. Just as an addendum another relevant blog post on the SAP HANA site goes into a little bit of detail around the vulnerability in OpenSSL versus CommonCryptoLib and states that the latter is not affected by the bug, which I think addresses part of your original question:

Blog: No Heartbleed with SAP HANA | SAP HANA

Access via http (extended application services of SAP HANA) uses CommonCryptoLib from SAP and is therefore not affected by this vulnerability.