LiveCache Hot Standby Configuration

[Former Member]

Hello.

Currently we are analyzing possibility for LiveCache Hot Standby configuration implementation. Weu2019ve read various documents about Hot Standby, for example:

. SAP SCM 4.1 HotStandby liveCache on pSeries and IBM TotalStorage

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100677

. Managing Serviceguard Extension for SAP, Chapter 4.

http://docs.hp.com/en/T2803-90011/ch04s01.html

In these documents is stated that one of the main reasons for Hot Standby is the speed of LiveCache recovery. For example, here is except from , p.7:

«A traditional failover scenario must first take over and activate the resources (disk and network), restart the application and rebuild this large memory structure, reload the memory structure with the most critical data (continuing to loading the rest in background), perform any rollback requirements and then any redo actions. All this must be done before liveCache is ready to resume production. This effort can require a time-span from several minutes to several hours depending on the state of the application at crash time. An application level data resynchronization with the SCM system may also be necessary to achieve data consistency between the SCM database and the liveCache after a liveCache recovery.»

Or from :

«It is a key aspect here, that liveCache is an in-memory database technology. While the bare instance restart as such is quick, the reload of the liveCache in-memory content can take a significant amount of time, depending on the size of the liveCache data spaces. Modern Supply Chain Management (SCM) business scenarios can not afford unplanned loss of liveCache functionality for the span of time that it takes to reload the liveCache after failover.»

But when we started to test traditional failover for LiveCache (without Hot Standby), weu2019ve discovered that LiveCache start time is nearly independent from data volume in LiveCache. This is because LiveCache restarts asynchronously, i.e. it restores small amount of data into main memory, starts and continues to restore remaining amount of data into main memory in background mode.

So while some performance loss is expected during such background restore, technically speaking, LiveCache restarts and ready to operate after very small amount of time (less than one minute).

In , it also stated that Hot Standby configuration enables to enforce data consistency on application level when LiveCache is restated. For example, as stated in , p.7: u201CWhile job is running, perform the fail-over where in connection to liveCache1 is broken and liveCache 2 is connected. Following a failover, the data on livecache2 is expected to be identical to that of liveCache1; however, the job is expected to fail since connection to liveCache has been broken.

If the data is consistent, on re-running the job, we expect successful completion of the job which can be verified by looking in the new Application log. The date change is used to help verify that previous data is consistent and a new re-planning job (SNP) actually ran.u201D

Or, from , p.31: "Following the failover which occurred in the middle of an SNP planning job, the liveCache data was reset to a consistent state such that the subsequent rerun of the planning job completed successfully and the output data was correct.".

Unfortunately, in , there is no information about this: how application is expected to perform if there is no Hot Standby? In this case, will inconsistencies arise at the application level? What kind of inconsistencies? Etc.

So, can anyone clarify the questions above, and what can be a reason for Hot Standby implementation?

Regards,

Eugeny.

Edited by: Eugeny Tikhomirov on Jan 26, 2009 6:17 PM

[former_member229109 · Accepted solution]

Hello Eugeny,

Your thread is too big & the questions are lost in copied parts from the Hot Standby documents.

Please post your questions only again, if I missed them.

1) "But when we started to test traditional failover for LiveCache (without Hot Standby), weu2019ve discovered that LiveCache start time is nearly independent from data volume in LiveCache. This is because LiveCache restarts asynchronously, i.e. it restores small amount of data into main memory, starts and continues to restore remaining amount of data into main memory in background mode.

So while some performance loss is expected during such background restore, technically speaking, LiveCache restarts and ready to operate after very small amount of time (less than one minute)."

-> You are using the liveCache instance => you are SAP customer.

Please overview the SAP notes::

820824 FAQ: MaxDB / SAP liveCache technology

869267 FAQ: MaxDB LOG area

952783 FAQ: MaxDB high availability

For SAP liveCache documentation see the SAP note 767598.

And you could read more about "Advantages" and "Disadvantages" at the sections "4. Which solution is preferable in which case?" and

"3. Which mechanisms does MaxDB offer to minimize downtimes?" in the note 952783.

The liveCache restart will be longer if the liveCache was crashed during the high loading time and it's dependent on how much redo/undo logs has to be done < see the SAP note 869267 >.

The liveCache applications will run slow after the liveCache will be restarted online until the data will be cashed to the main memory. It's dependent of the amount liveCache application data you have, the configuration of the database parameters and the OS resources, for example.

2) "Unfortunately, in [1], there is no information about this: how application is expected to perform if there is no Hot Standby? In this case, will inconsistencies arise at the application level? What kind of inconsistencies? Etc."

And as you know in your test the Application job will be canceled and you will need to restart it again.

For example, if you are not using the "Hot Standby" and the liveCache was shutdown during the liveCache application job run, the job will be canceled. After the liveCache restart, I assumed that the data & log volumes were NOT corrupted, it's recommended to run the internal < //om17 > and external inconsistencies < //ccr > checks.

How to check that the data of the planning versions are in a consistent status, please, see the SAP note 425825.

Please run the programs /sapapo/ts_lcm_reorg, /sapapo/ts_lcm_reorg_snp and /sapapo/ts_lcm_cons_check_all to check the consistency of time series data.

It's dependent of what applications you are running on your system and what kind of a disaster you think of.

The SAP customers could go

at service.sap.com/liveCache

-> Best Practices for Solution Management: mySAP SCM

&& at service.sap.com/scm -> Technology

-> Information on SCM 4.1 and prior releases

Information on SCM 4.1 and prior releases you will find here. -> Consistency Checks

3) You could create the ticket to the component 'BC-DB-LVC' to get more details about Hot Standby procedure.

4) At SAP SCM 4.1 HotStandby liveCache on pSeries and IBM TotalStorage

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100677

-> Review the document HotStandbyPoC.SAP.Colgate.pdf

< This document has more details >

Thank you and best regards, Natalia Khlopina

[Former Member · Accepted solution]

Hello Natalia.

Thank you for your response. Of course, weu2019ve already read most information about the subject you are referring to. The problem is this information is too general. For example, itu2019s recommended to run consistency checks after LiveCache restart. But it does not mean that inconsistencies will always occur when LiveCache is restarted (without Hot Standby). Also, is it recommended to run consistency checks after restart when Hot Standby is used? If yes, then there probably are no advantages for Hot Standby in terms of saving time required for these checks during restart.

As you know, Hot Standby configuration has special requirements for hardware (storage system) which make it more expensive than classic failover configuration. So when making a decision about usage of such configuration, the essential advantages must exist. Is it possible to construct example of the situation when Hot Standby will always provide obvious benefits over classic failover in terms of restart time or data consistency? For example, is there a situation when restart time without Hot Standby will always be much more than one minute or when data consistency after restart without Hot Standby will always be broken on system or application level?

Regarding data consistency on application level, such example has supposedly been provided in whitepaper «SAP SCM 4.1 HotStandby liveCache on pSeries and IBM TotalStorage». But, as Iu2019ve said, unfortunately there is no information about how this scenario wouldu2019ve been performed without Hot Standby: would inconsistencies arise, and what kind of inconsistencies?

Best regards, Eugeny.

[former_member229109 · Accepted solution]

Hello Eugeny,

1) Concerning inconsistencies on the SCM system to check => It's the basic question for APO application.

There are notes & documentation available, I gave them to you in my reply above.

2) "But, as Iu2019ve said, unfortunately there is no information about how this scenario wouldu2019ve been

performed without Hot Standby: would inconsistencies arise, and what kind of inconsistencies?"

For example, if you run the DP liveCache application routine & the liveCache was crashed, it's recommended to run

the programs /sapapo/ts_lcm_reorg and /sapapo/ts_lcm_cons_check_all to check the consistency of time series data.

It's dependent of what applications you are running on your system and what kind of a disaster you think of, as you know,

you will select the HA solution and also will have the list of the application inconsistencies checks reports/transactions to run after

the system restart.

As it's written in the documentation:

"A hot standby system does not offer any protection against errors and inconsistencies caused by the user or an application.

To secure yourself against these types of errors, you have to execute regular data and log backups, even in a hot standby system ."

3)"Is it possible to construct example of the situation when Hot Standby will always provide obvious

benefits over classic failover in terms of restart time or data consistency?

For example, is there a situation when restart time without Hot Standby will always be much

more than one minute or when data consistency after restart without Hot Standby

will always be broken on system or application level?"

As you already saw about "Advantages" and "Disadvantages" for HA solutions < Hot Standby is one of them > at the sections

"4. Which solution is preferable in which case?" and

"3. Which mechanisms does MaxDB offer to minimize downtimes?" in the note 952783.

Hot Standby is an integrated solution using a Storage System and automatic failover via a High Availability Cluster, there is implementation by IBM, as you know:

http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100442

And more information on hot standby system available at SAP link :

http://help.sap.com/saphelp_nw70/helpdata/EN/70/57d43fdd561165e10000000a114b1d/frameset.htm

< see "Architecture of a Hot Standby System" & "Behavior of the Hot Standby System when Errors Occur" >

One of the examples :

As you know, the master and standby instances each have their own kernel, cache, a separate MaxDB X Server, DBM Server and so on.

Assume the master server was shutdown due the power shutdown < reasons could be different on that>.

And the SNP planning was running at that time. You have to get the results of SNP planning ASAP =>

Hot Standby solution will help you in this situation.

-> I recommend you to create the OSS ticket at component 'BC-DB-LVC' in case you have interest to implement Hot Standby Solution,

your disaster scenario could be discussed to find best solution for you.

It's also good to know the version of your system, the version of the liveCache & what OS and OS resources you have.

Thank you and best regards, Natalia Khlopina