HANA - Data Volumes

[Former Member]

Hello Team,

I would like to know how the changes that happens in the HANA memory is saved to the disk. From my understanding the data is loaded from source system like ERP, BW etc to the HANA data volumes and then the data is loaded from the data volumes to the Server Memory.

And whatever changes are happening to the records on the HANA memory is captured at the log volume so incase the the server memory goes down due to power loss etc we can recover the HANA system from the data volumes and also applying the changes saved at the log volumes. The question is are the changes saved to disk as well, if so how does HANA application handle it? Are the changes saved at the log volume applied from the logs to disks or the changes at the Server memory itself flushed after sometime to disk? What is this process called? Savepoint is the procedure by which the changes are saved in the log volumes. Do we have a similar teminology for saving data to disk?

Thanks for the clarification.

Thanks and Regards,

Archana

[former_member93896]

Hello Archana,

your description of the restart process is pretty much correct: Data structures in memory are restored from persistency based on the last savepoint, then we apply changes from redo and undo logs.

However, when load data or changing it using SQL statements, changes are written to a log in memory first. Only at the time of commit, the system flushes the changes out of the queue to persistent storage.

Redo log entries are written to log volumes at latest when a transaction ends and the changed data itself is written at latest when a savepoint is performed.(There are other operations like a delta merge or buffer replacement that might trigger a flush as well, but this goes beyond this posting here).

Regards,

Marc

SAP Customer Solution Adoption (CSA)

[Former Member]

Hi Marc,

With your reply , Can we confirm that HANA transactions are recorded in the below Manner
Memory --> LogVolume --> Save point --> Data Volume .

Here i consider save point just as transaction or time frame , after which log volume Data is flushed to Data volume .

Thanks,

Razal

[lbreddemann]

He Razal

The savepoint is an actual activity performed by a specific set of threads in SAP HANA.

Key here is to see that changed data is stored in two separate ways:

1. The changes to be made are stored in the redo log buffer. Upon commit or when a certain filling degree of the redo log buffer is reached, this information is written to the log volumes.

2. The changes are applied to the tables or more precisely to the data structures that keep the table data.

And that's it concerning any transaction.

Independent from the transaction that actually changed data at some point SAP HANA will write out those parts from the data structures mentioned in step 2 to the data volumes.

As you see, redo log data is never stored or moved into the data volumes.

It's completely different information in fact. Redo log data is always about a change of data, while the data in the data volumes (written out by the savepoint) is always about a specific state of data.

Or simpler yet: the data volumes may contain data in version 1 (think UNDO information) and version 2 (current data) while redo log contains the description how to get from version 1 to version 2.

- Lars

[Former Member]

Thanks Lars ,
That makes the concept clear to me

[Former Member]

Hello  Lars,

From your reply, could you please confirm if my understanding is correct.

Step1 - Whenever a data is changed/loaded to relational engine and as soon as the transaction is complete, the redo log buffer collects info about the changes and keeps it in memory.

Step2 - This redo log buffer is written to log voumes. (does the save point again writes the redo log to log volume?)

Step3 - So when Savepoint is performed, the changed data will be written to data volume and the latest log position will determined.

Could you please clarify how logs are written to log volumes

Thanks... Marikannan

[lbreddemann]

Hey there

The log buffer doesn't collect anything. Just before any data gets changed in the system, this change is logged first by writing this information into the log buffer.

Once the log buffer is full enough or a COMMIT occurs, it is written out to the log volume (it is flushed).

A savepoint doesn't write the same information again to the log volumes. The only thing it does with log volumes is to note that a savepoint occurred at this specific log sequence.

Not really sure what's unclear, so maybe you rephrase your question...

- Lars

[esjewett]

Hi Archana,

My understanding is that all changes are saved to the log volume (a persistent disk or flash memory store) as well as being applied to the in-memory data representation. So at any given time, "log store + disk-based data store = in-memory data". Because of this, HANA can always rebuild the current in-memory data from the log and disk store in the event of a power failure.

There are a couple of ways that log files can be merged into the disk-based data store, but if I am recalling my conversations with the HANA team correctly, HANA uses a strategy that involves taking a snapshot of the in-memory store, saving that to disk, and then truncating the log files from before the snapshot. This keeps the log file at a reasonable size, as we are only storing data since the last snapshot.

Cheers,

Ethan

[Former Member]

Hi Ethan,

Thanks for the explanation, that answers my questions.

Thanks and Regards,

Archana