SOLMAN_SETUP 6.4 Set Up LMDB (Sync Stuck at 0%)

[former_member245319]

Good Day,

i am in the process of setting up solman.

SLD LMDB SYNC Status: Content Synchronization from SLD to LMDB is not finished (progress 0%)

What i have done: All previous setup steps green, with no errors.

Updated CIM Model & CR Content on Solman to latest:

Model Version (SLD/active):                    1.2.44

SAP CR Content Version (SLD/active):   SAP_CR 11.3 (produced 06/29/2015)

I am using only Solman SLD.

Setup RZ70 on Solman and 3 Satalite systems. Data Collection and Job Scheduling started on all 4 systems with no errors.

RZ70 Config:

Host: Solman IP

Service sapgw00

Rank Source 2000, Target 3000.

On solman i can see that Job "SAP_LMDB_LDB_0000000001" is running and finished without errors.

Job Log:

Job started

Step 001 started (program AI_LMDB_R_SYNC_RUNNER, variant LDB_0000000001, user ID SOLMAN_BTC)

Target namespace active currently locked for content sync by user SOLMAN_BTC

Job finished

From everything i can see and do. All seems OK!

RFC on Satellite Systems: SLD_UC, Connection Test OK!

RFC on Solman System: LMDB_SyncDest1, Connection Test OK!

Also the SAP_LMDB_LDB_0000000001 Job is not showing on any of the Satellite Systems? Not sure if this is correct.

On SLDAgentCandidates page:

SMD View: 4 Agent, Status: Started (Green)

SLD View: 4 Agents, Status: Waiting for Agent Registration.

Any advice you can provide will be appreciated.

Thank you

Regards,

Johann

[former_member245319 · Accepted solution]

Hi Rishav,

I am aware that the job log should show more info. That is why i was curious as to why its not showing.

Nothing has changed since my last post.

I have included everything relevant to this problem in a word doc. You can open the doc from below link.

Dropbox - SOLMAN_SETUP

Thank you so much for all your assistance.

Regards,

Johann

[rishav54 · Accepted solution]

Hi,

Please let me know if you had already checked below note.

1555955 - Troubleshooting for LMDB Performance Issues

Symptom

You are using a SAP Solution Manager system with release 7.1 SP01 or
higher.

You are experiencing long response times when accessing data in the Landscape
Management Database (LMDB) AND a lot of work
processes are accessing table LMDB_P_CLASS in the work process
overview (transaction SM50 or SM66).

Or:

The initial synchronization of landscape information between the System
Landscape Directory (SLD) and the SAP Solution Manager LMDB is hardly making any
progress.

Thanks

Rishav

[former_member245319 · Accepted solution]

Hi Rishav,

Thanx for the feedback. I have actually gone thru that note before.

I also thought that it might be that its working overtime and somethings holding it up like table stats.

Anyway i looked at the note again and made dubbely sure everything is OK.

I don't have loads of work processes being used up by the Sync only one.

Included a doc with reference to that note. Also double checked system parameters everything checks out.

I personally think it's your or scenario

"The initial synchronization of landscape information between the System

Landscape Directory (SLD) and the SAP Solution Manager LMDB is hardly making any

progress."

Not sure why it would take so long and what the hold up would be.

As you can see from all my notes and screenshots everything looks OK.

Do you have any ideas?

PS. the note also makes mention or i saw it somewhere else if you have an Oracle DB you can run the table stats almost every hour or so. been doing that every few hours. . .

PS. please find the latest report on the SAP Note in the drop box folder.

Thank you for all you support.

Regards,

Johann

[Former Member · Accepted solution]

Hi Joshan,

Can you please check and let us know below things.

Is there any ABAP dump entry in ST22 related to LMDB? ]

Did you perform SLD-LMDB connectivity setup after SLD CR content and CIM upgrade? If yes, what was the initial content version of LMDB ?

Can you suspend and reschedule the LMDB synch job.

Regards,

Jaskirat Singh

[former_member245319 · Accepted solution]

Hi Jaskirat,

Thank you for the feedback.

There is no dumps in ST22 and no errors in SM21.

1 Background Job Active:

BGD

3908

Running

Yes

34182

CL_LMDB_CIM_PERS_UTIL=========CP

001

SOLMAN_BTC

Insert

LMDB_P_PQ_NAMES

I performed the LMDB Setup (Step 6.4) after:

• CR & CIM content update.
• RZ70 Job Scheduled on DEV, QAS, PROD & SOLMAN

CR CONTENT:

Before Update:

CIM: 1.6.21

CR Content: SAP_CR 7.6 (produced 11/17/2011)

After update (current):

CIM: 1.6.44

CR_Content: SAP_CR 11.3 (produced 06/29/2015)

I have also Deactivated the Sync and Reactivated.

No change in behavior. Everything is still the same.

All Solution Manager Profiles parameters are correct according to Note: 1582842

No users are locked, i believe all my users have the correct Auth. and i checked the RFC's for RZ70 and for the SLD sync, all connecting without error.

i have also worked thru Note: 1555955

This is an Oracle DB and i have scheduled stats for all tables noted in above Snote.

The only possible thing i could think that might be a problem is on Step 6.3 Agent Candidate Administration:

On the SMD View Tab:

All 4 systems are listed and status: Started (green)

On the SLD View Tab:

All 4 systems are listed and status: Waiting for Agent Registration

This should show connected? if Yes, why is it not establish a connection?

Any info you could provide that will get me past this setup phase will be appreciated.

Thank you

Regards,

Johann

[Former Member · Accepted solution]

Hi Joshan,

Did you face any issue while connecting LMDB-SLD inn 6.4 step?

Also please can you check Snote 1675381,  It may help you.

Regards,

Jaskirat Singh

[former_member245319 · Accepted solution]

Hi All,

Please have a look at the 2 docs i shared earlier. I have also included a SM12 output.

Nothing has changed from when i uploaded the docs.

Dropbox - SOLMAN_SETUP

Also as requested:

@ Jaskirat - If you are talking about when i added the the System for Sync, no connection errors.

As for the background Job. i am not getting any errors of sort.

The Log for the active Job that has now been running for ever:

All Finished Job logs are the same:

The job is still stuck on that one table:

I have deactivated , reactivated the Sync. Checked users (lock, Auth), RFC's for connectivity.

I have executed stats for all tables in note: 1555955

Any assistance is appreciated.

Thank you in advance.

Regards,

Johann

[Former Member · Accepted solution]

Hi Joshan,

Can you perform below things.

Check and verify deadlock in DB01

Suspend all background jobs (BTSTRNS1)

Cancel running LMDB-SLD sync job .

Reschedule suspended jobs (BTCTRNS2)

Check and run LMDB-SLD sync job.  Make sure you remove LMDB related table lock entry from SM12 before executing job.

Regards,

Jaskirat Singh

[former_member245319 · Accepted solution]

Hi Jaskirat,

I actually managed to get it resolve on my own. Pretty much went with what you said. Sync finished very early this morning.

I deactivated the sync killed the jobs deleted locks even restarted twice. Problem wasn't resolved until eventually in one of my many attempts of stopping all the jobs and restarting sync it eventually started syncing. Still not sure what the hold up was, i cant find any reason to why it was hanging.

Just glad its working now.

Thank you everyone for your assistance : Rishav you you have been a great help tx.

Regards,

Johann

[former_member182657]

Hi,

Have a look with SAP doc at Connect LMDB to System Landscape Directory (SLD) - Managing System Landscape Information - SAP Libra...

& check with

Monitoring the Synchronization

as covered by Procedure from the document.In resume if possible share the snap shot of CR & CIM details from the solution manager to overcome my doubts related with default CIM namespace for the sync.( Here i could be wrong but just need to clear my doubts on it ).

Hope this will help you.

Regards,

Gaurav

[rishav54]

Hi,

Can you schedule frequent "Check and update optimizer stats" In DB13 also please check the SM12 locks frequently and take an appropriate action for same.

Regards

Rishav

[rishav54]

Hi,

It seems there are locks getting created in SM12, Please check below

• Transaction SM12 (Select Lock Entries) displays locks.Under Table name, enter LMDB*. During content synchronization, a change in the LMDB can be blocked by another user. If so, the active sync job is cancelled because changes must be processed in the order a in which they occurred in the source system. The next scheduled sync job will continue at the same point, later. Locks will usually have been released in the meantime, and pending changes can be processed. If a lock persists, check the lock entries.

For performance issues with the full synchronization job, see SAP note 1555955.

This job will take some time minimum 8-10 hours.

There are lots of SAP Note implementation available, have you checked

1669645,1766713, 1758666, 1555955, 1681591

Further please find below relative informaiton for your scenario

• Transaction SM37 (Simple Job Selection) displays jobs and logs. Jobs usually run under theSOLMAN_BTC system user. The name of the synchronization job for the target namespace isSAP_LMDB_LDB_<namespace ID> e.g. SAP_LMDB_LDB_0000000001.The job log for the initial, full synchronization contains information like the following:Full Sync: Number of instances to be synced from destination LMDB_SyncDest1: 373920.From this you can extrapolate job duration: A job should write about 10 objects per seconds into the LMDB. For more information about performance, see SAP Note 1594654.

• Transaction SLG1 (Analyze Application Log) display logs.Under Object, enter AI_LMDB to find the application logs related to the LMDB.All relevant information is stored in the job log. Transaction SLG1 is only required for low-level analysis.To increase the detail level of SLG1 logs, start transaction SM30 and maintain the V_LMDB_LOG_CONF view. The log level can be set per user and sub-object in the LMDB. Several log levels are supported, for example:
  • Error (default)
  • No logging
  • Error and Warning
  • Error, Warning and Info

Regards

Rishav

[former_member245319]

HI Rishav,

I did go thru the lock entries and deleted what i believed might have locked the job from running.

I re-initiated RZ70 on all system re-activated the sync and when i checked the Job Log i got an error. Tried 1 or 2 things then decided to delete all locks, delete the jobs and start over. Still no affect.

Its at this point that i realized something else was holding the job back so i figured a SAP restart could not do any harm.

Gave that a go and check the Job Log, its now been running for over 1.463 sec. Still no entries inside log (as seen below). But at least its working. Not running for only 15 sec and finishing.

SLG1 is also showing green.

I will give it a few minutes and check back in. Might still be collecting data.

Background Job (only 2 entries):

• Job started
• Step 001 started (program AI_LMDB_R_SYNC_RUNNER, variant LDB_0000000001, user ID SOLMAN_BTC)

I am aware that the Job runs about 8hours. How long before i see more log info on the Job?

Will let you know how it goes in 30 - 60min.

Thank you for your feedback.

Regards,

Johann

[former_member245319]

Hi,

As stated earlier i have an active Job running now. Not finishing in a meer 5 sec with below log info. The current Job has been running for over 4000sec.

In SM50 i can currently see its busy inserting  data on Table LMDB_P_PQ_NAMES.

On SM37 there are now a few jobs that have finished after the active job with :

Target namespace active currently locked for content sync by user SOLMAN_BTC

Job finished

From all the documentation i have found and that you have provided. It seems to indicate that the active Background Job needs to finish before it will proceed to the next Background Job. Is this correct?

The log on SOLMAN_SETUP is still showing 0% progress.

My question is, is this normal? Should i not have seen some progress in the last hour?

If the Sync was to run say 10 Hours then my progress should have shown 10% by now.

your guidance is appreciated.

Thank you.

Johann

[rishav54]

Hi,

Pleaes can you check the job logs for the active job

Does any job log had information similar to this

Full Sync: Number of instances to be synced from destination LMDB_SyncDest1: 373920

Job log should looks like as below.

Job log overview for job: SAP_LMDB_LDB_0000000916 / 15144401

Date Time Message text Message class Message no. Message type

06.06.2011 15:14:44 Job started       00     516      S
06.06.2011 15:14:44 Step 001 started (program AI_LMDB_R_SYNC_RUNNER, variant LDB_0000000916, user ID SOMMERMAR)       00     550      S
06.06.2011 15:14:44 Content sync started with extended SID generation on LMDB_CS_MESSAGE     017      I
06.06.2011 15:16:42 Update Subscription LMDB_SyncDest1 with CL-Id: 1307366202000005000 LMDB_CS_MESSAGE     022      I
06.06.2011 15:17:13 Full Sync: Number of instances to be synced from destination LMDB_SyncDest1: 805698 LMDB_CS_MESSAGE     014      I
06.06.2011 15:18:09 Full Sync. of SAP_Product: 4145 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:19:38 Full Sync. of SAP_Product finished: 4145 creations, 0 modifications, runtime 145 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:19:38 Full Sync. of SAP_DependencyContext: 3 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:19:38 Full Sync. of SAP_DependencyContext finished: 3 creations, 0 modifications, runtime 0 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:21:10 Full Sync. of SAP_GlobalUniqueID: 16465 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:25:11 Full Sync. of SAP_GlobalUniqueID finished: 16465 creations, 0 modifications, runtime 333 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:25:11 Full Sync. of SAP_AllowedNamespacePrefix: 42 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:25:12 Full Sync. of SAP_AllowedNamespacePrefix finished: 42 creations, 0 modifications, runtime 1 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:25:12 Full Sync. of SAP_CustomAttributes: 3 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:25:12 Full Sync. of SAP_CustomAttributes finished: 3 creations, 0 modifications, runtime 0 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:25:12 Full Sync. of SAP_Supplier: 38 source instances, 0 target instances LMDB_CS_MESSAGE     018      I
06.06.2011 15:25:13 Full Sync. of SAP_Supplier finished: 38 creations, 0 modifications, runtime 1 s LMDB_CS_MESSAGE     019      I
06.06.2011 15:25:20 Full Sync. of SAP_InstalledSoftwareFeature: 1221 source instances, 0 target instances LMDB_CS_MESSAGE     018      I

Also please check the SM12 entries side by side. I guess SOLMAN_BTC user had the required authorization.

Regards

Rishav