Batch Job finishing without updating the database

[Former Member]

Dear SAP Basis Gurus and SAP Abap Gurus,

I am executing a sap report to create business partner in SAP CRM system. Now i have used parallel processing methodology in this report for business partner creation.

This report works fine with 80% of total dialogue processors in the system(quality/development/testing sap servers) used with multiple batch jobs in parallel (example if total dialogue processes in system is 100 then i schedule this report with 10 batch jobs and 8 dialogue processor per batch job).

But in production system we have 12 application servers with (150 dialogue processors, 20 update 1, 10 update 2 and 30 batch processes) per application server. So total resource is

dialogue processor : 1800

update 1: 240

update 2 : 120

batch jobs available: 360

Now i am scheduling the business partner creation report with 84 batch jobs and assigned 17 dialogue processor per batch job.

But only 7-8 jobs are remaining active after scheduling and updating the database creating customer and rest of the jobs are finished in 1 second without updating anything (note: there is nothing wrong with data creation as we write application log for error while data creation and nothing has come up in application log). Now i reprocesses the rest of the batch jobs which completed in 1 second earlier and didn't update, again 7-8 jobs are remaining active and creating business partner and rest of them are finished in 1 second and so on.

This report is working fine in other sap systems like quality/development/testing, and this problem is only happening in production.

I am not at all familiar with basis configurations, So kindly help me out with this issue.

Thanks for your help

Sudipto

[feng_shi · Accepted solution]

Hi,

Please update the incident with information:

1, SAP_BASIS release and kernel version of production system.

2, How did you assigned  dialogue processor to one batch job exactly ?

3, if you define rdisp/rfc_min_wait_dia_wp parameter and what's the value

Thanks and best regards,

Shi

[Former Member · Accepted solution]

1, SAP_BASIS release and kernel version of production system. – 742 pl 300

2, How did you assigned  dialogue processor to one batch job exactly ?  - This is done programatically, where in one batch job executes the report once with input file and inside the report selection screen we pass dialogue processor as parameter (example 3)we have logic within code to divide the input file records (say 90) across 3 dialogue processor (so 30 records each). And the customer creation function module is called for these 3 dialogue processors.

function used for spawning function in dialogue processor

CALL FUNCTION 'function module for BP create'

       STARTING NEW TASK lv_task

       DESTINATION IN GROUP DEFAULT

       PERFORMING sub_get_log ON END OF TASK

       EXPORTING

         im_commit_count       = fp_commit   "commit freq

       TABLES

         lt_customer           = lt_customer

         lt_customer_bp        = lt_customer_bp

         lt_customer_bp_fail   = lt_customer_bp_f

         lt_partner            = fp_it_gt_partner

       EXCEPTIONS

         communication_failure = 1

         system_failure        = 2

         resource_failure      = 3.

3, if you define rdisp/rfc_min_wait_dia_wp parameter and what's the value - 7

[feng_shi · Accepted solution]

Hi,

For SAP Kernel 7.4x, rdisp/rfc_min_wait_dia_wp is deprecated and the parameter rdisp/scheduler/prio_normal/max_quota and rdisp/scheduler/prio_low/max_quota can be used (SAP note 2001276-Changed configuration as of 7.40 SP2).

The parameter rdisp/scheduler/prio_normal/max_quota limits the maximal number of normal and low priority requests that can be processed in parallel.

you can use RZ11 to check the details about the parameters.

please Remove parameter rdisp/rfc_min_wait_dia_wp from profile, restart the system, Then the rdisp/scheduler/prio_normal/max_quota will take the default value 80%, low and normal requests can take up 80% of the work processes.

(Sync/Async RFC session, HTTP session are Normal priority )

it seems that you are using pRFC, GROUP is DEFAULT, please check if DEFAULT rfc group has enough resouce for processing.

But only 7-8 jobs are remaining active after scheduling and updating the database creating customer and rest of the jobs are finished in 1 second without updating anything 

just in case, please let me confirm, what did you mean "rest of the jobs are finished in 1 second" ??

what is the job status ? cancelled , right ?

if job cancelled immediately, it means that at the time you dispatch a call, none of the application servers in the group has enough resources to execute the RFC.

Thanks and best regards,

Shi

[Former Member · Accepted solution]

Rest of jobs finished in 1 second means...jobs were in finished status but they didnt create any business partner (basically dialogue processor never processed the function module which was executed in parallel processing)

[Former Member · Accepted solution]

Thanks Shi for the inputs.

We tried the configuration you suggested with our basis consultants but now

when i schedule even 1 job with only 2 dialogue processor per batch job, still the jobs are finishing without creating any business partner(updating database).

Same is true when i schedule 84 jobs with 2 dialogue processors per job.

Cheers

Sudipto

[feng_shi · Accepted solution]

Hi Sudipto,

when i schedule even 1 job with only 2 dialogue processor per batch job, 
still the jobs are finishing without creating any business partner(updating database).

if so, the root cause should be at job program but not at system resource configuration.

Thanks and best regards,

Shi

[Former Member · Accepted solution]

Hi Shi,

The business partner creation is working fine in all environments except production as mentioned in issue description. Had it been problem with job program it would fail in all environments, isn't it.

Apart from this i ran the job program in production system in dialogue mode in debugging and the business partner creation is working fine. But when i schedule the same program in batch with 1 batch job and 2 dialogue processor, which is not even 1% of system load(1800 dialogue processors), this program doesn't update anything.

There has to be some basis setting which is missing causing this failure.

Appreciate your help in issue triage.

Cheers

Sudipto

[Former Member · Accepted solution]

Yesterday we did another exercise with our basis consultant:

I scheduled 2 future dated  jobs with 17 dialogue processor each, so that all jobs were in released status. Now the basis consultant manually executed those released jobs on only one application server instance out of 12 available application server instances in production. And the jobs worked fine.

So he said it might be problem with spawning the customer creation report over dialogue processor across multiple application servers.

Can you please help rectify the setting or configuration which can correct this issue of spawning dialogue processors across multiple application server instances in production.

[rejiomusic · Accepted solution]

Hey Sudipto,

two hunches, but may be I am wrong:

1. Ask your BASIS guy to keep the central instance out of these processing and run the same batch job.

2. can you schedule a single job in an instance and then let them increase the number of dialogue processes as per your system capacity, which can run in any instances as needed.

Let me know if this helps?

thanks,

Reji

[Former Member · Accepted solution]

Hi Reji,

Regarding 1) we have no central instance in our system we have distributed architecture where ASCS is on different host and PAS is on different host and there are additional app instances.

Tried 2) and it worked on only one app server but didn't work on rest 10 app servers in production system.

One new thing we have noticed and this might be the root cause of the issue.

We have 11 application servers in our production environment and i tested all 11 application server for their capability to run batch jobs by manually scheduling 2 batch jobs on each server, but to my surprise only one out of 11 worked fine and was able to process the jobs successfully, Rest 10 had same result (jobs finished in 0 seconds without updates).

Which brings us to conclusion that the 10 application servers are not able to process jobs. We did health check in RZ04 for each faulty application server and everything appear fine. Checked in SM61 as well and for all the application servers the four control objects(job starter/zombie cleanup/event controlled scheduler/time-controlled scheduler) are active.

Please guide on how can our basis guys fix the faulty application servers, so that they are able to schedule and process the jobs successfully. I think if all the app servers are able to schedule/process job successfully the issue will be resolved.

Cheers

Sudipto

[rejiomusic · Accepted solution]

Hey Sudipto,

Can you check this discussion. This may help you.

Background Jobs Issue | SCN

regards,

Reji

[rejiomusic · Accepted solution]

Did this help?

[rejiomusic · Accepted solution]

hey Sudipto,

Any luck on this?

Regards,

Reji

[Former Member · Accepted solution]

Hi Reji,

I think it was the problem with the application servers. our basis team rebuilt the whole system and after that when i was testing with a small data volume the issue was not being replicated.

So i am assuming that rebuilding the system must have fixed the app servers capability to run batch jobs. But we're yet to test with a large data volume after the system build.

So will post my observations after that. If issue is fixed i'll close this question.

Thanks for your help and useful guidance.

Hi All,

Many Thanks to the other contributors to this blog as well. It was a great learning experience from you all.

Cheers

Sudipto

[Former Member · Accepted solution]

The Issue is Fixed once the production system was rebuilt and the app servers are now able to schedule jobs.

[yakcinar]

Hello Sudipto,

Can there be lock issues since you are running jobs in parallel?

Does each wp process different parts of the data?

Don't you see any logs in SM21?

Also check in ST11 last modified dev_wx file contents.

There could be limit for some parameters also.

Regards,

Yuksel AKCINAR

[Former Member]

Hi Yuksel,

Thanks for your reply.

Each dialogue processor processes a function module which creates business partner.

we checked the locking issue but it is not as this report works fine in other systems with multiple batch jobs and multiple dialogue processor

No Logs in Sm21 with the migration user which i scheduled the job with or any other user relevant to these jobs

Nothing relevant to the job in ST11 as well

Cheers

Sudipto