Message status "DLNG" blocks the whole PI server for message transfer.

[antony_ferminus]

Hello Experts,

I am busy withe the IDOC2JDBC scenario through PI.

Our server is PO 7.4 single stack (Kernal version 7.40.3xxx.3xxx.2xxxxxxxxxx)

I sometimes had some probles withe the message status "DLNG" (Delivering), and after the restart of the communication channel or the JAVA stack, the messages status changed to "To be delivered", or canceled with errors or i canceled them manually, and the message process start working as normal and,  when one create or update happens in the backend ECC, i received the data to the external Data warehouse through JDBC.

Last two days i had 5 messages stucked in the Delivering status, and they blocks the other messages to process, because, i can not cancel them. All the other messages comming now are put to the status "To be delivered", and nothing has been written to the external data ware house. I restarted Com. Channel, JAVA stack even the whole server. The status does not changes.

I can see the log of receiving messages in the IDOC sender communication channel through RWB cc monitoring.

There are no logs in the JDBC receiver channel.

I refered the sap note 1623356 no use.

One article says

On newer versions or SP level of XI/PI, it is possible to release the threads held up by the "Delivering" messages by stopping the corresponding channel. This will require the correct version/SP level as described by OSS note 1604091.

This note is for max version 7.31. It means there sould be a similar solution for PO version 7.4 too.

We are completly blocked at this moment.

Any kind of suggestions are appreciated.

Thanks in advance.

Regards,

Antony.

[antony_ferminus · Accepted solution]

Hi Experts,

I have been reading around and changed some of the JDBC receiver Communication Channel Configurations. The changes i made are bellow.

Maximum Concurrency  100 (it was 1 before)

Exactily -One Handling

    Presistence                 local ( did not changed it)

    Conflict Resolution      error ( it was Redo before)

In the advanced mode i enabled the

    Database Auto-Commit-Enabled

    Disconnect from Database After Processing Each Message

driver:oracle.net.CONNECT_TIMEOUT    5000

driver:oracle.jdbc.ReadTimeout                 30000

I saved the communication channel, activated it.

In the RWB i stoped the JDBC channel and restarted it.

Restarted the JAVA stack.

Then i saw those 5 Messages became  to the status "Waiting". I selected them and cancelled them.

From that moment the other messages were processed and the data warehouse reseved the IDoc data's.

For now this issue is solved, but i do not thing this is the final and effective solution for this problem.

Hope to hear any other more effective solution.

Regards,

Antony.

[engswee · Accepted solution]

Dear Antony

Please refer to my blog for a effective approach to preventing Delivering/To Be Delivered messages.

You should set a non-zero pool waiting time for your JDBC receiver. There are also other details/reference there for further performance tuning.

Rgds

Eng Swee

[antony_ferminus · Accepted solution]

Hi Eng Swee,

Thanks for the reply, appreciated.

I refered to your blog before. As you said i add the non-zero pool waiting time to the JDBC receiver now.

I still have the same question about what you wrote bellow,

1)   On newer versions or SP level of XI/PI, it is possible to release the threads held up by the         "Delivering" messages by stopping the corresponding channel. This will require the correct      version/SP level as described by OSS note 1604091.

        Is there already somthig released(patch) for PO 7.4 to stop and start the CC to change the      status of "Delivering" ?

2) NWA -> Configuration Management -> Infrastructure -> Java System Properties -> Details ->      Services > XPI Service: Messageing system >       messageing.system.queueParallelism.maxReceivers

     Can i change it to 1,2,or 10 ?

Hope to hear.

Regards,

Antony.

[engswee · Accepted solution]

Hi Antony

1) If you are already on PO 7.4, I expect that the functionality should be already in place.

2) For ICO configuration on single stack, you need to set both the max receiver and queue types. Refer to note 1493502 for further details. If you set max receiver of 10 and your work threads for the adapter are also 10, you could possibly have a single interface blocking the whole adapter. I would suggest probably to try out with 2 or 3 first, and adjust accordingly as required.

You can try the setting like below:-

messaging.system.queueParallelism.maxReceivers = 2

messaging.system.queueParallelism.queueTypes = icoall

Rgds

Eng Swee

[antony_ferminus · Accepted solution]

Hi Eng Swee,

Thanks for the quick reply.

I changed the settings for messaging system properties as you mentioned above after refering the note 1493502.

It will take effect after the restart of Java stack.

Thanks a lot for helping me out.

Regards,

Antony.

[antony_ferminus · Accepted solution]

Hi Eng Swee,

I have been testing around. The JAVA stack was resterted and i got the results to the external DataWarehouse without any problem.

This morning i saw 4 messages were in the Delivering status again from the test of my college. Once again afer the restart of the JAVA stack the 2 messages could be cancelled. But other 2 are still there.

I tested again and i am sure now the parallelisam is working because i got the IDOC from the Comunnication Channel for BUPA is coming through but not the HRMD because the 2 delivering status messages are blocking them.

The stop and the restart of the communication channel not helping even in version 7.4. We contacted to SAP also for this. No answers yet.

Rgards,

Antony.

[engswee · Accepted solution]

Hi Antony

If stopping the channel in 7.4 is not releasing the worker threads, then it might be possible that the bug/issue was reintroduced since they last fixed it.

Do keep us updated on the feedback from SAP. I am interested to know if indeed this is a new issue/bug in the latest version - maybe the new AAE IDoc adapter does not support this stop/start feature.

Rgds

Eng Swee

[antony_ferminus · Accepted solution]

Hi Eng Swee,

Indeed, i will update you when i receive any feedback from SAP.

Regards,

Antony.

[engswee · Accepted solution]

Dear Antony

One more thing: did you try stop/start on the sender IDoc channel and also for the receiver JDBC channel. For ICO the worker thread at the sender adapter is used, but I'm not too sure myself which channel is supposed to release the worker thread.

Rgds

Eng Swee

[antony_ferminus · Accepted solution]

Hi Eng Swee,

I stopped and restarted both of them. But no use.

I could not resend the messages because of the status DLNG.

I always get the message  "Unable to schedule 2 of 2 messages for processing; update the status":

Regards,

Antony.

[antony_ferminus · Accepted solution]

Hi Eng Swee,

In the message monitor i see the log like bellow.

Time Stamp                      Type Description

06.06.2014 09:08:42.073 Warning Adding message to blacklist. Reason: Message (in delivering                                              state) found during startup

06.06.2014 09:08:42.074 Warning Message added to blacklist

06.06.2014 09:08:42.076 Information Message status set to TBDL

06.06.2014 09:08:42.333 Information The message was successfully retrieved from the send queue

06.06.2014 09:08:42.343 Information Message status set to DLNG

Is ther i can do something for this blacklist to change the status from Delivering.

by this documentation may be i can disable it, but will it help?

Administrator Tasks for Blacklisting - Advanced Adapter Engine - SAP Library

Regards,

Antony.

[vishal1889 · Accepted solution]

Hi Antony

We also had similar kindly of situation in the past where SAP suggested to manually change the status of message in RWB.

You can execute below query to change the status in RWB, bust as this is occurring again and again I would still suggest to ask SAP to provide proper analysis and permanent solution.

Check whether the message  has been received by the receiver system:

(1) If message was received, please execute the SQL statement below to manually change the status of the message to Delivered:

UPDATE "SAP<SID>DB"."BC_MSG" SET STATUS='DLVD' WHEREMSG_ID='enter you message ID'

(2) If message was not received, please execute the SQL statement below to manually change the status of the message to system error, and then restart it in RWB:

UPDATE "SAP<SID>DB"."BC_MSG" SET STATUS='NDLV' WHERE MSG_ID='enter your message ID'

Do share the outcome

Regards

VJ

[antony_ferminus · Accepted solution]

Hi Vishal,

Thanks for the reply.

In my case the first option should work, but i do not have the direct access to the DB.

I have to wait till tomorrow for our basis college to come and execute the query.

We are still waiting for the answer from SAP.

I will let you know the result.

Regards,

Antony

[antony_ferminus · Accepted solution]

Hi Vishal,

We executed the query and the observed results are bellow.

1) In the RWB the status of the messages were still Delivering. When we click on it and open the messages, we see the status success.

2) When i open the PIMON i see the status Delivered.

Visually i see some changes but these 2 messages are still blocking the other messags to deliver.

Again i cleared the cache, restarted the java stack, but no use.

Regards,

Antony

[vishal1889 · Accepted solution]

Hi Antony

Thanks for sharing the feedback, even if changing the status value in database is not helping then I believe waiting for SAP response would be the best option.

In the meantime you can try to execute the second query and see it the status changes to system error, then may be you can try to cancel them.

All the best for your results

Regards

VJ

[antony_ferminus · Accepted solution]

Hi All,

At last we managed to solve the problem through the sap note 1296132.

Thanks for all the information.

Regards,

Antony.

[vishal1889 · Accepted solution]

Good to hear that your issue is resolved

Regards

VJ