A few questions on the SAP PI Messaging System when AAE has been used

[Former Member]

Hi Gurus,

We are using SAP PI 7.4 Dual Stack system and ICO has been used for all the integration scenarios, which means we are using just the advance adapter engine in JAVA stack . The ABAP integration engine has not been used.

Let's take an example. Say SAP sends an IDOC to PI and PI forwards the message to a third party system after mapping via a REST receiver adapter.

This is the process steps as in my understanding in PI:

1. Message has been pushed to the IDOC_AAE sender adapter;
2. Message has been processed by the module processor;
3. Message has been put to Messaging system and been persistent in DB;
4. Message has been put into the dispatcher queue;
5. Message has been picked up from dispatcher queue and put to the IDOC_AAE send queue(asynchronous sender);
6. Determine the receiver via the definition in ICO;
7. Determine the receiver interface via the definition in ICO;
8. Execute mapping program and get the target message;
9. Message has been put to the REST Receiver adapter from the IDOC_AAE send queue(only sender adapter queue will be used in AAE);

If my understanding is correct, i have the these bellowing questions:
1. About the Message Prioritization.
When this prioritization occurs? Normally we just have one thread for dispatcher queue. it means all messages will be queue up in that queue. we know that the definition of Message Priority is based on interface level, rather than adapter level and we know that by defaul, HIGH stands for 75% of resource, NORMAL(or no configuration) stands for 20% of resource and LOW stands for 5% of resource. Then what does this 'resource' mean here? The number of threads of the adapter specifiy queues? I do not think so. Because the threads belong to certain adpater specific queues and the max number of threads per these queues has already been configured in XPI Service: Core. On the other hand, if the messages have been queued up regardless of its interface, how can it cut the line according to the priority defined?

For example: message type: IDOC_A has been defined as HIGH prority and message type: IDOC_B has been defined as LOW prority. we have 1 IDOC_A message sent out right after 100 IDOC_B has been sent to PI.
step 1, they will be put into the dispatcher queue with IDOC_A message in the last place. Since there is only thread, IDOC_A will be put to the IDOC_AAE queue after all 100 IDOC_B be put into IDOC_AAE queue.
step 2, by default there are maximium 5 threads for IDOC_AAE queues. if the target system is a bit slow, all of the 5 queues can be used at this moment.
Then obviously IDOC_A should be the last one in one of these 5 queues. Since in AAE, there is only sender queue used, i cannot see any possibility that it can cut the line from its HIGH prority until it has been put to the REST receiver adapter.

Then the question is:
what does this 'resource' mean here? is it inside the adapter specific queue? when it will be consumed in PI or when the defined priority will be taken into consideration?

2. About 'wait time in the dispatcher queue' in the performance monitor
let's take the example as above.

An IDOC has been sent to PI AAE and been forwarded to a third party system via REST receiver adapter.
For scenario like that, we will get two records in the performance monitor. One is for the outbound IDOC message and the other is for the inbound REST message.
what confuse me is, for both messages, you will find a figure for 'MS:Message_Wait_in_Disp_Queue' which very strange.

For IDOC message, it does make sense. Because it is coming from SAP and it has been put in the dispatcher queue before being picked up to IDOC_AAE queue. whereas for the inbound REST message after the mapping, i cannot understand why it has a figure for 'wait time in the dispatcher queue' here. It should have nothing to do with the dispatcher queue.

So my questions are:

Has the message after the mapping ever been put to a dispatcher queue in PI? If yes, is it another dispatcher queue or still the same one for the incoming message from sender system? if it is the same one, how can the Messaging System know that it is an incoming message from sender system or it is a message after mapping? How can they be sequenced in the queue?

thanks

[HarshC · Accepted solution]

You have to remember that queues in Java are regular queues. They don't follow FIFO or EOIO. See details below.

Queues in the Messaging System
The queues in the Messaging System behave different then the qRFC queues in ABAP. The ABAP queues are strictly First In First Out (FIFO) queues and are only processed by one Dialog work process at a time. The Messaging System queues have a configurable amount of so called consumer threads that process messages from the queue. The default number of consumer threads is 5 per adapter (JDBC, File, JMS, …), per direction (inbound and outbound) and Quality of Service (asynchronous (EO, EOIO) and synchronous). Hence in case a message is taking very long other messages that arrived later can finish processing earlier. Therefore Messaging System queues are not strictly FIFO.

1) When does prioritization happen?

When moving messages from Dispatcher Queue -> Adapter Send Queue.

From the performance doc see step 3:

1) Enter the sender adapter
2) Put into the dispatcher queue of the messaging system
3) Forwarded to the File send queue of the messaging system (based on Interface priority)
4) Retrieved from the send queue by a consumer thread
5) Sent from the messaging system to the pipeline of the ABAP Integration Engine (if no Integrated
Configuration (AAE) is used
6) Processed by the pipeline steps of the Integration Engine
7) Sent to the messaging system by the Integration Engine
8) Put into the dispatcher queue of the messaging system
9) Forwarded to the JDBC receive queue of the messaging system (based on Interface priority)
10) Retrieved from the receive queue (based on the maxReceivers)
11) Sent to the receiver adapter
12) Sent to the final receiver.

2) Why do I see "Wait Time" in Dispatcher Queue

Multiple entries in performance monitor are usually because of a message split.

If your question is about wait time in Dispatcher Queue. This is usually when the Dispatcher Queue is unable to find a Adapter Send Queue to send to. This could be because of different reasons:

• All threads for the adapter send queue are in use
• You have setup maximum threads for Receiver System or Receiver Interface

[Former Member · Accepted solution]

Hi Harsh,

appreciated for your neat and tidy statement.

For the 1st one, my confusion is:

The number of thread for the dispatcher queue is normally 1. Therefore I assume the messages in the dispatcher queue should be First In First Out. Consider we have 5 number of threads for IDOC_AAE Sender Queue and the processing time for each of messages is almost the same. The IDOC_A message with HIGH priority has been sent after 100 IDOC_B messages with LOW priority. Therefore in the dispatcher queue, the sequence should be like this:

IDOC_B|IDOC_B|IDOC_B|...|IDOC_B|IDOC_A, with the sequence from left to right.

If the dispatcher program has no ability to prioritize messages inside the queue, obviously IDOC_A will be picked up at last. We know that one thread of the adapter specific queue can only handle one message. It means the earliest time that IDOC_A can be allocated to an IDOC_AAE queue is after 96 IDOC_B messages have been released by the IDOC queue. Does it make sense? Then the IDOC_A has been allocated to one of the IDOC queue with other four still have unprocessed IDOC_B messages. If it is at this time that the priority has been taken into consideration, the majority resource of other 4 IDOC queues has been given to the IDOC queue for IDOC_A and let it process faster. The IDOC_A message should be still behind those 96 IDOC_B messages. The effect of Prioritization should be very tine I am afraid and which is not the case in reality.

So I would presume the priority should be considered by the dispatcher program when the message is still in the dispatcher queue. if yes, the name of dispatcher queue is a bit confusing. It is better called dispatcher pool 🙂

If it is the case, there should be three queues in parallel in the pool, HIGH, NORAML and LOW . The dispatcher program will take 75% of time supervising the HIGH queue, 20% of time supervising NORMAL queue and just 5% of time checking LOW queue. of course, if the queue should be empty according to the configuration, those time resource will be allocated to other queue(s). I cannot find any document explaining this mechanism inside the PI dispatcher queue. just guessing. 🙂

By the way, we are using AAE which means there is no receiver queue.

For the second question.

can I ask, when the message is doing a message-split via multi-mapping, is it inside an adapter specific queue or is it inside the dispatcher queue or is it out of the send queue? after mapping, will the target message be placed back to the dispatcher queue? thanks

[HarshC · Accepted solution]

1) Queue vs Pool: I agree, the naming makes no sense. It's more like a pool. I believe the dispatcher thread uses 75/20/5 split when picking messages from the pool. I too would love to get my hands on additional documentation.

2) Message Split: I believe all pipeline steps happen within the adapter send queue. Therefore the mapping split would also happen there. However all new messages start processing, by first being placed in the dispatcher queue. So if RD/ID creates split messages those get put back in the dispatcher queue. There is an example of this listed in the performance doc. I'm on my phone so can't get to it. Look for an example with the JMS adapter.

[Former Member · Accepted solution]

Hi Harsh,

do you mean 'SAP PI Performance Check Guide'? it did not say the spitted message(child messages) will be put back to the dispatcher queue. but just the send queue.

cheers.

[HarshC · Accepted solution]

You are correct.

Step 4b does say that splits are put back in the send queue(not dispatcher queue).

1) Enter the JMS sender adapter
2) Put into the dispatcher queue of the messaging system
3) Forwarded to the JMS send queue of the messaging system
4) Message is taken by JMS send consumer thread:
a. No message split used:
In this case, the JMS consumer thread will do all the necessary steps previously done in the ABAP
pipeline (like receiver determination, interface determination, and mapping) and will then also
transfer the message to the backend system (remote database, in our example). Thus, all the
steps are executed by one thread only.
b. Message split used (1:n message relation):
In this case there is a context switch. The thread taking the message out of the queue will process
the message up to the message split step. It will create the new message and put it in the Send
queue again from where it will be taken by different threads (which will then map the child
message and finally send it to the receiving system)

In that case, I'm not sure why your messages show twice in performance monitoring. Could you share the full audit log for one of the messages?

[HarshC · Accepted solution]

Going back to Prioritization. I have a question of my own.

Even though the documentation says the following:

1) Prioritization happens when moving messages from Dispatcher Queue -> Adapter Send Queue

2) You can define prioritization rules based on all the following fields.

• Sender Party
• Sender Component
• Receiver Party
  
• Receiver Component
• Interface
• Interface Namespace

I don't understand how prioritization is possible based on ReceiverParty/ReceiverComponent.

The ReceiverParty/ReceiverComponent are identified during pipeline steps.

The pipeline runs in Adapter Send Queue.

This is after the message has already left the dispatcher queue.

Any ideas? 🙂

[Former Member · Accepted solution]

Hi Harsh

Again, I could not find any office document on this detail topic. just assuming by using what has happened in the performance monitor and message monitor.

I think both the Prioritization and message splitting occurs in the dispatcher queue. The following analysis is all based on this assumption.

we know that the Prioritization is based on the interface and it occurs in the dispatcher queue as discussed above. without message splitting(receiver determination, interface determination), the program cannot know the receiver interface from the message perspective. Therefore, rather than in the adapter specific queue, the message splitting process should occur in the dispatcher queue. by using this assumption, it can explain why we have 'wait time in dispatcher queue' statistic for those target messages after mapping. For example: we have one source message, IDOC_A from SAP and it will be splitted into two messages Target_1, Target_2 for two inbound interfaces respectively.

for any reason, the speed that the target system receives message Target_1 is slower than Target_2 and the speed of both of these two inbound interfaces are slower than the speed SAP sends IDOC_A to PI. AND, the max number of threads for IDOC_AAE send queue is ONLY 2. (by default, it is 5. but to make it simple, let's say 2 here)

Then what will happen is:

1. IDOC_A messages will be queued up in the dispatcher queue;

2. First IDOC_A message has been split(or duplicated is more preside) for Target_1 and Target_2 in the dispatcher queue. No mapping program will be executed. No target message has been generated at this step;

3. The duplicated instance of IDOC_A has been assigned to IDOC_AAE send queue for different threads respectively.

4. Message Target_2 has been generated and been sent to target system, while message Target_1 is still being generated.

5. Second IDOC_A message has been split(or duplicated) for two inbound interfaces in the dispatcher queue.

6. The duplicated instance of the second IDOC_A for Target_2 has been assigned to IDOC_AAE queue with the released thread. It could be the one for Target_1 of course. but let's suppose it is Target_2 here.

However If the prioritization has been configured for the inbound interface whose message type is Target_1 with value HIGH, the duplicated instance of second IDOC_A for Target_1 will have 75% of chance to be picked here.

7. The second Taregt_2 message has been generated and sent to target system. the first target_1 has been generated but it is on the way to be transmitted to its target system.

8. Third IDOC_A message has been splitted in the dispatcher queue.

9. Randomly this time the instance of second IDOC_A for Target_1 has been assigned to the IDOC_AAE queue.

10. In the IDOC_AAE queue, we have two threads for Target_1 now. One is under generation. The other is waiting for an HTTP response.

11. Now in the dispatcher queue, we have:

1) lots of IDOC_A messages waiting to be split;

2) One duplicated instance of IDOC_A for Target_1 is waiting to be picked up;

3) One duplicated instance of IDOC_A for Target_2 is waiting to be picked up;

So I assume:

the wait time in the dispatcher queue of those inbound interfaces should be, I guess, the wait time of the duplicated( split or after RD/ID ) instance of source message in the dispatcher queue to be picked up to the adapter specific queues.

I do not think part of the pipe line process( RD/ID) in AAE is purely done in the adapter specific queues. The RD/ID should be taken into consideration in the dispatcher queue stage.

[Former Member · Accepted solution]

This is from the performance monitor of one of our inbound interface. The average wait time in the dispatcher queue is outstanding here.

[HarshC · Accepted solution]

"How dispatcher queue can use receiver party/servcie for message prioritization?"

Stephen, I agree with your analysis.

As per the following link RD happens in dispatcher queue is put into the dispatcher queue.

https://help.sap.com/viewer/5cf7d2de571a45cc81f91261668b7361/7.5.6/en-US/48b2dfe86b156ff4e10000000a4...

Measuring Points of the Adapter Engine


MS:module:... (e.g. MS:module:CallSapAdapter, MS:module:sap.com/com.sap.aii.af.soapadapter/XISOAPAdapterBean) - processing time in the adapter module
MS:stage:SI - Virus scan inbound
MS:stage:BI - before Inbound XML Validation
MS:stage:VI - after Inbound XML Validation
MS:stage:MS - after Receiver Determination
MS:Message_Put_In_Store - persist the message to the database
MS:Message_Put_In_Disp_Queue - put the message in the dispatcher queue
MS:Message_Wait_In_Disp_Queue - waiting time in the dispatcher queue
MS:Message_Put_In_Queue - put the message in the processing queue
MS:Message_Wait_In_Queue - waiting time in the processing queue
MS:Message_Update_Status - update the message status and persist the new status into the DB
MS:stage:AM - after Mapping execution
MS:stage:SO - Virus Scan Outbound
MS:stage:VO - after Outbound XML Validation
MS:stage:AT - end of transport processing block. End of messaging system processing and hand-over to adapter.

I also see this in performance monitoring. RD in Dispatcher Queue.

[HarshC · Accepted solution]

Also attaching performance logs of a message goes through RD split:

Steps before Split.

Steps After Split:

[Former Member · Accepted solution]

good finding. so the only main task inside the adapter specific queue is mapping for non-bpm scenarios. I think this topic can be closed at the moment.

thanks

[Ryan-Crosby]

Hi Stephen,

In regards to your first question - I believe what will happen here is that IDOC_A will be sent using one of the IDoc send queue threads at the next available point when one of the IDOC_B transmissions has completed. By default you have 5 threads available which means that the system will process 5 messages at any single point in time. Once a single IDOC_B has been sent successfully then the system would have the opportunity to prioritize IDOC_A ahead of any remaining IDOC_B messages and you should observe 1 IDOC_A message being transmitted while 4 other IDOC_B messages are being transmitted. As for the second question the only time I have observed a second entry for the receiving interface is if I have some type of acknowledgement enabled. I don't believe anything would be placed in the dispatcher queue on the receiving side because at that point the system has already dispatched it, unless it is something such as an acknowledgement and the system is treating it like a new message.

Regards,

Ryan Crosby

[Former Member]

Hi Ryan,

thanks for your answer. For question 1, I still have something unclear.

if all the messages have been queued up in the dispatcher queue, how can an IDOC send queue thread be assigned to IDOC_A on priority? remember, normally we just have one thread for dispatcher queue. Consequently the messages in the dispatcher queue should be First In First Out.

But yes, your right. what you have explained is the same as what does occurred in our system. The message with HIGH priority did find a way to cut the line somehow. 🙂

for the second question. The REST Receive adapter will finally receive a HTTP status response, 200 or 400 etc. from the target system. may be it is a kind of ack. that has been populated into the dispatcher queue.

thanks anyway. I didn't expect to get an answer at quickly.

[Ryan-Crosby]

HI Stephen,

Yes, as Harsh has already mentioned the naming choice in a data structures sense is poor as any true queue would be FIFO in observation. As far as a second message any acknowledgement or message split would begin processing through the normal sequence of steps and also need to be dispatched before it can be handled the messaging framework.

Regards,

Ryan Crosby