This blog is part of a series where my goal is to offer you a practical overview of SAP Integration Suite, Advanced Event Mesh. As I continue to write more blogs, I will provide summaries on the page below.
Advanced Event Mesh: Create your first event broker
To create a queue, open the broker manager and click on "Queue."
In the basic settings, there are a few important considerations:
Access Type:
If maintaining the order of message processing is crucial, use exclusive, as one consumer may process a message faster than others.
Queue Size: Limit the size of the queue in megabytes. This helps manage the storage capacity allocated for the queue.
Consumer Count: Limit the number of consumers that can connect to the queue. This ensures control over the concurrency of message processing
In the example below, I've configured my queue with a capacity of 5gb, set to exclusive mode, and restricted the number of consumers to a maximum of three.
After you have created your queue, you will see it displayed like this:
I've connected the consumers (applications) to the queue. On the summary tab, we can observe that there are no messages currently being exchanged, although I have reached the maximum limit of connected consumers.
On the Consumers tab, we can view all connected consumers listed by their respective names.
If an attempt is made to connect a fourth consumer, the connection is rejected. The consumer is connected to the broker, but it cannot establish a connection to that specific queue due to reaching the configured limit.
A queue can have multiple topic subscriptions, allowing a single queue to listen to various events. For instance, consider a queue representing an application interested in events such as sales orders, purchase orders, invoices, payable accounts, and receivable accounts. The queue receives different kinds of events and delivers them in the received order, following the First In, First Out (FIFO) principle.
To listen to an event, navigate to the Subscriptions tab. Remember that you can utilize wildcards (* and >) to assist in filtering the messages.
In my example, my applications are interested in the event "Sales Order Created," as depicted in the image below.
I have a producer application that has published five instances of that event.
The three applications attached to the queue are awaiting events. However, only Application 1 has received those events, as image below. This is a result of the "exclusive mode" configuration that I set up for the queue.
This is how exclusive mode operates: the first consumer connected to the queue receives all the messages.
On the consumers tab, you can check the messages that have been delivered to the consumers. This allows you to monitor how the system is functioning.
I have now configured my queue to a non-exclusive mode and published an additional 5 messages. As observed, the messages were delivered in a round-robin manner, demonstrating the broker's effective load balancing across the consumers
When setting up a queue in a non-exclusive mode, it's important to understand if your scenario allows for it. For example, consider the following:
Imagine I have two instances of a microservice, and they are all listening to events related to sales orders. Instance one receives an event indicating that a sales order with number 1000 was created and starts processing it. Meanwhile, instance number two receives an event updating the same sales order. Consequently, it attempts to update data that hasn't been created yet by the first instance, leading to an error.
Acknowledging / Unacknowledging messages
When publishing messages using guaranteed messaging mode, we seek assurance of message delivery to both the broker and the consumer. If the consumer is not connected, we expect the broker to persist the message and deliver it once it is connected.
With guaranteed messages, both the broker and the consumer can send a response confirming receipt of the message and indicating successful processing.
Let's examine the example below:
The response confirming that a message can be processed is known as an acknowledgment (ACK), while the indication of being unable to process it is known as an unacknowledgment (NACK).
In a second example, let's imagine that the broker sends an acknowledgment (ACK) to the producer, confirming its ability to receive and store the message. Now, the broker is tasked with delivering the message to the consumer. The consumer, unable to process the message, responds with a negative acknowledgment (NACK).
Simulating this with my broker, I have already sent two messages to two consumers that were able to process them. However, with my third consumer, I am deliberately forcing it to reject all messages.
In the image below, two consumers sent me an ACK.
In the subsequent image, the third consumer sent me a NACK, and the message was redelivered to another consumer for processing it.
When a consumer replies with a NACK, we have some options to deal with it:
For messages to be stored in a Dead Message Queue, the producer must declare that the message is eligible for it, otherwise, it will be discarded.
In the upcoming blog, we will explore the replay feature on the broker.
Advanced Event Mesh: Create your first event broker
Queues
To create a queue, open the broker manager and click on "Queue."
In the basic settings, there are a few important considerations:
Access Type:
- Exclusive: The first connected consumer will receive all messages, while others are on standby, waiting for their turn.
- Non-exclusive: Messages are delivered in a round-robin fashion, as a load balancer.
If maintaining the order of message processing is crucial, use exclusive, as one consumer may process a message faster than others.
Queue Size: Limit the size of the queue in megabytes. This helps manage the storage capacity allocated for the queue.
Consumer Count: Limit the number of consumers that can connect to the queue. This ensures control over the concurrency of message processing
In the example below, I've configured my queue with a capacity of 5gb, set to exclusive mode, and restricted the number of consumers to a maximum of three.
After you have created your queue, you will see it displayed like this:
I've connected the consumers (applications) to the queue. On the summary tab, we can observe that there are no messages currently being exchanged, although I have reached the maximum limit of connected consumers.
On the Consumers tab, we can view all connected consumers listed by their respective names.
If an attempt is made to connect a fourth consumer, the connection is rejected. The consumer is connected to the broker, but it cannot establish a connection to that specific queue due to reaching the configured limit.
A queue can have multiple topic subscriptions, allowing a single queue to listen to various events. For instance, consider a queue representing an application interested in events such as sales orders, purchase orders, invoices, payable accounts, and receivable accounts. The queue receives different kinds of events and delivers them in the received order, following the First In, First Out (FIFO) principle.
To listen to an event, navigate to the Subscriptions tab. Remember that you can utilize wildcards (* and >) to assist in filtering the messages.
In my example, my applications are interested in the event "Sales Order Created," as depicted in the image below.
I have a producer application that has published five instances of that event.
The three applications attached to the queue are awaiting events. However, only Application 1 has received those events, as image below. This is a result of the "exclusive mode" configuration that I set up for the queue.
This is how exclusive mode operates: the first consumer connected to the queue receives all the messages.
On the consumers tab, you can check the messages that have been delivered to the consumers. This allows you to monitor how the system is functioning.
I have now configured my queue to a non-exclusive mode and published an additional 5 messages. As observed, the messages were delivered in a round-robin manner, demonstrating the broker's effective load balancing across the consumers
When setting up a queue in a non-exclusive mode, it's important to understand if your scenario allows for it. For example, consider the following:
Imagine I have two instances of a microservice, and they are all listening to events related to sales orders. Instance one receives an event indicating that a sales order with number 1000 was created and starts processing it. Meanwhile, instance number two receives an event updating the same sales order. Consequently, it attempts to update data that hasn't been created yet by the first instance, leading to an error.
Acknowledging / Unacknowledging messages
When publishing messages using guaranteed messaging mode, we seek assurance of message delivery to both the broker and the consumer. If the consumer is not connected, we expect the broker to persist the message and deliver it once it is connected.
With guaranteed messages, both the broker and the consumer can send a response confirming receipt of the message and indicating successful processing.
Let's examine the example below:
- The publisher application sends an event to the broker.
- The broker receives the message and, if quota is still available, persists the message and places it in a queue.
- The broker replies to the consumer, confirming that it can be processed.
- Once the consumer is connected to a queue, the broker pushes the message to the application.
- The consumer application processes the message and replies to the broker confirming successful processing.
- The broker deletes the delivered message.
The response confirming that a message can be processed is known as an acknowledgment (ACK), while the indication of being unable to process it is known as an unacknowledgment (NACK).
In a second example, let's imagine that the broker sends an acknowledgment (ACK) to the producer, confirming its ability to receive and store the message. Now, the broker is tasked with delivering the message to the consumer. The consumer, unable to process the message, responds with a negative acknowledgment (NACK).
Simulating this with my broker, I have already sent two messages to two consumers that were able to process them. However, with my third consumer, I am deliberately forcing it to reject all messages.
In the image below, two consumers sent me an ACK.
In the subsequent image, the third consumer sent me a NACK, and the message was redelivered to another consumer for processing it.
When a consumer replies with a NACK, we have some options to deal with it:
- When a consumer replies with a NACK and disconnects from the session, the broker will redeliver the message to another consumer connected to the queue. If the application sends a NACK but remains connected, the message redelivery will persist for this specific application.
- Set up 'Try forever': As the name suggests, it keeps attempting to deliver the message to that consumer as long as the application is connected. The broker pushes the message to it, and you can set up a delay, for example, trying again after N milliseconds.
- Set up a maximum redelivery count: It will attempt to redeliver the message N times, and if it continues to receive a NACK, it will either store the message in a dead message queue or
discard the message.
For messages to be stored in a Dead Message Queue, the producer must declare that the message is eligible for it, otherwise, it will be discarded.
In the upcoming blog, we will explore the replay feature on the broker.
- SAP Managed Tags:
