#SAPCAP: Resilience Pattern - Timeouts using CDS M...

Ajit_K_Panda · ‎11-09-2023

Introduction

Resilience of a cloud application is its ability to maintain its functionality, availability, and performance even when failures, disruptions or unexpected events occur. Resilience strategies in cloud applications involve implementing design patterns, architectural practices, and technologies that ensure the system can gracefully handle failures and recover from them without causing significant disruptions to users.

In this blog post, I'll discuss a resilience pattern called timeouts within the context of developing applications on BTP using the Cloud Application Programming Model (CAP).

Refrence Github Repository: cap_resilient_timeout

Timeouts

Timeouts are similar to alarms in computer programs. When a task, such as running an API, reading a file, or executing a database query, takes too long, the program stops waiting. This prevents operations from becoming stuck, which could otherwise inefficiently use server resources. Using timeouts strategically is essential to ensure that applications are more resistant to external attacks caused by resource exhaustion, such as Denial of Service (DoS) attacks, and Event Handler Poisoning attacks.

Resilience patterns or strategies like timeouts can be applied to both inbound and outbound requests of a CAP application.

For outbound requests from CAP application like consuming external services, SAP Cloud SDK can be utilized, offering standard resilience patterns like timeouts, retries, and circuit breakers. If you want to explore this topic further, you can refer to this page: how to add resilience.
For incoming requests to CAP application, such as those initiated by a UI5 application, out-of-the-box support is currently unavailable. SAP Standalone Approuter provides timeout functionalities. Nevertheless, we can leverage the concept of middlewares and cds-plugin, to introduce timeout functionality.

CDS Middlewares:
CAP application framework leverages Express.js to serve services over a specific protocol. To enhance flexibility and modularity, the framework utilizes a middleware architecture. Middlewares are functions that can intercept and modify incoming requests or outgoing responses.

For each service served at a certain protocol, the framework registers a configurable set of express middlewares like context, trace, auth, ctx_auth, ctx_model etc. More information is available at this page: cds.middlewares

CDS Plugins:
The concept of CDS plugins allows developers keep specific functions separate from the main application code. These plugins can be used in multiple applications promoting modularity and code reuse. With plugins, capabilities of CAP application cab be enhanced by connecting to standard events and adding event handlers. If you want to get your hands dirty with plugin concept, refer following resources:

Timeouts for Incoming Requests

Let’s look at different options to apply timeout resilience for incoming requests to CAP application.

→ Via Standalone Approuter

The default-env.json file is used to set specific configurations as environment variables for the approuter. INCOMING_CONNECTION_TIMEOUT is one such environmental variable, that is used to set a timeout period for all requests.

Should a response fail to return within the configured timeframe, an error message - 'Error: socket hang up' - will be generated.
Application router can forward the incoming request to destinations based on route configuration. The destinations configuration can be provided by the destinations environment variable or by destination service. For each destination, it is possible to provide timeout configurations. When destination configuration is provided using default-env.json, timeout parameter is used and When destination is configured on BTP using cockpit, then HTML5.Timeout parameter is used.

If response is not returned within the configured timeframe, an error message – ‘504 Gateway Timeout’ – will be generated.

→ Via CDS Plugin

CAP Service Details:
Before diving into the timeout plugin code, let's first examine the CAP application code to gain a deeper understanding of the plugin's context and functionality.

// Service Details: service.cds 
service MyService {
  function getData(wait: Boolean) returns array of String(20);
}

// Service Logic: service.js
const cds = require("@sap/cds");
const { setTimeout } = require("timers/promises");

module.exports = cds.service.impl(async function (srv) {
    srv.on("getData", async (req) => {
        if (req.data.wait) {
            console.log('[DB] reading data from db - started!');
            await setTimeout(5 * 1000);
            console.log('[DB] reading data from db - finished!');
        }

        if (req.data.wait) {
            console.log('[API] reading data from api - started!');
            await setTimeout(5 * 1000);
            console.log('[API] reading data from api - finished!');
        }

        if (req.data.wait) {
            console.log('processing both sets of data started!');
            await setTimeout(5 * 1000);
            console.log('processing both sets of data finished!');
        }
        return ['StringData1', 'StringData2'];
    })

})

Service MyService has a function called getData which contains following logic:
- Fetch data from database
- Fetch data from api
- Process both data from above steps
All of the above steps add 5 seconds if wait parameter is set to true. This is done to test our application. Note that, we are simulating 5 seconds using setTimeout function. The setTimeout function from the timers/promises module in Node.js allows you to utilize the setTimeout function as a promise, enabling asynchronous operations with a more convenient syntax.

In summary, we have a function in MyService which takes around 15 second to process and return some data.
Implementation details of the timeout plugin:
Plugins are loaded by adding the local plugin folder as workspace in package.json
```
"workspaces":["resilience-plugin"]
```
It is possible to release these plugins as npm package and use it in multiple applications.

Additionally, plugin must contain cds-plugin.js file. This file contains the logic for your plugin and integrates it into your CAP (Core Data and Services) application. Let’s look at our current example:
```
const cds = require("@sap/cds");
const resilienceTimeout = require("./resilience_timeout");

const options = {};
options.timeout = 10000;
options.onTimeout = function (req, res) {
  let message = { errorCode: "TimeOutError", 
                  errorMessage: "Your request could not processed within a timeframe" };
  res.status(500).send(JSON.stringify(message));
};

cds.middlewares.add(resilienceTimeout.timeoutHandler(options));
module.exports = cds.server;
```
In the provided code snippet, timeout logic (resilience_timeout.js) is integrated into the CAP application as a middleware, accompanied by specific configurations. This is achieved by utilizing the add method from cds.middlewares.

Let's proceed to understand how timeouts can be configured and applied to all requests.

// Default Options: Values and Functions
const DEFAULT_DISABLE_LIST  = [ "setHeaders", "write", "send", "json", "status", "type",
                                "end", "writeHead", "addTrailers", "writeContinue", "append",
                                "attachment", "download", "format", "jsonp", "location",

                                "redirect", "render", "sendFile", "sendStatus", "set", "vary"   ];

const DEFAULT_TIMEOUT       = 60 * 1000; 
const DEFAULT_ON_TIMEOUT    = function (req, res){
    res.status(503).send({error: 'Service is taking longer than expected. Please retry!'});
}

//Implementation: Functions and Handlers
initialiseOptions = (options)=>{
    options = options || {};

    if (options.timeout && (typeof options.timeout !== 'number' || options.timeout % 1 !== 0 || options.timeout <= 0)) {
        throw new Error('Timeout option must be a whole number greater than 0!');
    }

    if (options.onTimeout && typeof options.onTimeout !== 'function') {
        throw new Error('onTimeout option must be a function!');
    }

    if (options.disable && !Array.isArray(options.disable)) {
        throw new Error('disable option must be an array!');
    }

    options.timeout     = options.timeout     || DEFAULT_TIMEOUT;
    options.onTimeout   = options.onTimeout   || DEFAULT_ON_TIMEOUT;
    options.disableList = options.disableList || DEFAULT_DISABLE_LIST;
    return options;
}

timeoutMiddleware = function (req, res, next){
    req.connection.setTimeout(this.config.timeout);
    res.on('timeout', (socket)=>{
        if (!res.headersSent) {
            this.config.onTimeout(req, res, next);
            this.config.disableList.forEach( method => {
                res[method] = function(){console.error(`ERROR: ${method} was called after TimeOut`)};
            });
        }
    });
    next();
}

timeoutHandler = (options)=>{
    this.config = initialiseOptions(options);
    return timeoutMiddleware.bind(this);
}

module.exports = {timeoutHandler};

In above code, there are 3 methods to note:
- timeoutHandler: this method initializes some configurations by calling initialiseOptions method and return a middleware which applies the timeout logic.
- initialiseOptions: this method initializes configurations like timeout period, response
  method when timeout happens etc.
- timeoutMiddleware: this method is nothing but a middleware which does the following
  1. sets timeout period for all requests using connection.setTimeout method
  2. Adds a handler for timeout event where response is set when timeout happens
  3. Also disables list of method with some error logic

In Node.js, req.connection.setTimeout is a method used to set the timeout duration for a specific HTTP request connection. When you set a timeout using this method, you are defining the maximum amount of time the server will wait for activity on the connection. If no data is sent or received within the specified timeframe, the server will automatically terminate the connection and emits a timeout event.

Testing the Plugin:

Send a request by calling getData function with wait=true parameter then it will return a timeout error after period provided in configuration as shown below:

### Wait=TRUE, Timeout Happens
GET http://localhost:4004/odata/v4/my/getData(wait=true)

Response

Final Thoughts:

It is important to understand that, even if a timeout is triggered, and the plugin sends a response, the processing of logic within CAP application service handlers does not stop. In the previous example, if a timeout happens and the request handler sends a timeout response to the client, it means that the request is terminated from the client's perspective, but the asynchronous operations that were already initiated will continue to execute on the server.

If you want to stop asynchronous operations when a timeout occurs, you would need to implement additional logic to cancel or abort these operations. This might involve using libraries or methods specific to the asynchronous tasks you're performing. For example, database libraries often provide mechanisms to cancel queries, and HTTP request libraries might have options to abort ongoing requests.

In Node.js, headersSent is a property of the response object that indicates whether the response headers have already been sent to the client. You can use this field to control some execution and also rollback some of earlier executions or transactions using tx.rollback(). More information is available here: srv.tx

It's important to handle asynchronous operations carefully, especially in scenarios where timeouts and other errors can occur. Proper error handling and cleanup mechanisms should be implemented to ensure the stability and reliability of your application.