Tips for running CAP nodejs in SAP BTP

mauriciolauffer · ‎06-21-2023

CAP documentation has a best practices section for nodejs apps. In this blog, I won't talk about anything you can find there. I want to share best practices on the configuration to take your app to the next level of cloud-native applications. I'm going to share some tips for running CAP nodejs applications in SAP BTP. The examples will target Cloud Foundry, but these tips apply to both Cloud Foundry and Kyma/Kubernetes environments (with minor variations).

#1 Define nodejs engine

If you don’t define the nodejs version required by the application, it’ll take whatever is the default value set by the CF buildpack in use. Which can cause issues when running it on places other than your local machine.

For instance: you have nodejs v12 installed on your computer, you develop an app that works fine until you deploy it to SAP BTP. The app doesn’t work there. After checking the logs, you notice some errors because the app is running on node v18 and some APIs have changed.

Setting node engine also avoids issues when multiple developers are working on the same app. No more the classic line: “it works on my machine”.

If nodejs engine is not set when deploying to BTP, you’ll get a WARNING message. I’m sure you always check the logs after deploying your applications, right? 😉

[STG/0] STDOUT -----> Installing binaries

[STG/0] STDOUT engines.node (package.json): unspecified

[STG/0] STDOUT engines.npm (package.json): unspecified (use default)

[STG/0] STDOUT [31;1m**WARNING**[0m Node version not specified in package.json or .nvmrc. See: http://docs.cloudfoundry.org/buildpacks/node/node-tips.html

[STG/0] STDOUT -----> Installing node 18.15.0

The fix is quite simple! Just specify it in the package.json file.

// package.json

"engines": {

  "node": "^18"

}

Now, you made clear to people and machines which nodejs version is required to run the app.

[STG/0] STDOUT -----> Installing binaries

[STG/0] STDOUT engines.node (package.json): ^18

[STG/0] STDOUT engines.npm (package.json): unspecified (use default)

[STG/0] STDOUT -----> Installing node 18.15.0

Yes, you could also define the NPM version. TBH, I don’t care about it, I always use the default one shipped with the nodejs version specified in the package.json.

#2 Define memory and disk-quota

If you don’t define the memory and disk space required by the application, it’ll take whatever is the default value set by the CF buildpack in use. Similar to node engine.

This might be OK for some apps. However, other apps may need more memory than the default value. Or, the app has a tiny memory footprint much smaller than the default one. Setting the memory accordingly to its real usage will help you make better use of the resources available. You don’t want to waste precious resources in your subaccount.

Set the parameters memory and disk-quota in mta.yaml or manifest.yml files to define the container memory size.

// mta.yaml

modules:

  - name: cap-prd-best-srv

    type: nodejs

    path: gen/srv

    parameters:

      buildpack: nodejs_buildpack

      memory: 512M

      disk-quota: 1024M

#3 Set env var OPTIMIZE_MEMORY

By default, nodejs doesn’t know how much memory is available to use in the container/server/computer. A default memory limit is imposed, these limits vary based on the nodejs version. Nodejs garbage collector runs based on these values, and it may let the app accumulate too much garbage in the memory before cleaning it up. If the unused memory isn’t freed up, it may reach the memory limit allocated to the container and cause Out Of Memory errors crashing the app.

To fix this issue, you can explicitly configure the nodejs memory limit with node option --max-old-space-size.

$ node --max-old-space-size=2048 server.js

I don’t like this approach because I need to explicitly tell what is the memory limit which will vary from app to app. I prefer an easier and more dynamic approach to set the environment variable OPTIMIZE_MEMORY in the mta.yaml or manifest.yml files. This will always set max-old-space-size based on the available memory in the container. Another good reason to define memory as seen in #2 😉

// mta.yaml

modules:

  - name: cap-prd-best-srv

    type: nodejs

    path: gen/srv

    parameters:

      buildpack: nodejs_buildpack

      memory: 512M

      disk-quota: 1024M

    properties:

      OPTIMIZE_MEMORY: true

#4 Use the SAP BTP Application Autoscaler service

Elasticity and scalability are some of the best features cloud-native applications can be built on. Autoscaling, up and down, based on system load and specific metrics is awesome and too good to be ignored.

The Application Autoscaler service is free and easy to set up! Try to use it as much as possible and don’t stress whether your app will go down when users start flocking to it after that killer feature has been released.

In the following example, I’m creating an autoscaling policy that will keep a minimum of 2 instances and scale up to 5 instances whenever memory usage goes over 75% of the allocated memory.

// mta.yaml

modules:

  - name: cap-prd-best-srv

    type: nodejs

    path: gen/srv

    parameters:

      buildpack: nodejs_buildpack

      memory: 512M

      disk-quota: 1024M

    properties:

      OPTIMIZE_MEMORY: true

    requires:

      - name: cap-prd-best-log

      - name: cap-prd-best-autoscaler

        parameters:

          config:

            instance_min_count: 2

            instance_max_count: 5

            scaling_rules:

              - metric_type: memoryutil

                threshold: 75

                operator: '>'

                adjustment: '+1'

                breach_duration_secs: 60

                cool_down_secs: 60

              - metric_type: memoryutil

                threshold: 75

                operator: '<='

                adjustment: '-1'

                breach_duration_secs: 60

                cool_down_secs: 60



resources:

  - name: cap-prd-best-autoscaler

    type: org.cloudfoundry.managed-service

    parameters:

      service: autoscaler

      service-plan: standard

#5 Run $ cds-serve

You may’ve noticed a warning message when running $ cds run or $ cds serve on the terminal or in the BTP logs.

> cap-prd-best@1.0.0 start

> cds run





Warning: `cds run` will be removed with the next major version.

Use `cds-serve` instead in your start script. Set it with:



npm pkg set scripts.start="cds-serve"



[cds] - loading server from { file: 'srv\\server.js' }

[cds] - loaded model from 2 file(s):

Since March 2023 release, cds run and cds serve have been deprecated and were replaced by cds-serve. Update your scripts and package.json to use the new $ cds-serve.

// package.json

"scripts": {

  "start": "cds-serve"

}

#6 Use command to start the app

If you don’t define the command to start the application, it’ll take whatever is the default value set by the CF buildpack in use. Mostly, this is not a big problem because nodejs_buildpack uses $ npm start as the default command to start any application, and it’s very likely you’ve set npm start in your package.json file.

Set command in mta.yaml or manifest.yml files.

// mta.yaml

modules:

  - name: cap-prd-best-srv

    type: nodejs

    path: gen/srv

    parameters:

      buildpack: nodejs_buildpack

      memory: 512M

      disk-quota: 1024M

      command: npm start  #// do not use npm start

You may say "whatever, it's the same as not setting anything". True, if we'd use npm start to execute our apps, which we won't. This is the base for the next tip #7…

#7 Graceful shutdown: do not use $ npm start

Executing $ npm start has 2 issues: 1 minor and 1 major. It doesn’t matter whether you use command: npm start or not.

Minor issue: when you execute $ npm start, NPM will run your app as its child process. Therefore, rather than 1 process running in the container, you’ll have 2 processes: NPM + your app. This causes a tiny extra memory consumption. Not a big issue, but I like keeping my containers as lean as possible. Also, if there's any problem with NPM (not your app), everything will crash.

Major issue: NPM swallows termination signals sent to the application, it doesn’t forward signals to its children processes (your app) which makes it impossible to gracefully shut down nodejs apps.

To fix these issues, we use $ node to directly execute cds-serve. No overhead! No issues!

modules:

  - name: cap-prd-best-srv

    type: nodejs

    path: gen/srv

    parameters:

      buildpack: nodejs_buildpack

      memory: 512M

      disk-quota: 1024M

      command: node ./node_modules/@sap/cds/bin/cds-serve

“Wait, wait, wait! Slow down! Too many new terms here”. I hear you... Gotta make you understand. Never gonna give you up. Never gonna let you down...

Rick Astley - Never Gonna Give You Up

What are termination signals?

"A signal is an asynchronous notification sent to a process or to a specific thread within the same process to notify it of an event. Common uses of signals are to interrupt, suspend, terminate or kill a process."

Your running application is a process in the OS layer. Whenever a process is killed, the controller terminal sends a termination signal to the application saying: “hey, I'm about to kill this process. If you want to do something with that, now it's the time”. Examples of termination signals: SIGINT, SIGTERM, SIGKILL...

If you're a javascript developer, think about onunload event. If you're an ABAP developer, think about a BAdi called before closing a program. For example: open a terminal, run $ npm start, press CTRL + C to kill the process, a signal SIGINT is sent to the app and the app stops. This is a Windows example (I'm a Windows person), but the concept is the same in other OS.

[cds] - server listening on { url: 'http://localhost:4004' }

[cds] - launched at (...), version: 6.8.2, in: 613.1ms

[cds] - [ terminate with ^C ]

What is graceful shutdown?

A graceful shutdown allows the application to complete all unfinished tasks, clean up resources, and reject all new incoming requests notifying it is going offline. When an application doesn't handle termination signals, it may cause issues such as data loss, hanging connections consuming unnecessary resources (until timeout), server accepting new income requests while shutting down, and more.

As you may have noticed, CAP is a framework that abstracts a lot of things for us, including graceful shutdown. Whenever a CAP application receives a termination signal, the expressjs server is closed, HTTP connections and event handlers are terminated. We shouldn't need to do anything to handle termination signals unless it's something CAP doesn't control. For instance, you manually connected to a MongoDB.

Let's have a look at what CAP is doing for us. The below code is from CAP v6.8.4. We can see CAP is listening to signals SIGINT, SIGHUP, SIGHUP2 and SIGTERM.

// @sap/cds v6.8.4

// node_modules\@sap\cds\bin\serve.js



cds.shutdown = _shutdown //> for programmatic invocation

process.on('unhandledRejection', (_,p) => _shutdown (console.error('❗️Uncaught',p)))

process.on('uncaughtException', (e) => _shutdown (console.error('❗️Uncaught',e)))

process.on('SIGINT', cds.watched ? _shutdown : (s,n)=>_shutdown(s,n,console.log())) //> newline after ^C

process.on('SIGHUP', _shutdown)

process.on('SIGHUP2', _shutdown)

process.on('SIGTERM', _shutdown)



async function _shutdown (signal,n) {

  if (signal) DEBUG?.('⚡️',signal,n, 'received by cds serve')

  await Promise.all(cds.listeners('shutdown').map(fn => fn()))

  server.close(()=>{/* it's ok if closed already */})   // first, we try stopping server and process the nice way

  if (!global.it) setTimeout(process.exit,1111).unref() // after ~1 sec, we force-exit it, unless in test mode

}

Let's run a test in DEBUG mode to get a message when a signal is received. I like using .env files for it.

// .env

DEBUG=cli

Open a terminal, run $ node ./node_modules/@sap/cds/bin/cds-serve and kill the process. We'll see the termination signal SIGINT has been received and processed by our CAP application.

$ node ./node_modules/@sap/cds/bin/cds-serve



[cds] - loading server from { file: 'srv\\server.js' }

[cds] - loaded model from 2 file(s):



  db\schema.cds

  node_modules\@sap\cds\common.cds





[cds] - server listening on { url: 'http://localhost:4004' }

[cds] - launched at (...), version: 6.8.4, in: 421.997ms

[cds] - [ terminate with ^C ]





[cli] - ⚡️ SIGINT 2 received by cds serve

[cli] - ⚡️ cds serve - cds.shutdown

[cli] - ⚡️ cds serve - server.close(d)

[cli] - ⚡️ cds serve - process.beforeExit

[cli] - ⚡️ cds serve - process.exit

Now, do the same, but run $ npm start instead.

$ npm start



> cap-prd-best@1.0.0 start

> cds-serve



[cds] - loading server from { file: 'srv\\server.js' }

[cds] - loaded model from 2 file(s):



  db\schema.cds

  node_modules\@sap\cds\common.cds





[cds] - server listening on { url: 'http://localhost:4004' }

[cds] - launched at (...), version: 6.8.4, in: 541.124ms

[cds] - [ terminate with ^C ]





[cli] - ⚡️ SIGINT 2 received by cds serve

[cli] - ⚡️ cds serve - cds.shutdown

[cli] - ⚡️ cds serve - server.close(d)

[cli] - ⚡️ cds serve - process.beforeExit

[cli] - ⚡️ cds serve - process.exit

WHAT?!?!? The termination signal has been received in both cases! I wrote this bunch of stuff for nothing?!? 😭

Remember I said I was a Windows person? This happens in Windows only. Other OS wouldn't receive the signal in the $ npm start scenario. The problem would surface only when deploying to SAP BTP because the CF nodejs_buildpack runs your app in a Linux container. I'm pretty sure you don't run Windows containers in Kyma as well. Just to make it clear, it's not a "BTP issue".

Alright, let's run another test, this time on SAP BTP.

Prepare your app to use npm start. You can even use command to start the app like in #6. Make sure to set the env var DEBUG=cli to get the shutdown log entries. Build and deploy it to SAP BTP ($ cf deploy ?). The app will automatically run after deployment has been successfully completed. Stop the app ($ cf stop ?) and get its logs ($ cf logs ?).

Towards the end of the log, you should find the following entries saying the app has started, container was healthy, and the app has stopped. No log entries for the signals as we've seen before when running local. There! That's the issue! NPM has swallowed the signal and we couldn't gracefully shutdown our app.

[APP/PROC/WEB/0] STDOUT [cds] - server listening on { url: 'http://localhost:8080' }

[APP/PROC/WEB/0] STDOUT [cds] - launched at (...), version: 6.8.4, in: 205.094ms

[CELL/0] STDOUT Container became healthy

[API/17] STDOUT Stopping app with guid fdc7e06e-da60-4dd1-b487-6d1e4bf29cd6

[CELL/0] STDOUT Cell 5458fb1f-ef6d-440f-bb4f-799e53d5917d stopping instance b878c935-52a7-425e-604d-eb89

Repeat the process, but this time start the app like in #7 using command: node ./node_modules/@sap/cds/bin/cds-serve.

Check the logs and you'll see the signal entries there. No SIGINT, but SIGTERM because it's a Linux container. Our app was gracefully shutdown! All HTTP connections were closed, no new connections were accepted once the stopping process has started. Hooray!!!

[APP/PROC/WEB/0] STDOUT [cds] - server listening on { url: 'http://localhost:8080' }

[APP/PROC/WEB/0] STDOUT [cds] - launched at (...), version: 6.8.4, in: 208.836ms

[CELL/0] STDOUT Container became healthy

[API/23] STDOUT Stopping app with guid 9ecf2c9c-7423-45c9-8ec0-0b9775ea0f5c

[CELL/0] STDOUT Cell 5458fb1f-ef6d-440f-bb4f-799e53d5917d stopping instance 2b87737d-8917-44d6-6570-67d8

[APP/PROC/WEB/0] STDOUT SIGTERM signal received: shutting down...

[APP/PROC/WEB/0] STDOUT [cli] - ⚡️ SIGTERM 15 received by cds serve

[APP/PROC/WEB/0] STDOUT [cli] - ⚡️ cds serve - cds.shutdown

[APP/PROC/WEB/0] STDOUT [cli] - ⚡️ cds serve - server.close(d)

[APP/PROC/WEB/0] STDOUT [cli] - ⚡️ cds serve - process.beforeExit

[APP/PROC/WEB/0] STDOUT [cli] - ⚡️ cds serve - process.exit

[APP/PROC/WEB/0] STDOUT Exit status 0

[CELL/0] STDOUT Cell 5458fb1f-ef6d-440f-bb4f-799e53d5917d destroying container for instance 2b87737d-8917-44d6-6570-67d8

[PROXY/0] STDOUT Exit status 137

[CELL/0] STDOUT Cell 5458fb1f-ef6d-440f-bb4f-799e53d5917d successfully destroyed container for instance 2b87737d-8917-44d6-6570-67d8

Last, but not least. If you want to add custom logic when shutting down, you can easily add event handlers to the signals you need:

// ./srv/server.js

const cds = require('@sap/cds');



process.on('SIGINT', function() {

    console.log('SIGINT signal received: shutting down...');

});

process.on('SIGTERM', function() {

    console.log('SIGTERM signal received: shutting down...');

});



module.exports = cds.server;

References