Custom HTTP responses and debugging python functions in Kyma

[ajmaradiaga]

In this blog post, I will share details on how to create custom responses and debug python 🐍 functions locally using Visual Studio Code.

As the requirements of a function increase in complexity... it is likely that you will need to handle errors, return different status codes, and return different types of payloads. This is to better communicate what's going on within the function (you don't want to communicate everything back as an HTTP 200). Also, to speed up development it is important to be able to run and debug your code locally, as it will run in the Kyma runtime. Fortunately, the Kyma CLI allows us to debug a function locally. This is achieved by running our code within a container, simulating the Kyma serverless environment.

The Kyma documentation is available on GitHub. Meaning that anyone can contribute to it. This is exactly what I've done to include details on how to create custom responses, as well as share the configuration required to debug python functions locally. You can check out the Pull Request (PR) here - https://github.com/kyma-project/kyma/pull/16669. Fingers crossed🤞the PR is approved in the future The PR was approved 🎉 :-).

Let's get to it... how can we create custom responses and debug python functions locally....

Custom HTTP Responses

In a Node.js function, the response object is included within event.extensions. This is not the case with a python function. For python, there is no response object included in the event and there are no details on how to return a custom response.

The functions specification differ depending on the runtime used to run the Function - https://kyma-project.io/docs/kyma/latest/05-technical-reference/svls-08-function-specification/.

To demonstrate how we can create custom responses, I developed a function that all it does is "revealing/expanding" any URL included in the body of the request. The function handles two content types: application/json and text/plain and carries out basic validation to ensure that the request data is correct.

By using the HTTPResponse object in bottle, we are able to control the response returned by our Kyma function to the client. It is possible to specify the HTTP status, as well as include different headers and custom payloads. In the code below I've included a couple of examples, returning an HTTP 200, 400, 500 and responses with different content types.

import json

import re

from urllib.parse import urlparse



import requests

from bottle import HTTPResponse

from requests.exceptions import ConnectionError





SUPPORTED_CONTENT_TYPES = ['text/plain', 'application/json']





def reveal_url(url, remove_tracking=True):

    temp = urlparse(url)



    target_url = None



    if temp.scheme != '':

        print(url)

        try:

            r = requests.get(url, allow_redirects=False)



            if r.status_code == 301 and 'Location' in r.headers:

                target_url = r.headers['Location']

            else:

                print(r.status_code)

        except ConnectionError:

            print("Invalid URL but will proceed processing...")



    return target_url





def prepare_error(status, message):

    content_type = 'application/json'

    headers = {

        'Content-Type': content_type,

        'Additional': 'Test'

    }

    response_payload = json.dumps({'error': message})



    return HTTPResponse(body=response_payload, status=status, headers=headers)





def main(event, context):



    # Retrieve the Pickable request object

    request = event['extensions']['request']



    ###########

    # Lets have a look at the headers and query params of the request

    ###########



    print(f"Request headers:")

    for k, v in request.headers.items():

        print(f"- {k}: {v}")



    print(f"Request query parameters:")

    for k, v in request.query.items():

        print(f"- {k}: {v}")



    print("Request attributes: ")

    print(dir(request))



    # Check that the function support the Content-Type sent

    if 'Content-Type' not in request.headers:

        return prepare_error(400, 'No Content-Type specified. Content types supported: text/plain and application/json.')

    elif request.headers.get('Content-Type') not in SUPPORTED_CONTENT_TYPES:

        return prepare_error(400, 'Invalid Content-Type. Content types supported: text/plain and application/json.')



    content_type = request.headers.get('Content-Type')

    response_payload = None



    if content_type == 'application/json':

        #########

        # Process JSON payload

        #########



        # Bottle will automatically populate this property if the body is

        # of type application/json

        json_request = request.json



        urls = json_request['urls']



        resolved_urls = {}

        for u in urls:

            resolved_urls[u] = reveal_url(u)



        response_payload = json.dumps({

            'resolved_urls': resolved_urls

        })

    elif content_type == 'text/plain':

        #########

        # Process Plain Text

        #########



        # Converting from bytes to str

        text = request.body.read().decode('UTF-8')



        # Get all URLs included in the text

        urls = re.findall(

            '(?:(?:https?|ftp):\/\/)?[\w/\-?=%.]+\.[\w/\-&?=%.]+', text)

        print(urls)



        response_payload = text



        for u in urls:

            # Resolve URL and replace it in text

            resolved_url = reveal_url(u)



            if resolved_url is not None:

                response_payload = response_payload.replace(u, resolved_url)



    else:

        return prepare_error(500, f"Unmanaged Content-Type: {content_type}")



    #########

    # Prepare function response

    #########



    headers = {

        'Content-Type': content_type

    }



    response = HTTPResponse(

        body=response_payload, status=200, headers=headers)



    return response

Why use HTTPResponse and not HTTPError, LocalResponse, or Response?

If you checked out the bottle API documentation, you might have seen other possible options of objects that we can use to return the results back to the client. In my case, I tested out HTTPResponse, HTTPError, LocalResponse, and Response and got different results when using each one of them. I've captured the differences in the table below.

When testing the different objects, in all cases I was trying to set the exact same data:

• Status code: 400 (Bad Request).


• Headers: Specifying Content-Type and an Additional header.


• Content-Type of response: application/json.


• Payload: { "error": "Invalid Content-Type. Content types supported: plain/text and application/json." }.

You can see this in the code, it is part of the prepare_error method.

Object	Status Code	Headers	Content-Type	Payload
Response	200	Not modified	text/html; charset=UTF-8	JSON structure included
LocalResponse	200	Not modified	text/html; charset=UTF-8	No body included in response
HTTPError	400	Modified	application/json	Is an HTML error page that contains the error and status code
HTTPResponse	400	Modified	application/json	JSON structure included

As you can see in the table above, HTTPResponse is the only object that communicates what's expected to the client.

Debugging

The Kyma documentation includes the configuration required in Visual Studio Code to debug a Node.js function locally. Unfortunately, it is missing the configuration required for a python function. If you are like me, a lot more comfortable with python than Node.js, it comes quite handy to be able to debug your python function locally and use this to speed up our development instead of deploying our code to Kyma and then testing it there and checking the logs in Kyma. It will take forever to develop a function.

So, in the .vscode directory, create the launch.json file with the configuration below. This will allow you to run the function in debug mode (kyma run function --debug) and connect to it from Visual Studio Code.

{

    "version": "0.2.0",

    "configurations": [

        {

            "name": "Python: Kyma function",

            "type": "python",

            "request": "attach",

            "pathMappings": [

                {

                    "localRoot": "${workspaceFolder}",

                    "remoteRoot": "/kubeless"

                }

            ],

            "connect": {

                "host": "localhost",

                "port": 5678

            }

        }

    ]

}

To be able to debug the function locally you require the Kyma CLI installed locally. When setting the  --debug flag, the container where your function will run also exposes port 5678, apart from port 8080, to allow remote debugging. The video below shows you how to debug your function locally.

👀 In case you are curious/want to dive a bit deeper.... To learn about how bottle is used within the Kyma runtime.... you can go to the container's filesystem and check out the kubeless.py and ce.py files in it. This is how I end up finding out about HTTPError and that led me to HTTPResponse when reading the bottle API documentation.

Thanks for reading this far 😀. I hope this blog post is useful to you in the future when writing python 🐍 functions in Kyma.