SAP Generative AI Hub is now available with the SAP AI Core extended service plan. By leveraging Generative AI Hub, SAP partners and customers can build AI-powered extensions and apps on SAP BTP.
As part of the SAP AI Core, Generative AI Hub provides access to the most common large language models (LLMs) from different providers. To integrate an LLM via Generative AI Hub into an application, there are a few important steps to follow, that will be covered in this blog:
- Create an SAP AI Core service instance.
- Create an LLM configuration to reference a model provider specific executable.
- Use the configuration to create a deployment.
The configuration and deployment can be created either via the SAP AI Launchpad UI or using the SAP AI Core APIs. - For the created deployment, SAP AI Core provides a unique URL that can be used to access the LLM. Use this URL to infuse generative AI features into your application. Embedding the use of the URL into an application is not covered in this blog. Instead, the use of the deployment URL is demonstrated with Postman.
Prerequisites
- You have an SAP BTP Enterprise Account.
- You have an entitlement for SAP AI Core, Service Plan "Extended".
- You have an entitlement for SAP AI Launchpad, Service Plan "Standard".
- You have an entitlement and free quota for SAP BTP, Cloud Foundry Runtime.
SAP AI Core Setup
First, make sure with your global account administrator that all services listed in the Prerequisites section have service plan quota assigned to your global account.
As subaccount administrator, go to the account level, Entitlements -> Entity Assignments screen and select your subaccount. Click Edit, then Add Service Plans to configure entitlement.
Add 1 unit quota for the extended service plan then save entitlements update.
Create SAP AI Core Service Instance
Navigate to your subaccount and select Services -> Instances and Subscriptions. Click on Create. Fill out the fields as shown below and click Create.
In case you have an existing standard plan service instance, the creation will fail as shown below.
Update existing SAP AI Core service instance instead. For that, go back to Services -> Instances and Subscriptions and search for the SAP AI Core standard service plan service instance. Click on Update button illustrated below:
In the next screen, select extended plan instead of the standard plan and Update Instance.
Create Service Key
On the Instances and Subscriptions screen, find your SAP AI Core service instance and choose Create Service Key from the three dots dropdown.
Enter the name and click Create.
Download and store your service key json file locally. You will use it in the next steps.
SAP AI Launchpad Setup
As subaccount administrator, go to the account level, Entitlements -> Entity Assignments screen and select your subaccount. Click Edit, then Add Service Plans to configure entitlement.
Add 1 unit quota for the standard service plan then save entitlements update.
Subscribe to SAP AI Launchpad
SAP AI Launchpad is a SaaS application, and you need to subscribe to it. Follow these SAP Help steps to create a subscription: https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/subscribe-to-sap-ai-launchpad
Set up User Access to SAP AI Launchpad
In your subaccount go to Security -> Users and select your user. Assign role collections as required. Follow these SAP Help steps: https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/allow-access-to-sap-ai-launchpad
If you want access to Generative AI Hub features, don’t assign role collections containing “without_genai”. If you want to be editor for specific features, assign only the editor role collections, and not both, viewer and editor role collections for that feature.
Create Connection between SAP AI Launchpad and SAP AI Core
Following subscription, you’ll be able to launch SAP AI Launchpad application. Go to Instances and Subscriptions screen in your subaccount. You’ll see your SAP AI Launchpad application under the Subscriptions. Click on it to open it.
Workspaces app is used to manage connections between SAP AI Launchpad and AI runtimes such as SAP AI Core. AI runtimes provide resources to process AI activities such as training and inference.
In the Workspaces, you need to add a new AI API connection. Click on Add and fill out the fields as shown below. Under the Service Key field, click the upload symbol to upload the SAP AI Core service key json file that you stored locally as per Create Service Key section.
Click on Create and the connection will be established as shown below.
You also need to select a resource group so that you can perform ML Operations tasks. Resource groups represent a virtual collection of related resources within the scope of your AI runtime. Runtime entities such as executions, deployments, configurations, and artifacts belong to a specific resource group and are not shared across resource groups.
During subscription to SAP AI Launchpad, when your tenant is onboarded, a default resource group is automatically created. For your connection, you can select the default resource group as shown in the picture above.
Create a Configuration
To be able to deploy and consume LLMs you first need to create a configuration that will reference a model provider specific executable.
An executable is used to define training or serving pipelines for an AI use case. An executable that is used to serve (deploy) an AI model is called a serving executable.
Access to the LLMs is provided under the global AI scenario foundation-models, which is managed by SAP AI Core. Individual models are provided as executables in the form of serving templates and accessed by choosing the corresponding template for the desired model.
See models that are currently supported here:
In SAP AI Launchpad, go to ML Operations -> Configurations and click Create.
Give your configuration a name, select the foundation-models scenario and the version, and select the serving executable.
Click Next to enter Input Parameters.
Select model name you want to deploy. See model version in the link provided above. Note that the model version can be given as latest. If the model version is not specified, it will be set to latest by default.
Click on Next -> Review -> Create to create the configuration.
When you now click on ML Operations -> Configurations on the left-hand side menu, you’ll see the created configuration.
Create a Deployment
A deployment runs a model for serving (inferencing) purposes.
A serving executable is an AI pipeline that serves or deploys a model for online predictions. As we saw in the previous section, you use a configuration to specify values for the serving executable, such as the model as an input artifact. The configuration is then used to create a deployment.
A running deployment can only reference one configuration. Multiple deployments can be created using the same configuration, resulting in separate endpoints for online predictions.
You use SAP AI Launchpad to create deployments for your runtime connection. Deployments that are implemented on an SAP AI Core runtime produce HTTPS endpoints. These endpoints can be used to access the LLM.
Following the previous step, click on the created configuration, then on Create Deployment.
Keep Duration as Standard and click on Review then Create to create the deployment.
Once the deployment created, wait for the current status to be set to RUNNING.
It could take a few minutes for the deployment to go from UNKNOWN status to RUNNING status.
Take note of the deployment URL as you will use it in the next section.
Please note that both, the configuration and the deployment can also be created by using SAP AI Core APIs. See the API Reference on the SAP Business Accelerator Hub https://api.sap.com/api/AI_CORE_API/resource/Configuration,
and https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-... for details. Use the AI_API_URL from your SAP AI Core service key as the "apiurl".
Consume Generative AI Models
Now that you have a deployment running, you can run inference requests against it, and it will return the results from the LLM. LLMs have broad applicability for a range of natural language processing tasks, including text prediction and generation, content summary, translation, content classification and categorization, conversational AI and chatbots, etc.
Models from Azure OpenAI are accessed through a private instance of the chat-completion API.
The endpoint to use is {{deploymentUrl}}/chat/completions?api-version={{api-version}}.
For more info on this API and version to use, see https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions
With each inference request, you need to pass SAP AI Core access token and AI-Resource-Group key value in the header. To get a new token, use OAuth 2.0 client credentials grant type and find client ID, client secret and access token URL values in your SAP AI Core service key. See pictures below:
The query is sent in the body of the request, under “messages”. There are other optional parameters that you can include, and for their definitions, see https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/consume-generative-ai-models-using-s...
The picture below shows the body and response for a text summarizing request. For more examples and details, see the same link above.
In conclusion, all the steps covered in this blog are prerequisites to integrating an LLM via SAP Generative AI Hub into extensions and applications built on SAP BTP, with the purpose of adding generative AI features and functionalities to your application. Embedding the use of deployment URL programmatically into your application can be at the basis of building such functionalities.
If you are a partner that needs structured support in building your solutions on SAP BTP, check out our Co-Innovated with SAP service: Co-Innovated with SAP Service | Innovation | Partner Benefits Catalog | Partner Marketplace
If you need additional details, reach out to us at coinnovate@sap.com.
- SAP Managed Tags:
2 Comments
| User | Count |
|---|---|
| 14 | |
| 11 | |
| 10 | |
| 10 | |
| 9 | |
| 8 | |
| 7 | |
| 7 | |
| 7 | |
| 7 |
