The power of cloud in your AI application
Introduction
The advent of SAP HANA Cloud's Vector Store engine marks a significant milestone in the realm of intelligent data applications. This feature enhances the ability to manage and retrieve unstructured data, such as text, images, and audio, by storing them as high-dimensional vectors or embeddings.
In this blog, we will demonstrate how to use the HANA Cloud Vector Store in tandem with the product master API from S/4HANA Cloud or On-Premise. We will also discuss the integration of embedding models with the vector store. HANA Vector Store is available since 2nd April 2024.
Storing and Retrieving Data with SAP HANA Cloud Vector Store
The HANA Cloud Vector Store (VS) is designed to handle unstructured data by converting it into numerical representations known as embeddings. These embeddings are then stored in a format that allows for efficient retrieval based on similarity or relevance.
Connecting to HANA Cloud Vector Store
To connect to the HANA Cloud VS, we use the hana_ml library, which provides a ConnectionContext that facilitates the connection to the HANA database:
def get_hana_connection(conn_params: dict)->Connection:
""" Connect to HANA Cloud """
connection = None
try:
conn_context = ConnectionContext(
address = conn_params["host"],
port = 443,
user = conn_params["user"],
password= conn_params["password"],
encrypt= True
)
connection = conn_context.connection
logging.info(conn_context.hana_version())
logging.info(conn_context.get_current_schema())
except Exception as e:
logging.error(f'Error when opening connection to HANA Cloud DB with host: {conn_params["host"]}, user {conn_params["user"]}. Error was:\n{e}')
finally:
return connection
The DB host would be something like this:
<some uuid>.hna0.eu10.hanacloud.ondemand.com
Embedding Data
Using the open-source embedding model "intfloat/multilingual-e5-small", we transform the product data into embeddings:
from langchain_community.embeddings.sentence_transformer import SentenceTransformerEmbeddings
EMBEDDING_MODEL = "intfloat/multilingual-e5-small"
embed = SentenceTransformerEmbeddings(model_name=EMBEDDING_MODEL)
Of course, you can use e.g. OpenAI Embeddings like "text-embedding-3-small" or "text-embedding-ada-002", then benefits of using open-source embeddings however, can be privacy (no 3rd-party involved), cost (no charges for using embeddings) and low latency (conversion happens local). That is particularly beneficial when working with large data sets.
Storing Embeddings
We use API API_PRODUCT_SRV from S/4HANA Cloud or On-Premise to retrieve the product data and create a text representation out of it using e.g. dimensions, material and or sales texts and others. Then we'll create a list of Documents (a langchain object) like so:
# Append the product information in text format to the list
product_in_text.append(
Document(page_content=descriptive_text, metadata={"product_id": product_id})
)
We then generate the embeddings and store them in the HANA Cloud VS using the HanaDB class from the langchain_community.vectorstores module:
from langchain_community.vectorstores.hanavector import HanaDB
def embed_product_data(conn: Connection, embed: SentenceTransformerEmbeddings, s4data: list):
vector_db = HanaDB(embedding=embed, connection=conn, table_name="S4PRODUCTDATA")
vector_db.add_documents(s4data)
return True
You see it's very simple to work with it. This will create a table with the name you specify. When we check the HANA Database Explorer you'll find the following data model:
Below is an example of our product master data, as text representation, with the according metadata (product number) and the vector representation.
Retrieving Similar Products
To retrieve similar products based on a query, we use the as_retriever method, which allows us to perform similarity searches within the vector store:
def retrieve_data(vector_db: HanaDB, query: str):
retriever = vector_db.as_retriever(k=5) # Retrieve top 5 similar items
similar_products = retriever.retrieve(query)
return similar_products
If you use langchain's LCEL representation you can model the complete chain like in below example where retriever is the returned object of retrieve_data.
rag_chain = (
{
"query": itemgetter("query"),
"language": itemgetter("language"),
"context": itemgetter("query") | retriever
}
| my_prompt
| llm
)
Deleting Data
To maintain data hygiene or to refresh the embeddings, we may need to delete existing data from the HANA Cloud VS which works also very easy.
def delete_embeddings(vector_db: HanaDB):
vector_db.delete(filter={}) # Deletes all embeddings; filters can be applied as needed
Integrating with Generative AI Hub
The Generative AI Hub serves as a gateway to various large language models (LLMs), including GPT-3.5, GPT-4 and others. With the gen_ai_hub Python library, we can easily integrate these models into our applications. You get started with it this way:
$ pip install generative-ai-hub-sdk
Setting Up the Generative AI Hub Client
We initialize the Generative AI Hub client, which will allow us to interact with the available LLMs:
from gen_ai_hub.proxy import GenAIHubProxyClient
GENAIHUB_PROXY_CLIENT = GenAIHubProxyClient()
You can read more about how to use it (and how to provide the authentication) on both
Using the Generative AI Hub for Product Search
We can use the Generative AI Hub to process natural language queries and retrieve relevant product information. Here's a simplified example of how to use the hub to generate responses:
from gen_ai_hub.proxy.langchain import init_llm
def get_model(model_info: dict)->any:
set_proxy_version("gen-ai-hub")
model_name = "gpt-4-32k" # Azure OpenAI GPT-4 32k
llm = init_llm(model_name=model_name, proxy_client=GENAIHUB_PROXY_CLIENT)
return llm
def generate_response(llm: BaseLanguageModel, query: str)->any:
llm_chain = LLMChain(prompt=prompt, llm=llm)
response = llm_chain.invoke(question)
return response
You can also implement streaming of the response by using something like this:
def streaming_output(query: str, rag_chain: RunnableSerializable)->any:
text_response=""
try:
for response in rag_chain.stream({"query": query, "language": "en"}):
text_response += response.content
logging.debug(text_response) # This can be a lot of output
yield text_response
except Exception as e:
# Implement some error handling
logging.error(f"Error was {e}")
return text_response
Conclusion
As we've explored, the integration of SAP HANA Cloud Vector Store with S/4HANA's product master API and the Generative AI Hub unlocks a new horizon of possibilities for advanced product search capabilities. By harnessing the power of embedding models, developers can craft applications that not only respond to natural language queries with remarkable accuracy but also maintain privacy and cost-efficiency. The seamless process of storing, retrieving, and managing embeddings in the HANA Cloud Vector Store, when combined with the cutting-edge capabilities of generative AI, equips businesses with the tools to create dynamic, intelligent applications that evolve alongside their data.
The future of Business AI is bright, and SAP's ongoing enhancements to its ecosystem are a testament to the commitment to innovation. As developers and partners, we stand on the cusp of a transformative era where the applications we build today will shape the data-driven enterprises of tomorrow. So, let's continue to push the boundaries of what's possible with SAP's Generative AI Architecture!
Call to action
Are you ready to take your S/4HANA applications to the next level? We invite you to dive into the world of intelligent data applications with SAP HANA Cloud's Vector Store. Start by experimenting with the code snippets provided, explore the resources for a deeper dive, and share your experiences with us. Your insights and innovations could pave the way for the next breakthrough in Business AI.
You definitely got to check out our SAP HANA Cloud Vector Store Early Adopter program!
Have questions or thoughts to share? Join the conversation below in the comments, or reach out to us directly. If you found this post helpful, please share it with your network to help others on their journey to unlocking the full potential of their data with SAP.
Let's innovate together – your next intelligent application is just an idea away!
Note: The code snippets provided in this blog are for illustrative purposes and should be integrated into a larger application context for full functionality.
Want to understand embeddings better? Check out my blog: Generative AI: Some thoughts on using Embeddings
- SAP Managed Tags:
3 Comments
