WebIDE for SAP HANA: Configuration of Deployments to XSA (hdi)

[torsten_bauer]

Hi,

I have 2 spaces in my Organisation DEV and PROD.

There is not dedicated dev system available at the moment, so the development takes place in the DEV space and gets deployed into the PROD space by using the mtar which is created in the WebIDE by building the whole project.

We're using HANA 2.0 SP03.

Is there any way to change objects during deployment? - to make an example:

I have a virtual table created under the src folder in the hdb module of the project which points to the remote source system with name devdb:

VIRTUAL TABLE "promotion_lookup" AT "devdb"."testtb"."dbo"."promotion_lookup" 

On the other hand I have a remote source system named proddb where also the database has a different name (efashion) - so the correct content for the prod system of the hdbvirtualtable file would be:

VIRTUAL TABLE "promotion_lookup" AT "proddb"."efashion"."dbo"."promotion_lookup" 
after deployment. 

My test project setting looks like the following (I've left out the .che folders):

TESTPROJECT |
|___ db
|    |___ cfg
|    |    |___ .hdconfig
|    |    |___ promotion_lookup.hdbvirtualtableconfig
|    |
|    |___ src
|    |    |___ .hdconfig
|    |    |___ promotion_lookup.hdbvirtualtable
|    |___ package.json
|
|___ mta.yaml 

Content of promotion_lookup.hdbvirtualtable:

VIRTUAL TABLE "promotion_lookup" AT "devdb"."testtb"."dbo"."promotion_lookup"

Content of promotion_lookup.hdbvirtualtableconfig:

{   
  "promotion_lookup":    
  { 
    "target":       
    { 
      "remote": "proddb",         
      "database": "efashion", 
      "schema": "dbo", 
      "object": "promotion_lookup" 
    } 
  } 
} 

If I build only the src folder - I can access the devdb system by using the virtual table promotion_lookup.

Once I build the db folder or the whole project the virtual table points to the proddb system.

But that is not the way I want it to work. What I'm looking for is to configure that this change gets only done once I click on the TESTPROJECT_0.0.1.mtar and choose "Deploy to XS Advances"

Is there any way to change / configure this for a deployment szenario? Even if I would have a seperated DEV and PROD system I would probably face the same problem. Of course I would be able to setup different connections by using the same name on both systems - but once the database would also be different one it won't work.

So I assume there is somewhere a missing bit to me to make such a scenario happen.

Thanks in advance Torsten

[thomas_jung · Accepted solution]

Don't hard code the values in the *config files. Instead use the deployment place holder values as described in the HDI deployer documentation.

On startup, the HDI Deployer recursively scans the local cfg/ folder and picks up all files with a .*config suffix, e.g. all .hdbsynonymconfig, .hdbvirtualtableconfig, etc. files. For all collected files which contain .configure markers in their content, it applies the configuration templating and creates transient configuration files which are then deployed to the HDI container.

For a synonym configuration file cfg/LOCAL_TARGET.hdbsynonymconfig

{
    "LOCAL_TARGET" : {
        "target" : {
            "schema.configure"    : "logical-external-service/schema",
            "database.configure"  : "logical-external-service/database",
            "object"              : "TARGET"
        }
    }
}

It's not always applicable to use schema.configure, database.configure, etc. in the configuration template files. Therefore, the HDI Deployer provides a generic way of copying a set of properties from the bound service, e.g. schema, database, remote source, etc. if they are present, although the template file doesn't mention them.

For the configuration file cfg/LOCAL_TARGET.hdbsynonymconfig this could looks as follows:

{
    "LOCAL_TARGET" : {
        "target" : {
            "*.configure"         : "logical-external-service",
            "object"              : "TARGET"
        }
    }
}

When the HDI Deployer encounters a *.configure entry, it simply copies all well-known fields which are present in the bound service into the configuration file. The well-known fields are currently remote, database, and schema.

With this approach you can configure such parameters via User Provided Service in the target space and the values will be replaced at deploy time.

[torsten_bauer · Accepted solution]

Thanks Thomas, but I still not get it - hopefully you can point me into the right direction.

I had another problem where I had to give access to the remote source to the #OO user of my container. I solved that by using a User provides service and hdbgrants.

I've created the UPS with the following statement in DEV and PROD space - as mentioned it is on the same Hana System in the same organisation. DEV and PROD are only differenciated by the two spaces.

xs cups dwh-grantor -p "{\"host\":\"hdx\", \"port\":\"30015\", \"user\":\"DWH_GRANTOR\", \"password\":\"XXXXXX\", \"driver\":\"com.sap.db.jdbc.Driver\", \"tags\":[\"hana\"]"

Now I'm back to my initial question, where you've gladfully answered it and I think I understood it a bit - but can't put it into a solution.

Are well-known fields added to the xs cups comand like driver and tags or am i totally misunderstanding it?

Could I kindly ask for a bit more information on the above? What does LOCAL_TARGET mean? Is that a keyword?

and logical-external-service is that my dwh-grantor (User provided service)?

The above mentioned UPS I have created in PROD and DEV - if I need to change the remote source for PROD in prod_source and for DEV in dev_source - to make an example. Where do I define that? So that during runtime the right one is used?

Thanks a lot! Think once I've understood that I start understanding the mechanism - but for the moment I'm lost in Docs, Videos and other postings and might not see the missing bit

And schema - is that the name of the container which gets created at the end? because currently I have a proper name in the DEV space (even without the User etc in the name)but in PROD it gets generated with the UID or what kind it is name. Maybe due to the fact that a container with the name already exists on the system - but in a different space

Torsten

[thomas_jung · Accepted solution]

>Are well-known fields added to the xs cups comand like driver and tags or am i totally misunderstanding it?

I'm sorry but I'm not sure what you mean by that. The User Provided Service is used to create a connection string from Java and Node.js applicati

ons simply by binding to the service instance. This is why we have these fields. However I'm not sure if that really addressed your question or not.

>What does LOCAL_TARGET mean? Is that a keyword?

In that example LOCAL_TARGET is simply the dummy name of the database object in the local container. For instance in the HDBSYNONYM* example this would be the name of the local synonym. Don't read too much into the name used in the example. Its generic just because this is from general documentation. If you want a more specific example, have a look at the openSAP content:
https://github.com/SAP/com.sap.openSAP.hana5.example/blob/hana2_sps03_2/core_db/cfg/centralDB.hdbsyn...

>and logical-external-service is that my dwh-grantor (User provided service)?

It refers to your User Provided Service, but shouldn't be hardcoded with the actual service name. It should be a logical replacement name (and alias if you will) for the service that was defined in the mta.yaml. This way the actual service instance name can be overridden at deploy time with an mtaext file if needed. See the same above sample and its mta.yaml for a further example of this:
https://github.com/SAP/com.sap.openSAP.hana5.example/blob/hana2_sps03_2/mta.yaml

>For instance in the HDBSYNONYM* example this would be the name of the local synonym.

Yes exactly.

>if I need to change the remote source for PROD in prod_source and for DEV in dev_source

You change it in the configuration of the User Provided Service. That's the main point of the User Provided Service, to place this kind of configuration and settings outside of the design time and in a realm that can be be maintained separately by an admin.

>but in PROD it gets generated with the UID or what kind it is name. Maybe due to the fact that a container with the name already exists on the system - but in a different space

Yes if both spaces point to the same HANA MDC container, the schema obviously will conflict and the system is trying to protect you here. Really if you have different spaces used in such a way they should really point to different MDC containers.

[torsten_bauer · Accepted solution]

Still problems understanding this.. sorry about that.

Planned was to use the deploy function out of Web IDE for SAP Hana (so no command line -> no mtaext) - if possible.

There are 3 things I need to configure:

1) Remote Source + Remote Source Database - different for DEV and PROD

2) Different hdi-container names - different for DEV and PROD - but still readable names to communicate to users which are using Analysis for office.

While I develop I need the configuration for the DEV space and during deployment via WEB IDE to PROD.

I have created a User Provided Service in both spaces - with the same name (full statement in my 1st answer)

I understood that I need to generalize my configuration - for the right management for the #OO user I already got it working with the User Provided Service.

Also I think I understood the concept a bit more - especially due to your explanations - but still don't get it into working code.

How and where do I configure that (1) and 2) above) in the User Provided Service? And how to bring that into connection with my yaml or my virtual table config? Sorry - coming from a different world where my databases had schema (Sybase, Sybase IQ, MSSQL), my tools had configurations (SAP DataServices) etc. I also haven't found kind of a best practice guide or full guidance but I think I will try to put something together to share once i understood it with a complete example.

Please bear with me..maybe you can tell me how I tell my User Provided Service in PROD that it is PROD and that for PROD i need this database and that remote source and use the Schema Name X

Thanks & Sorry again

Torsten

Here is my current yaml

ID: DWH
_schema-version: '3.1'
description: DWH
modules:
 - name: DWH-DWF
   properties:
      memory: 512M
   requires:
    - name: DWH-dws
   type: com.sap.xs.dwf
 - name: DWH-Backend
   properties:
      memory: 512M
      API_END_POINT: ${xs-api-url}
   provides:
    - name: DWH-Backend
      properties:
         service_url: ${default-url}
   requires:
    - name: DWH-container
    - name: DWH-dws
   type: javascript.nodejs
 - name: DWH-Database
   requires:
    - name: DWH-container
      properties:
         TARGET_CONTAINER: ~{hdi-container-name}
         memory: 2048M
         HDI_DEPLOY_OPTIONS:
            auto_undeploy: true
    - group: SERVICE_REPLACEMENTS
      name: cross-container-service-1
      properties:
         key: ServiceName_1
         service: ~{the-service-name}
   type: com.sap.xs.hdi
resources:
 - name: DWH-dws
   properties:
      dwf-edw-client-name: ${service-name}
   type: com.sap.xs.dwf-edw-client
 - name: DWH-container
   parameters:
      service-keys:
       - name: dwf-monitor
   properties:
      hdi-container-name: ${service-name}
   type: com.sap.xs.hdi-container
 - name: cross-container-service-1
   parameters:
      service-name: dwh-grantor
   properties:
      the-service-name: ${service-name}
   type: org.cloudfoundry.existing-service
version: 0.0.5

[thomas_jung · Accepted solution]

>How and where do I configure that (1) and 2) above) in the User Provided Service?

You don't configure the User Provided Service connection details in your project or in the mta.yaml. The configuration of the User Provided Service is done by XS command line or XS Admin Cockpit.

>User Provided Service in PROD that it is PROD and that for PROD i need this database and that remote source and use the Schema Name X

You configure this in the User Provided Service settings. I don't understand the disconnect here.

[torsten_bauer · Accepted solution]

Thanks Thomas - my missing link was that I need to add the well-known fields remote and database to the cups jason string - like for my example:

xs cups dwh-grantor -p "{\"host\":\"hdx\", \"port\":\"30015\", \"user\":\"DWH_GRANTOR\", \"password\":\"XXXXXX\", \"driver\":\"com.sap.db.jdbc.Driver\", \"tags\":[\"hana\"], \"remote\":\"prod_source\""

and then reference it in the .virtualtableconfig as

"remote.configure": "dwh-grantor/remote",
"database.configure": "dwh-grantor/database",

sorry - might seems to obvious for you - but that was my problem - at the end too simple...

So to summarize - if I need more differnet sources then I would need to create a user provided service for each source to handle the "connection" change. As I can only use those 3 well-known fields - and not any others yet (appart from the standard host, port, user, password, driver and tags) - correct?

And to my problem with having PROD and DEV within the same ORG and the same MDC - what would
you suggest /advice without having a second hana system to seperate PROD and DEV? Or should we use different tenants on the same host for this

Or would it be possible or an idea to configure it the way that the hdi containers are created like:

DEV:  DWH-container_DEV
PROD: DWH-container

I assume I would need to misuse a user-provided service to deliver a schema name for each container with
the certain name - that is now easy - but how can I get the schema.configure then into the yaml or is there a different way via cfg/ folder etc?

resources:
- name: hdi_db
  parameters:
  config:
  schema: AUTH_TEST4
  properties:
  hdi-container-name: ${service-name}
  type: com.sap.xs.hdi-containe

This config would result in a container name hdi_db - correct? How to put in schema.configure here?

Thanks again
Torsten

[thomas_jung · Accepted solution]

>And to my problem with having PROD and DEV within the same ORG and the same MDC - what would

you suggest /advice without having a second hana system to seperate PROD and DEV?

Really if you have separate spaces that need any kind of isolation, they should really be mapped to different HANA MDC Containers. That would defintely be the recommendation.