Send DWC to AWS S3 (data Lake)

[abhishek_bhattacharjee]

Hello Gurus,

I have certain business reqirement to send Master/Transactional data from DWC to AWS (Data Lake - S3).

Can anyone help me on, what are the options to send the data from DWC to AWS-S3. Also pros, cons of using these options.

Any document or link would be useful.

Thanks,

Abhi

[XaviPolo · Accepted solution]

To consume DWC data from external tools, you have to do several things:

- Enable access to the DB (aka openSQL) in the Space > Configuration > Database Access. Here you can create a user with read-only access, and you will get the host, port, user and pass for connection.

- This will make it possible to query the analytical views that are exposed in that space. You can't access tables or other type of views.

- You must also enable access to DWC from the IP where connection to DWC is stablished (your PC, Amazon cloud, etc). This is done in the DWC > Configuration > IP Allowlist. Simply add the public IP.

After that, you can access to DWC using a JDBC HANA driver from Glue (or with other tools using ODBC driver, Python hdbcli, etc)

[abhishek_bhattacharjee · Accepted solution]

Thank you Xavier for answer.

[XaviPolo]

I think the simplest option is to configure the "openSQL" access in DWC, and expose the view containing the data (it must be of analytical type).

This will make it accessible from a remote system, for example from Amazon Glue, from where you can do the upload to S3.

Regards,

[abhishek_bhattacharjee]

Hello Xavier,

Thank you for the quick reply. I have 2 follow-up questions.

Where to do the "openSQL" access?

What connecter we need to use AWS access?

Best Regards

Abhi