Connecting MS SQL Server or Oracle Databse to HANA and perform Delta loads

[Former Member]

This is a personal project that I am doing from home so cost is a big issue here.

I have a database in MS SQL server. I want to do analysis on the data in HANA since SQL server will not be able to handle the load. SAP has a trial version of HANA that I have running on a borrowed server. Right now I have been able to connect my MS SQL server to HANA using BODS 4.0. In order to get the BODS working I needed CMS running on the machine so I had to install Business Objects full suite to get the CMS. Please bear in mind I didn't have any guidance or knowledge and whatever I have done so far is from watching youtube videos. So far I have managed to get BODS and BO from an SAP friend on trial basis but of course this cannot go on for ever. Eventually when I am done with practicing creating my data model I want to use HANA on AMazon AWS with my SQL server. The problem is that i have not been able to find a solution where I can connect my SQL server to HANA and then send my delta data from SQL server to HANA as I need. With BODS I can do a manual delta upload (after doing the first/initial data load) from SQL server to HANA and it works perfectly.

Bearing in mind i'm completely new to HANA, can you think of a solution for connecting SQL server to HANA on Amazon AWS and being able to do delta loads manually? I have to do delta loads manually since I can only afford to run HANA on Amazon AWS for couple of hours a day. I do the delta load and then run my analytics, get the results and then shut down the AWS service. Then the next day I start the service, do the delta load and do my analytics and shut the service. I can even change the source database from SQL server to Oracle if needed.

[lbreddemann]

The core piece of delta loads is of course to find records that changed since the last data transfer.

An easy way to do that is to keep a last changed date with all your records in all tables.

That makes it easy to find those records that have changed.

It's not "pretty" but can - accepting an initial development investment - typically be very easily implemented (especially when the number of tables is rather small).

With MS SQL and Oracle there are also APIs available to do Change Data Capture (CDC) - but my gut feeling is that this would be overkill in this setup.