Skip to Content
0

export table delta record from HANA to .csv

Nov 07, 2017 at 05:49 PM

172

avatar image

Hi All,

We have a requirement to replicate changed data from HANA to AWS S3. Does anyone done this before and any valuable input to share?

Based on my research, BODS 4.2 SP07 support S3 as target protocol. To make the requirement simple, can BODS replicate the changed data on HANA by converted them into a .csv file and transfer it to S3 bucket?

Or a SLT is required on this solution, where changed data on HANA is captured by SLT and push to BODS, BODS will convert them into .csv and send to S3?

Or a write a store procedure to capture the data changes, converted them to .csv and then manually sync it to S3 bucket? If yes, any guidance on writing a stored procedure that mimic SLT technology?

Any input is welcomed and appreciate.

Thanks in advance,

Nicholas Chang

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

1 Answer

Dirk Venken
Nov 08, 2017 at 12:30 PM
1

With HANA as a source you can consider following options for incremental loads in DS:

  • Use last_modification timestamps in the source tables. Keep track of the timestamp(s) that you used in the previous incremental extraction (use a control table for that), initialise global variable and use them in the where-clause of your Query transforms.
  • Use a log table. Many applications log modifications in so called log-tables. The actual table modified does not contain the timestamp, but the log table does. Join the tables and proceed as above.
  • Use SLT Replication Server.
  • If none of those apply, keep a copy of the previous data extraction, compare with current data to calculate the delta (use Table_Comparison transform), then load the delta records only.
Show 1 Share
10 |10000 characters needed characters left characters exceeded

Hi Dirk,

Thanks for the input! Much appreciate

SLT is not support hadoop as target system.

seems like there's no easy development for the solution above..

Thanks again!

Nicholas Chang

0