Skip to Content
author's profile photo Former Member
Former Member

Best Practice for keeping track of former event

Hi,

in my scenario I am dealing with database events on HANA: If a database record is insert some logic shall be applied on that row and shall be written to another database table within HANA. The tricky thing is that the applied logic is based on what has already been inserted (e.g. insert the new record only if this has not been done before). This is somehow a replication task. But on last SAP TechEd, ESP has been named as one of the replication tools for HANA!!

Right now, I do not know how to solve this problem in a very efficient way. Using streams will not work as anytime the ESP will be re-started all data gets replicated again and duplicates the data on the target HANA tables ;-( Using windows (in combination with file store) does also not solve the issue, as the HANA tables are containing too much data. Therefore I created an additional HANA table to maintain a status flag of the records which have been replicated and wrote a JDBC library that looks for replicated rows and insert a flag to the ones that been replicated. My JAVA code works fine in a standalone program, but crashes the ESP server...

Before looking deeper into the crashing problem, I thought I would post a discussion and see whether there are other (better) approaches of keeping track on previously handled events. Any examples or suggestions?

Thanks and Best Regards.

Markus

Add a comment
10|10000 characters needed characters exceeded

Related questions

1 Answer

  • Posted on May 28, 2013 at 02:38 PM

    From what you describe - unless I've misunderstood - I don't think ESP is the right tool for you. ESP is not a replication tool for SAP HANA and is not recommended for such. It is one of the "data provisioning" tools available for HANA, and is recommended for use in capturing data from real-time event streams - but is not recommended for use in replicating data between databases. With that said, ESP can be used with SAP Sybase Replication Server: RepServer performs realtime change capture on a source database and turns those DB events into a real-time event stream that can be fed into ESP (ESP has support for RepServer) where the events can be processed or analyzed - e.g. you can monitor trends, patterns, etc. But even here - if you just want to replicate the data from the source into a destination DB, then RepServer doesn't need to go through ESP.

    But if you're looking to replicate data from one HANA table to another, then I don't think ESP is probably your best option.

    Now if you are working with an incoming stream of events (and yes, they could be database "events" - i.e. DB transactions), then it is possible to stream them through ESP and have ESP keep track of what's been stored in the target table. i.e. if there really is a need for ESP here, you can design your ESP data model to avoid duplicating data after a re-start.

    And again, maybe I've misunderstood the use case...

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.