Is it possible to create a realtime flowgraph with more than one realtime source?

[Former Member]

We want create a realtime flowgraph with many data sources, and we need to set realtime flag for all these data sources (data sources are virtual table from Oracle).

We tried to create a flowgraph with two data sources with realtime flag checked but, when we try to save it, we get this error:

"DP realtime flow graph may have only one realtime Data Source".

Do you have any suggestion?

Thanks in advance

[former_member240828 · Accepted solution]

Hi Enrico,

Flowgraphs currently only support one realtime source. In case a join partner is another virtual table it will still be able to match the incoming records with the latest versions of the non realtime virtual table. However the trigger for the execution of the delta flow is only the realtime enabled source.

One example:

2 Virtual Tables (Invoices and Customers) get joined and the output is pushed to a hana table.

Invoices will be the Realtime enabled source.

For the Initial load it will join all existing invoices and customers and write the result into the hana table.

If now a new Invoice comes in it will join it with the current version of the customer table and write it into the hana table.

If a new customer comes in, it won't trigger the delta flow.

-

Timo

[Former Member · Accepted solution]

Thanks a lot, Timo!

[Former Member · Accepted solution]

I and my collegues are looking for a workaround to managed 2 realtime table joined together. Let me explain our idea with an example:

A and B virtual tables joined and the output is pushed to C hana table.

We want create 2 flowgraphs:

1) A realtime, B not, joined and the output is pushed to C

2) A not realtime, B realtime, joined and the output is pushed to C

With these two flowgraphs together (we have to understand the correct type of join for each flowgraph)

, can we have the same behavior of a hypothetical (since flowgraph currently support only one realtime source) single flowgraph with 2 realtime virtual source table ?

Could it work? Has anyone already tried that?

[former_member240828 · Accepted solution]

Hi Enrico,

it can work, should however be only used with caution, as this might result in duplicate records in the output table. Please have a look at this example (Hana 2 Hana for simplicity):

Source Hana:

CREATE COLUMN TABLE RT_Invoices (Invoiceno BIGINT, CUSTOMERID INT , AMOUNT FLOAT);
CREATE COLUMN TABLE RT_Customers (CUSTOMERID INT , NAME NVARCHAR(255), COUNTRY NVARCHAR(2));
INSERT INTO RT_Invoices VALUES (1, 1, 3);
INSERT INTO RT_Invoices VALUES (2, 1, 4);
INSERT INTO RT_Invoices VALUES (3, 2, 5);
INSERT INTO RT_Customers VALUES (1, 'Some Ltd', 'US');
INSERT INTO RT_Customers VALUES (2, 'Other Inc', 'UK');

On the Target Hana:

Create virtual tables and create two flowgraphs on top. One with the invoice table in realtime mode, the other one with the customers as realtime source.

Execute first flowgraph (realtime execution will remain running) and truncate the table as our second flowgraph will output the exact same record set. After the truncate execute the second flowgraph.

Now we insert new records into both tables on the Source System:

INSERT INTO RT_Invoices VALUES (4, 2, 7);

INSERT INTO RT_Invoices VALUES (5, 3, 10);

INSERT INTO RT_Customers VALUES (3, 'New Firm', 'IT');

Results in the Flowgraph Output Table contain the Invoice 5 twice.

So if you want to achieve the replication with using multiple realtime flowgraphs writing into the same table, it can be tricky and you need to make sure that the result does not contain multiple entries. If the table would e.g. contain Primary Key columns and the insert would fail, the flowgraph execution might get stopped due to an invalid insert (unique primary key constraint violated)

Kind Regards,

Timo

[Former Member · Accepted solution]

Thank you very much for the detailed answer.