Skip to Content
0

pushdown the join between two different database tables

Oct 17, 2017 at 01:09 PM

69

avatar image

Hi ,

I have two source tables . Each table coming from different database system. ( One is from Oracle and another one is Sybase IQ). But join is not going to push down due to different source databases. But one table have large volume ( 10 + million) of data and second one I have less than 100 records. Due to not push down I need to read all 10+ million data to join.

Is there any way to push down the join between two different database tables.

Thanks & Regards,

Ramana.

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

1 Answer

Dirk Venken
Oct 17, 2017 at 01:13 PM
0

Copy the small table to the other database first (use a Data_Tranfer transform to do so). Then join.

Show 2 Share
10 |10000 characters needed characters left characters exceeded

Hi Dirk,

Thanks for your response. Large table is in operational database. Small table is one of the staging table.

Actually I am implementing the CDC logic. Table 1 has insert date & update date but table 2 doesn't have any dates.

But customer told that take the records based on Table 1 CDC.

In CDC flow I have loaded table 1 from Oracle to IQ using insert date & update date. In the next data flow I am joining Table 2 from Oracle with Table 1 from stage area ( IQ). Due to different databases it is not push down.

Thanks & Regards,

Ramana.

0
Venkata Ramana Paidi

That's not the best design then. You need some staging on Oracle for your table 1 CDC. Or don't materialise and just use a where-clasue (or a view) to filter out new and modified records only.

0