Skip to Content

SAP IQ 16 and Hadoop

Hello,

Using SAP IQ 16 and Hadoop independently, the need is to move data from some tables in IQ to Hadoop periodically. i.e like using "incremental data loads from IQ to Hadoop".

An idea is to extract the data from IQ to Unix flat files an then process them with "awk" in order to load the difference between the last extraction file and the current into Hadoop.

Do you know if there is any tool that can do something like this ?

Thank you

Regards

Jose

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

3 Answers

  • Oct 24, 2016 at 02:03 AM

    Do you mean iqunload?

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 24, 2016 at 12:19 PM

    Hi Jose,

    To write query results to files (IQ serer side), you may use the TEMP_EXTRACT_* set of database options

    [http://help.sap.com/saphelp_iq1608_iqrefso/helpdata/en/a6/2adda484f21015b5a6aea48ec128cf/content.htm]

    HTH

    - Volker

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 24, 2016 at 01:38 PM

    Hello Guys,

    Thank you. I found info on Apache Sqoop which might do the job, because we need to extract only the data ( from user tables) which has changed since the last extraction date.( that's why the analogy of an "incremental backup", but in this case just data from user tables).

    Regards

    Jose

    Add comment
    10|10000 characters needed characters exceeded