Skip to Content

Need to increase the Row Count in tables within a HANA Schema (for a POC)

Hello All,

I am currently part of a POC related to HANA-Sybase Integration.

We need huge amount of data to be loaded into some HANA tables(in fact 5 tables shown below).

Currently the record count in the tables are:

DIVIDEND_EVENT 387

SPLIT_EVENT 11005

STOCK_HISTORY 2840000

STOCK_QUOTE 11548892

STOCK_TRADE 5775442

The data needs to be increase proportionally across the 5 tables to ~2 billion rows in the STOCK_TRADE table.

All the tables have a common object between them, which is INSTRUMENT_ID(Commodity Market scenario)

Another constraint here is the data must be consistent (referential integrity to be maintained) between the INSTRUMENT_ID of the tables.

The current record count is related to year 2005. When I increase the data set, I need to uniformly distribute it across 10 years(2005-2015)

I wanted your help in deciding a Best/recommended approach by which we can load such massive meaningful dummy data into HANA tables.

Has anyone faced a similar situation somewhere?

PS: The above shown records were loaded though various CSV files into HANA by some other colleagues of mine, couple of years ago.

These CSV files are readily available with me.

Some thoughts from our side on the solution side: Excel formulas/Macros- Import/Export CSV Files; SQL procedure; Scripting at HANA Linux side..

BR

Prabhith

Add a comment
10|10000 characters needed characters exceeded

Assigned Tags

Related questions

1 Answer

  • Posted on May 14, 2015 at 07:39 PM

    Hello Prabhith,

    I am not familiarized with HANA SQL but I have few years of development and migration experience.

    If I were you I would write similar INSERT SQL statements like below for all tables to increase the number of rows.

    You can add YEAR * 1000000 to INSTRUMENT_ID in order not to make ID duplicate. You can decide number of 0s according to the length of field..

    INSERT INTO TABLE DIVIDEND_EVENT (INSTRUMENT_ID, YEAR , FIELD3 )

    ( SELECT INSTRUMENT_ID + 200600000 , 2006, FIELD3 FROM DIVIDEND_EVENT WHERE YEAR = 2005)

    INSERT INTO TABLE DIVIDEND_EVENT (INSTRUMENT_ID, YEAR , FIELD3 )

    ( SELECT INSTRUMENT_ID + 200700000 , 2007, FIELD3 FROM DIVIDEND_EVENT WHERE YEAR = 2005)

    INSERT INTO TABLE DIVIDEND_EVENT (INSTRUMENT_ID, YEAR , FIELD3 )

    ( SELECT INSTRUMENT_ID + 200800000 , 2008, FIELD3 FROM DIVIDEND_EVENT WHERE YEAR = 2005)

    ... an for the other years.

    Similar INSERTS for other SPLIT_EVENT, STOCK_HISTORY, STOCK_QUOTE, STOCK_TRADE tables, too.

    Hope this helps.

    Using excel is tiring and time consuming in my opinion and experience.

    Whenever there are alotof records in an excel file and you are using macros, formulas you need big memory and alotof time.

    Regards,

    Yuksel AKCINAR

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.