Skip to Content
author's profile photo Former Member
Former Member

Loading Primary Sales Histoiry - Strategy required for 7 crore records

Dear Gurus,

I need to load primary history data from legacy system based on oracle. The records could be around 7 crores. I would like to know the stragegy to be established before loading the data into BW. Please let me know the advantages/disadvantages of flat files over DB connect and db connect over flat files. It would be great if you can suggest me strategy to be followed before i start establishing the load procedure.

Thanks and Regards


Add a comment
10|10000 characters needed characters exceeded

Related questions

1 Answer

  • author's profile photo Former Member
    Former Member
    Posted on Oct 23, 2007 at 05:19 AM


    For this kind of load (70 million records), I would suggest that you stick with db connect. Some of the reasons for that are --

    1. Using flat file will be slower.

    2. There is an additional overhead of downloading the data into flat file and manipulating it.

    3. Debugging issues will be easier in db connect vs flat files.

    While coming up with some basic guidelines, please consider these.

    1. Divide the data into appropriate buckets for faster loads. This can be any field which makes sense like any time dimension.

    2. Try out some loads to figure out the optimum size.

    3. Since this is going to a one-time load, you can live up with some manual processes.

    4. Do not create very complicated views in Oracle to load. As much as possible load persistent data.

    In general, I have used db connect with oracle to load some big loads and I was quite ok with the load performance.


    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.