Skip to Content
author's profile photo Former Member
Former Member

Flat file performance issue

I would need to load a flat file into a cube, it has about 600,000 records. Is that too many records to be loaded from one flat file into a cube, if it is not, what other alternatives do I have.

Rashmi

Add a comment
10|10000 characters needed characters exceeded

Related questions

5 Answers

  • Best Answer
    Posted on Sep 22, 2007 at 09:17 AM

    Rashmi,

    600,000 records is not very big - depending on your server.

    Also in the performance tab of the cube - click on the create indexex ( Batch ) and then select drop indexes on load and rebuild indexes after completion of load - this will speed up your data loads.

    Arun

    P.S we routinly load more than 10 million records into our BI system everyday into DSO's and Cubes

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Sep 21, 2007 at 06:55 PM

    There are few things that u can do to improve data load performance from a flat file

    1) Parallel loading : Try segregating the data into multiple files and load them using different InfoPackages

    2) Loading data from Application server, rather than client (i think u must be doing this)

    -kiran

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Sep 21, 2007 at 08:19 PM

    Keep ASCII format. SAP recommends ASCII is more efficient than CSV.

    Ravi Thothadri

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Sep 22, 2007 at 09:21 AM

    600000- is not a big amount to load into BW.

    1. Load from Application Server.

    2. try load using ASCII file which will improve performence of data load compare to CSV.

    Nagesh Ganisetti.

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Sep 22, 2007 at 09:50 AM

    hi,

    most probably u can load that much of record's in to an cube. one cube can

    contain maximum of 100GB records.

    otherwise,

    you can load half part of the record in to one cube, another part of record in to

    other cube with primary key(same key)

    using multiprovider you can mingle these records in to your query.

    if helpful provide points

    regards

    harikrishna.N

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.