Skip to Content
author's profile photo Former Member
Former Member

Datahub impex generation is not in sequence?

Hi All,

When i am seeding data to hybris via datahub backoffice quick upload the sequence in which output impex generated is not the same the input is provided.

Example:

Input Data

 categoryId,categoryDescription,productHierarchy,isDeletedFlag
 "A","AAAAA","",""
 "B","BBBBB","",""
 "C","CCCCC","",""
 "D","DDDD","",""
 
 Output 1:
 
 INSERT_UPDATE Category;;catalogVersion(Catalog(id),version)[unique=true];name;description;supercategories(code,catalogVersion(catalog(id),version));code[unique=true]
 ;1;ProductCatalog:Staged;BBBBB;BBBBB;<ignore>;B
 ;4;ProductCatalog:Staged;AAAAA;AAAAA;<ignore>;A
 ;3;ProductCatalog:Staged;DDDD;DDDD;<ignore>;D
 ;2;ProductCatalog:Staged;CCCCC;CCCCC;<ignore>;C
 
 Output 2:
 
 INSERT_UPDATE Category;;catalogVersion(Catalog(id),version)[unique=true];name;description;supercategories(code,catalogVersion(catalog(id),version));code[unique=true]
 ;7;ProductCatalog:Staged;AAAAA;AAAAA;<ignore>;A
 ;8;ProductCatalog:Staged;BBBBB;BBBBB;<ignore>;B
 ;6;ProductCatalog:Staged;CCCCC;CCCCC;<ignore>;C
 ;5;ProductCatalog:Staged;DDDD;DDDD;<ignore>;D
 
 Output 3
 
 INSERT_UPDATE Category;;catalogVersion(Catalog(id),version)[unique=true];name;description;supercategories(code,catalogVersion(catalog(id),version));code[unique=true]
 ;17;ProductCatalog:Staged;AAAAA;AAAAA;<ignore>;A
 ;15;ProductCatalog:Staged;DDDD;DDDD;<ignore>;D
 ;19;ProductCatalog:Staged;BBBBB;BBBBB;<ignore>;B
 ;13;ProductCatalog:Staged;CCCCC;CCCCC;<ignore>;C

Could someone please help us how to get impex generated in the same sequence as input is provided?

Thanks,

G Gopinath

Add a comment
10|10000 characters needed characters exceeded

Related questions

1 Answer

  • Posted on Jun 27, 2019 at 06:55 PM

    Short answer it's not possible..DataHub does not maintain sequence of the input records.

    Each record loaded represents an item and we do not support dependency between items of the same type, e.g. if we have Category A and Category B - they are independent of each other and therefore it does not matter in which order they are loaded. That allows us to perform concurrent data processing in parallel threads under the hood and speed up the loading/composition/publication processes. Sure the output looks prettier but the price is 10 multiples slower application.

    However there are sometimes cases when dependency exists between items of different types, e.g. Unit must be created before Product; or even between items of the same type in cases of self-references, e.g. Category C may have Category A and Category B as its supercategories. In the former case we offer declaration of dependency between target item types:

     <item>
         <type>DependentCategory</type>
         ...
         <dependencies>
             <dependency>ApparelDependentProduct</dependency>
         </dependencies>
         ...
     </item>
    

    which guarantees correct order of their impex blocks in the impex file.

    For the self-reference case a custom GroupingHandler should be created to correctly split raw item together and then compose an item with self-references based on those groups

    Hope this helps

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.