cancel
Showing results for 
Search instead for 
Did you mean: 

RawItems not being mapped to Target Item

former_member583621
Participant
0 Kudos

We are working with Hybris Data Hub Extension SDK 6.2.0.4-RC1 and trying to create an extension to map fields from the MATMAS iDoc into ItalikaProduct which extends from Product. We followed the guide from here to get started although we noted some inconsistencies in the guide with the correct configuration. Also we read the data structure for Raw Item, Canonical Item and Target Item.

But the field we are trying to map isn't not being reflected into our desired target.

I share our extension xml files:

zidocpcmmaper-raw-datahub-extension.xml

 <extension xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.hybris.com/schema/"
            xsi:schemaLocation="http://www.hybris.com/schema/ http://www.hybris.com/schema/datahub-metadata-schema-1.3.0.xsd";
            name="zidocpcmmapper-raw">
 
     <dependencies>
         <dependency>
               <extension>saperpproduct-canonical</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-raw</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-target</extension>
             </dependency>
             <dependency>
                <extension>zidocpcmmapper-canonical</extension>
             </dependency>
     </dependencies>
 
     <rawItems>
         <item>
             <type>RawMATMAS</type>
             <attributes>
                 <attribute>
                     <name>E1MARAM-MATNR</name>
                 </attribute>
             </attributes>
         </item>
     </rawItems>
 
     <canonicalItems>
         <item>
             <type>CanonicalProduct</type>
             <attributes>
                 <attribute>
                     <name>ensamblikasapnumber</name>
                     <transformations>
                         <transformation>
                             <rawSource>RawMATMAS</rawSource>
                             <expression>E1MARAM-MATNR</expression>
                         </transformation>
                     </transformations>
                 </attribute>
             </attributes>
         </item>
     </canonicalItems>
 </extension>

zidocpcmmapper-canonical-datahub-extension.xml

 <extension xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.hybris.com/schema/"
            xsi:schemaLocation="http://www.hybris.com/schema/ http://www.hybris.com/schema/datahub-metadata-schema-1.3.0.xsd";
            name="zidocpcmmapper-canonical">

     <dependencies>
         <dependency>
               <extension>saperpproduct-canonical</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-raw</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-target</extension>
             </dependency>
     </dependencies>
 
     <canonicalItems>
         <item>
             <type>CanonicalProduct</type>
             <attributes>
                 <attribute>
                     <name>ensamblikasapnumber</name>
                     <model>
                         <localizable>false</localizable>
                         <collection>false</collection>
                         <type>String</type>
                         <primaryKey>false</primaryKey>
                     </model>
                 </attribute>
 
             </attributes>
         </item>
     </canonicalItems>
 
 </extension>

zidocpcmmapper-target-datahub-extension.xml

 <extension xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.hybris.com/schema/"
            xsi:schemaLocation="http://www.hybris.com/schema/ http://www.hybris.com/schema/datahub-metadata-schema-1.3.0.xsd";
            name="zidocpcmmapper-target">
 

     <dependencies>
         <dependency>
               <extension>saperpproduct-canonical</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-raw</extension>
             </dependency>
             <dependency>
                <extension>saperpproduct-target</extension>
             </dependency>
             <dependency>
                <extension>zidocpcmmapper-canonical</extension>
             </dependency>
     </dependencies>
 
     <targetSystems>
         <targetSystem>
             <name>HybrisCore</name>
             <type>HybrisCore</type>
             <exportURL>${datahub.extension.exportURL}</exportURL>
             <userName>${datahub.extension.userName}</userName>
             <password>${datahub.extension.password}</password>
             <exportCodes>
                 <exportCode>#% impex.setLocale( Locale.SPANISH )</exportCode>
             </exportCodes>
             <targetItems>
                 <item>
                     <type>ItalikaProduct</type>
                     <exportCode>ItalikaProduct</exportCode>
                     <description>Italika Products</description>
                     <updatable>true</updatable>
                     <canonicalItemSource>CanonicalProduct</canonicalItemSource>
                     <status></status>
                     <attributes>
                         <attribute>
                             <name>ensamblikaSAPNumber</name>
                             <localizable>false</localizable>
                             <collection>false</collection>
                             <transformationExpression>ensamblikasapnumber</transformationExpression>
                             <exportCode>ensamblikaSAPNumber</exportCode>
                             <mandatoryInHeader>false</mandatoryInHeader>
                         </attribute>
                     </attributes>
                 </item>
             </targetItems>
         </targetSystem>
     </targetSystems>
 </extension>

The field of the item type we are trying to map to looks like this:

 <itemtype code="ItalikaProduct" extends="Product"
                   autocreate="true" generate="true"
                   jaloclass="de.hybris.platform.yacceleratorcore.jalo.ItalikaProduct">
             <description>italika product v1</description>
 <attribute qualifier="ensamblikaSAPNumber" type="java.lang.String">
   <description>Ensamblica SAP Number</description>
   <modifiers/>
   <persistence type="property"/>
 </attribute>
 .
 .
 .                                                                                                                                                          
 </itemtype>    

The ItalikaProduct object is being generated and can be visualized in the apparel view, all of the other fields are displayed but the one we want to map isn't shown.

In our logs we have found some errors which I attach in PDF file.link text

Given our current extension xmls, what could be the cause that prevents the field from being mapped? Is our target extension xml correct? Is our understanding of how to configure a target item correct?

EDIT 04-11-2017:

I'm attaching the responde from the API calls for the raw item 19540 /datahub-webapp/v1/raw-items/19540 and one of its canonical items 25553 /datahub-webapp/v1/canonical-items/25553, I notice that in our ensamblikasapnumber attribute, no value is present, does that mean the mapping is not being done? API responses

Accepted Solutions (1)

Accepted Solutions (1)

former_member583621
Participant
0 Kudos

In the end we tested altering the saperpproduct extensions (canonical, raw and target) by replacing the export codes with value of Product to ItalikaProduct in the target extension and also added the additional attributes to canonical extension and the transformations in the raw, and only then the mappings we wanted got reflected on PCM, we have no idea why our custom extension wasn't used by DataHub at all.

We cleaned the database and tried again with our extension but the items that arrived at PCM were always Product type and not ItalikaProduct type, the CanonicalProductSalesAreaToCatalogMapping errors didn't seem to be part of the problem.

At this date we have no clue why a custom extension didn't work as explained by the guide.

So my recommendation to anyone having this kind of issue is to just directly alter the saperpproduct jars' extension xmls to save a lot of trouble, though I cannot garantee this workaround doesn't come with any drawbacks.

Answers (3)

Answers (3)

former_member224482
Active Contributor
0 Kudos

Look at the documentation : Making Settings in Backoffice for Product Master Data Replication

All related errors refers to missing CanonicalProductSalesAreaToCatalogMapping which is uploaded by the Backoffice.

0 Kudos

In hybris please configure sale area,catalog mapping etc. Upload meta data/config from Hybris to datahub.

former_member583621
Participant
0 Kudos

We are new to Hybris and Data Hub, could you elaborate on your answer? How can we validate if the configuration is done correctly?

former_member583621
Participant
0 Kudos

We noticed something, the development we're doing has to done in another Data Hub server, where we loaded our extension and noticed when Tomcat restarts and loads out extension it shows more information, I'm attaching a word file with an example of it.Example of log

The bold text is something we don't see in the other Data Hub server, is it because of missing configuration or is it because one server has a configuration to show more information on logs?

Slava
Advisor
Advisor
0 Kudos

You probably share the database between the DataHubs. That EXISTING_ITEM is normal. It tells you that the metadata you're loading already exists in the DB. Primarily it was done to let you know that you're accidentally overriding a type defined in a different extension. So, even if you restart first DataHub with autoInitMode=ignore, you will see the same message

former_member583621
Participant
0 Kudos

Both Data Hub servers are independent from each other and don't share any infracstructure, what I wanted to point out is that the bold text is something that is not present in the first server we tested, especially the parts where it says Loading canonical item Item and Loading canonical attribute Attribute, if those are ommited in our first Data Hub server, is it due to an error? or is it normal if those are ommited?

Slava
Advisor
Advisor
0 Kudos

Normally you don't see those messages when 1) DataHub is started for the first time 2) DataHub is restarted in drop-create autoInitMode. If DataHub is restarted in any other autoInitMode with the same extensions it had before, those messages are normal.

former_member583621
Participant
0 Kudos

I've checked the local.properties on the deployed webapp and kernel.autoInitMode is set to ignore, and but when you say DataHub is started for the first time, you mean every time the Tomcat service is restarted or just the first time DataHub is run?

Slava
Advisor
Advisor
0 Kudos

First time DataHub is run with that extension(s)

former_member583621
Participant
0 Kudos

I attach the image from backoffice where the sales area mappings are, the Country Tax Class on all of the entries are set to null, I didn't do this configuration nor I'm familiar with it, but is this the cause of the error?

Slava
Advisor
Advisor
0 Kudos

I'm not familiar with it either. Team Griffin should be able to help or somebody who have had to deal with this configuration. But I think once it's configured and pushed to DataHub the composition will succeed.

former_member583621
Participant
0 Kudos

I just pushed to Data Hub the configuration from the above table and before that the last push was made on March 30th which was 3 days before we started to work on the extension.

I'm told the null country class is not necessary unless we're sending prices too, which is not our case, still we haven't been able to pinpoint the real cause of our problem.

Still waiting response on the table database. schemaless_attributes to check the target items.

Slava
Advisor
Advisor
0 Kudos

I don't think target items matter at this point because we saw resolve() function did not work as expected during exposition. And it did not work because CanonicalProductSalesAreaToCatalogMapping did not contain master records for the CanonicalProduct composed. Think like this: primary key of CanonicalProduct is a foreign key to CanonicalProductSalesAreaToCatalogMapping. resolve() finds matching record in the master table and gets a field from that value. In your case there was no record in CanonicalProductSalesAreaToCatalogMapping matching primary key of the CanonicalProduct.

former_member583621
Participant
0 Kudos

I just noticed, in our iDoc, the sales organization is sent as IT01, and in backoffice the sales organization is saved as it01, could it be the change from upper case to lower case be the problem? Sales organization and distribution channel are the primary keys for CanonicalProductSalesAreaToCatalogMapping, and are also part of CanonicalProductSales.

Slava
Advisor
Advisor
0 Kudos

That would be a problem because resolve() is case sensitive

Slava
Advisor
Advisor
0 Kudos

Augustin,

I'm not really familiar with saperpproduct extensions, so I assume the following is true:

  • RawMATMAS and its attribute E1MARAM-MATNR are defined in saperpproduct-raw extension.

  • CanonicalProduct is a type defined in saperpproduct-canonical extension

  • Attribute ensamblikasapnumber does not exist in the CanonicalProduct defined by the SAP extension - you're adding this attribute to an already existing type.

  • ItalikaProduct does not exist in saperpproduct-target extension - you're creating a new target item type in your extension.

That said, I don't see obvious problems with your models, so I can't give you a definite answer but I can give you ideas how to troubleshoot this.

1) Make sure your extensions are successfully loaded and item types are defined as you expect: - After DataHub startup check the log for something like "Failed to load extension..." - make sure all your extensions are loaded. - Also, check in the log for messages similar to "ValidationException{failures=[ValidationFailure{ Note: Only ValidationFailure{FATAL - indicate problems with the models. Other than FATAL failures are not really failures but warnings. However, they may indicate that some of the attribute definitions you expect to have have failed to load. - Run these GET ReST requests to see what attributes exist in the item types inside DataHub:

```

 # Shows all attributes in the canonical type
 /datahub-webapp/v1/item-classes/canonical/item-types/CanonicalProduct/attributes
 # Shows details of a specific attribute
 /datahub-webapp/v1/item-classes/canonical/item-types/CanonicalProduct/attributes/ensamblikasapnumber

``` Unfortunately there is no way to check target items model.

2) Load raw items to DataHub. Check whether the loaded raw items contain E1MARAM-MATNR values.

3) Run compose. Check whether composed CanonicalProducts have ensamblikasapnumber attribute populated.

4) Run publication. Examine created target items and check whether their ensamblikaSAPNumber attributes are populated.

Hopefully, that will help you to find the phase at which attributes are not populated. Also, once you found that attributes are not there, check the log output to see, if there are exceptions related to item creation/attribute population. Depending on what you find, further actions can be planned.

Also, not related to your problem but you can simplify your extensions a bit:

  • There is no need to define RawMATMAS, if you not changing the type.

  • In your raw and target extensions, you don't need dependency other than dependency on szidocpcmmapper-canonical, which already defines dependency on all other sap extensions.

This is not important or critical though.

former_member583621
Participant
0 Kudos

Thanks for the answer, I have updated my question and attached a pdf with the errors that are being thrown in our Tomcat server.

From your suggestions, we can confirm our extensions are being loaded, no failed to load messages. We have yet to try the GET requests, but I'll inform you once we have something.

Since we are new to this, can you refer us to how to check your points 2), 3) and 4) ? Thanks. If you need more information about our case, we can provide it.

Slava
Advisor
Advisor
0 Kudos

For 2) and 3) refer to Raw Item and Canonical Items in https://help.hybris.com/6.2.0/hcd/3a24eb6baa18405cb2a16ea5dc06e676.html.

For 3, the only way is to check TargetItem table in the database. schemaless_attributes is cryptic but you still should see names of the attributes present.

Download Hybris Test Adapter from the Solution Book. It captures generated impex, so you can evaluate whether it makes sense.

Use the smallest data set possible and datahub.autoInitMode=drop-create for trouble shooting.

Slava
Advisor
Advisor
0 Kudos

The log indicates that CanonicalProductSalesAreaToCatalogMapping cannot be resolved for the CanonicalProduct, from which the target item is being created. See https://help.hybris.com/6.2.0/hcd/ea393309fe6f40eebcec24ec79950f73.html to understand how resolve works.

former_member583621
Participant
0 Kudos

Hi Slava, the first link you shared throws 404, the Data Hub server is located in a different place so we connect to it through VPN, so there's a limit to what we can access and see.

From the guide I shared, we tried to connect to /datahub-webapp/v1/item-classes/raw/item-types/RawMATMAS/attributes/ and /datahub-webapp/v1/item-classes/canonical/item-types/CanonicalProduct/attributes, both with localhost and the ip address to our Data Hub server, to check our extensions but we were unable to reach it.

Slava
Advisor
Advisor
0 Kudos

Just remove the "." after .html in the URL - it got accidentally pasted

former_member583621
Participant
0 Kudos

What we find weird is that CanonicalProductSalesAreaToCatalogMapping is not a canonical item we defined, or is it caused by our configuration? I read the article you linked but how am I to interpret it? Should I set in my tranformationExpression the following in my target item?:

resolve('CanonicalProduct')?.ensamblikasapnumber

Slava
Advisor
Advisor
0 Kudos

That type is part of the saperpproduct configuration. As Trilok indicated you have to log in to ECP as an admin and configure sales area, etc. in there. That configuration data must be pushed to DataHub, only then publication will work correctly. Maybe this link will help you https://help.hybris.com/6.2.0/hcd/efc4b188caff4ab38b207256de08e8af.html

former_member583621
Participant
0 Kudos

I looked at the deployed war in Data Hub server and the listed jars from the link you shared are present:

  • saperpproduct-raw

  • saperpproduct-canonical

  • saperpproduct-target

  • product-canonical Data Hub Extension

  • sapidocintegration

  • sapcoreconfiguration

I was told the configuration and sync were ready but as I mentioned I'm new to developing Data Hub extensions so I don't know if this error is due to a misconfiguration.

Is the CanonicalProductSalesAreaToCatalogMapping error something due to our extension or due to Data Hub configuration?

Slava
Advisor
Advisor
0 Kudos

It's due to misconfiguration. I'm not really familiar with SAP extensions but I know their ECP extensions and DataHub extensions share the same configuration. For that reason it should be done on the ECP side and then pushed to DataHub. Apparently it was not done in your case. I'm sure somebody from Team Griffin would help you in no time. Also, try to search this forum for an already existing answer. I'm sure you're not the first one who ran into this problem.

former_member583621
Participant
0 Kudos

Thanks for the tips, yeah we're been searching for related answers but nothing has come up that could shed some light into this, even now we're still unsure of what does the exportcode and type fields in the extension.xml means as we've seen different implementations and we're still learning all about this kind of development.

former_member583621
Participant
0 Kudos

On the other hand, I've already requested for the results from the Rest API calls and waiting for their reply, I'll post the results from the endpoints /datahub-webapp/v1/item-classes/raw/item-types/RawMATMAS/attributes/ and /datahub-webapp/v1/item-classes/canonical/item-types/CanonicalSalesProduct/attributes/ as soon as I can

former_member583621
Participant
0 Kudos

I got the results from checking the Rest APIs:

  • for /datahub-webapp/v1/item-classes/raw/item-types/RawMATMAS/attributes/ we can confirm E1MARAM-MATNR exists,

  • and for /datahub-webapp/v1/item-classes/canonical/item-types/CanonicalProduct/attributes/ we can confirm ensamblikasapnumber was added to CanonicalProduct

I tried the following for loading the raw items GET /datahub-webapp/v1/raw-items/RawMATMAS and this for running composition GET /datahub-webapp/v1/compositions but on both I got HTTP 404 Not Found. How can I know the item id for RawMATMAS and how to run compose?

Slava
Advisor
Advisor
0 Kudos

, I wonder if you can help with this what appears to be a configuration issue?

As to the exportcodes, think about them as hints, which help to adjust/fine tune impex headers for the published data. Don't worry about them until your publication will start working but items are rejected by the ECP.