Canonical item with same unique key duplicated

[former_member1031459]

Hi,

We are facing an issue with the datahub related to a Canonical item being duplicated, both on the database and on the target impex result.

We have a Canonical item with a very simple unique key. This item is composed of a Raw item from a SAP idoc that has many different segment definitions. On the database we see a Canonical item type with same integration key, but different canonicalItemId. One Canonical item row has incomplete, empty attribute values, while the other row has all attribute values correct. The problem is that there is a dependency between Canonical items. When using the 'resolve' expression, the Canonical item with the incomplete attribute values is resolved, resulting in a dependent Canonical item with empty values.

What we have discovered is when the number of Raw items on the database is greather than the datahub.composition.batch.size property value, the duplication of the Canonical item happens. It seems that on every iteration of a composition batch, the Canonical item cache get lost and an already existed Canonical item key is not found, so a new Canonical item is created with the same key.

We found when the datahub.composition.batch.size property value is reduced, we get more Canonical items duplicated, and when the value of the batch.size property is greather than the total raw items, the duplication does not happen, neither on the database nor on the impex file.

We are using Data Hub 5.3.0.0 with customized datahub extension, including the sapidocintegration and sapcoreconfiguration.

[rafal_gdowski]

It has been released to the artifactory here: http://repository.hybris.com/simple/dranura-release/com/hybris/datahub/datahub-webapp/5.4.0.1-RC1/ . However, the official patch is not yet available.

[Former Member]

Hi Rafal,

We are facing issue due to the above issue,i.e Canonical item with same unique key duplicated.

How can we disable this OOTB behaviour? Is there any way to delete the canonicalItem with "SUCCESS" status?

Any suggestion will be really helpful

Thanks, Shweta

[danpipe]

Hi David, we have also seen this issue occur. Do you have an ETA for 5.4.0.1?

[Former Member]

We have identified this as a bug that will be fixed in the 5.5 and 5.4.0.1 (patch) releases.

[Former Member]

Hi Fernando, This does indeed sound like a bug somewhere, we'll have to dig in a little further now to figure out where it is coming from. I have not been able to reproduce this behavior so far. Have you created custom composition handlers or changed the domain model? Or is this coming entirely from the ERP integrations as supplied by hybris/SAP? I think I'll probably need more information from you to look into this further, including some sample data that causes the problem, a list of all datahub extensions installed, and any custom model extensions/handlers. Maybe you can email me directly and I can post the answer here once we have resolution.

[former_member1031459]

Hi David, thanks for your answer.

All Canonical items on the canonicalitem table has the status column as SUCCESS, including the rows with the same duplicated integration key.

We have not defined a custom 'resolve' expression. We are using the default one used on the SpEL expression during target item creation.

[Former Member]

Hi Fernando, It is expected behavior that there are multiple canonical items created if some raw items related to that canonical item are in the first 'batch' (transaction), and others are in a subsequent transaction. Only one of these canonical items should have a composition status of 'SUCCESS'. This was the result of merging all previous canonical items with the same integration key (all previous have a status of ARCHIVED). Do you see multiple items with the same integration key and this SUCCESS status? Is the 'resolve' expression that you are referring to from a new extension that you have written and using during composition, or is it one used during target item creation from a SpEL expression?