How can I intercept a datahub error and retry the same data on a different item type?

[rahulverma94]

Hi Experts,

I have a complex use case scenario on which I request your help:

My requirement is:

We have 2 independent sets of item types in hybris - say MyProduct(extends Product) and MyVariant(extends VariantProduct) . Lets say there are 20 instances of MyProduct type and 5 instances of MyVariant item type in hybris.

Datahub has 2 target type - TargetMyProduct with exportCode MyProduct and the TargetMyVariant with export code MyVariant. Both these datahub item types process same set of product attributes as both of them eventually extends product item type in hierarchy.

While publication, each of these target item types in datahub will process (20+5=25) rows each and throw errors for type mismatch (5 unresolved errors for TargetMyProduct and 20 for TargetMyVariant publication).

Is there a way for datahub to know beforehand the itemtype instances in hybris so that it can filter and send to datahub adapter only the eligible rows for each of the target item types?

Thanks in advance! Rahul

[Slava]

Rahul,

if possible to separate those items into different raw and canonical types, I would go with Markus's recommendation as it's simpler. However, if it's not possible, here is another way to deal with it. You can create a custom Publication Grouping Handler. Inside that handler you have access to the canonical item type and to the target item type (through the TargetItemCreationContext passed into the handler). So, you could filter out CanonicalMyVariant items, if TargetMyProduct is being created, and filter out CanonicalMyProduct items, if TargetMyVariant, is being created.

[rahulverma94]

The question is how do I separate 2 types or apply filter on them, as the logic to filter the items(whether instance of that item already exists in hybris) is only known to hybris and not to datahub. Does that make sense?

[Slava]

1. DataHub knows itemtype in hybris but does not know instances of that type in hybris.

2. It's very unusual requirement for DataHub to publish item only once to hybris and, if it already exists, stop publishing. Usually DataHub synchronizes changes, so if the item has changed in the source system, that change will be propagated to hybris. In this rare case I don't see a way to filter out the item from publication without querying the hybris platform. Check whether there is a REST API in hybris already for checking whether the item exists or maybe be you will need to create a custom extension for that, but the approach is the same: use publication grouping handler, which queries hybris and makes the decision. It will slow down the publication significantly. So think, maybe something can be done on hybris side to ignore published existing items.

3. If knowing whether an item has been published or not is sufficient (rather than does it exist in hybris), then the publication grouping handler can check whether a publication status with status SUCCESS already exists for an item with that integration key. This is going to be much faster than querying hybris.

[rahulverma94]

For your point 3, could you help me with any Datahub service or method which can be used to verify the last publication status for an item in publication handler?

[Slava]

Yes, you can use CanonicalItem.getPublicationStatuses() method, which returns a Set<CanonicalItemPublicationStatus>. From CanonicalItemPublicationStatus you can read getStatus() to determine whether the publication was successful and getTargetSystemPublication().getTargetSystem() to know the target system, to which the item was published.

[mpern]

Is there any way to split the data / filter the data in datahub? So you publish the data with the correct type in the first place and don't need to fail the publication?

EDIT 2019-03-14

There are various ways to achieve that:

filter canonical items during publication using a SpEL expressionn

https://help.hybris.com/1811/hcd/d628a2d51ddc48788e1ace0fed427689.html

or, if you need more power, use any of the grouping, composition or publication grouping handlers

https://help.hybris.com/1811/hcd/5a653eb985934e3081c9d163dd430f12.html

[rahulverma94]

That's exactly what I am looking for but not sure how to achieve it.

[Slava]

I guess the problem arises from the fact that canonical items for both types are composed from the same raw items and/or target items are created from the same canonical items. I think what Markus suggests is to separate them from the raw (in the source system). So, the MyProducts would be loaded into RawMyProduct -> CanonicalMyProduct -> TargetMyProduct; the variants respectively will be loaded to RawMyVariant -> CanonicalMyVariant -> TargetMyVariant. This way products never contain variant items and variants never contain product items.

In other words, we treat them as completely independent types inside DataHub instead of as a type hierarchy.