Datahub cleanup configuration with SAP ERP replication

[eduard_unruh]

Hi everyone,

we are using hybris/datahub version 5.7.0.15 and want to integrate the datahub cleanup extension to remove unused data from the database. Our scenario is to replicate data from SAP ERP to hybris (using the sapproduct-extensions for that).

These are the possibilities to configure the cleanup extension:

 datahub.cleanup.rawitem.enabled=true
 datahub.cleanup.canonicalitem.enabled=true
 datahub.cleanup.targetitem.enabled=true
 datahub.cleanup.publisheditems.enabled=true
 
 datahub.cleanup.canonicalitem.time.delay=0

Are there any recommendations how to set this properties in our scenario? Can switch everything to true? Should we increase the delay for canonical item deletion?

Thanks in advance!

[eduard_unruh · Accepted solution]

We ended up with this configuration now:

 datahub.cleanup.rawitem.enabled=true
 datahub.cleanup.canonicalitem.enabled=true
 datahub.cleanup.targetitem.enabled=true
 datahub.cleanup.publisheditems.enabled=false

Setting datahub.cleanup.publisheditems.enabled to true still leads to problems in datahub.

[former_member224482]

Never delete canonical item with SUCCESS status.
The Datahub cleanup has been developed after the SAP ERP replication was shipped and therefore never tested.

Deleting canonical item with SUCCESS status breaks the basic assumptions of the data hub's normal operation.

[eduard_unruh]

I agree with you: never delete SUCCESS-Items because SAP extensions look into that items to resolve publications.

As I understand the cleanup extension, it only deletes CanonicalItems of the status ARCHIVED, as described here: https://wiki.hybris.com/display/release5/Automated+Elimination+of+Data+Hub+Auditing+Related+Database...

[former_member224482]

I recall the cleanup extension deleting the canonical item with SUCCESS status if the canonical item was published. This cause data inconsistency which leads to error when using resolve publications.

[former_member1300987]

Setting datahub.cleanup.canonicalitem.enabled=true will delete all canonical items in status ARCHIVED.

Setting datahub.cleanup.publisheditems.enabled=true will delete canonical items in status SUCCESS.

To use the latter property should be carefully considered, because in many scenarios, the presence of canonical items is required to be able to resolve values upon publication. For the standard SAP ERP scenarios, I would recommend using just

datahub.cleanup.rawitem.enabled=true datahub.cleanup.canonicalitem.enabled=true datahub.cleanup.targetitem.enabled=true