Publications status is "in-progress" for few data rows

[former_member1181677]

Hi All, We are using Data Hub 5.5 and have created Custom Target Adaptor which will convert data into “|” delimiter message. It is working fine with one input raw feed. In load test with multiple feeds, few rows in publicationaction table are “success” status rest are “In_progress” and number of rows in both publicationaction table and RawItem table are equal . So I went little bit deeper and found that, “In_progress” data rows are not going to Custom Target Adaptor. I am not sure why these rows are not processed through Custom Target Adaptor. Any idea on this

Let me know if you need any more information.

[former_member1181677]

I went little deeper and found root cause. In our custom Target Adaptor below if conditions is failing for some items. Because targetSystemPublication.getTargetItems() is return empty.

**public void publish(final TargetSystemPublication targetSystemPublication, final String serverURL) throws PublicationException { Preconditions.checkArgument((targetSystemPublication != null) && (targetSystemPublication.getTargetSystem().getTargetSystemType().equals(getTargetSystemType())));

     if ((targetSystemPublication.getTargetItems() != null && targetSystemPublication.getTargetItems().size() > 0))
     {
         final TargetSystem targetSystem = targetSystemPublication.getTargetSystem();
         final List<PizzaHutFileBlock> contentList = createFileBlocksToExport(targetSystemPublication, serverURL);
         
         final List<String> exportableFile = fileWithDataCallBacks(contentList, targetSystem);
         targetSystemPublication(targetSystemPublication, exportableFile, serverURL);
     }**
 }

Any idea ?

[rafal_gdowski]

The only thing I can think of is that your target items are created in a transaction, and if that's not flushed, you may not see the target items yet. So I'd make sure that the first transaction is done before you publish.

[former_member1181677]

agree, we have implemented publication logic in Target adapter. That's why we see some items are in "success" status in DB. The problem is custom adapter is not picking up. Any idea ?

But is working if we sent only one raw data, failed in case of load test.

[rafal_gdowski]

I am not sure what SUCCESS status you are talking about. When target items are created, there is an instance of CanonicalItemPublicationStatus created with a status of IN_PROGRESS. Something has to update that status to SUCCESS. The Data Hub will do that only when the hole TargetSystemPublication is set to SUCCESS (so something in your adapter has to do that).

Good Luck.

[rafal_gdowski]

If that's the case, then your custom adapter needs to 'pick up' those target items and publish them to the target system. In the sdk, the following method is avaliable:

 /**
      * Finds target items of a specified type for publish Execution
      *
      * @param publicationId id of TargetSystemPublication
      * @param targetItemClass - type of {@link com.hybris.datahub.model.TargetItem}
      * @param pageable - definition of which page of results to return
      * @return - {@link java.util.List} of matching sub-types of {@link com.hybris.datahub.model.TargetItem}
      */
     <T extends TargetItem> DataHubPage<T> findByPublication(final Long publicationId, final Class<T> targetItemClass, final DataHubIdBasedPageable pageable);
 

Once you get all of the items, your adapter will have to implement the publication logic. There is no magic there :)

[former_member1181677]

agree, we have implemented publication logic in Target adapter. That's why we see some items are in "success" status in DB. The problem is custom adapter is not picking up. Any idea ?

But is working if we sent only one raw data, failed in case of load test.

[former_member1181677]

Thanks for you response. yes we have specifying target system and pool. Target items are created only for "Success" items, for rest of items it is not creating in DB

[rafal_gdowski]

Are you specifying in your publication REST call what target systems to publish to?

 {"poolName":"GLOBAL",
   "targetSystemPublications":[{"targetSystemName":"myCustomTargetSystem"}]}

I would also verify in the db whether the target items are created. If they are, then it is probably some issue with your adapter.

[former_member1181677]

Thanks for you response. yes we have specifying target system and pool. Target items are created only for "Success" items, for rest of items it is not creating in DB