Error In Extracting data from ODATA service in CPI DS for larger batch sizes

[former_member608605]

HI Experts,

We are consuming data from ODATA service in CPI DS where the interface works only if we put the batch size = 5 (Tasks take very large amount of time)but fails if i specify the larger batch size in the adapter options in ODATA datastore.

Below is the error:-

2019-06-04 15:39:41 (E)(3064:1056) RUN-58105 |Data flow RT_T_S4HANA_TO_IBP_KF_INVENTORYTURNS_RTFTEST_RT_DF_KF_INVENTORYTURNS Error preparing to read <TurnOverSet1>: <Exception occurs when executing 'begin' method of the adapter operation with ID [44]. no JSON format expected , or ]>. 2019-06-04 15:39:41 (E)(6548:9164) RUN-58105 |Data flow RT_T_S4HANA_TO_IBP_KF_INVENTORYTURNS_RTFTEST_RT_DF_KF_INVENTORYTURNS Error preparing to read <TurnOverSet1>: <Exception occurs when executing 'begin' method of the adapter operation with ID [44]. no JSON format expected , or ]>.

The interface worked only once or twice with Batch size = 200 (which is default.)

Please suggest.

[former_member608605]

Hi Jeff,

Please share your views on the above Query.

Thanks,

Shubham.

[former_member608605]

HI Jeff,

Thanks for your reply.

As suggested by you, we have made the below changes in the agent configuration tool

1. Set trace mode to True for OData Adapter

2. Increase the JVM max memory size (-Xmx256m to -Xmx1024m).

After running the task in CPI DS , we do not see any increased performance with the smaller batch size = 5 as now also it takes around 2 hours to process around 25000 records.

when we specify larger batch sizes like 50, 100 then the task just finishes after fetching between 50 to 500 records which is actually strange as it normally extracts thousands of records with batch size as 5.

Also the task fails with Batch size = 200 with the error as specified in the initial post.

Please suggest.

Thanks,

Shubham

[jeffrey_kresse]

I would recommend a few changes in the adapter config in the Agent Configuration tool.

1. Set trace mode to True for OData Adapter

2. Increase the JVM max memory size (-Xmx256m to -Xmx1024m).

If the difference is evident in some batch sizes and not others I could imagine that this is a performance/resource issue, so increase the available memory when batch sizes are larger.

If issue still occurs after that see if you can find some more detailed errors from the adapter trace file %DS_COMMON_DIR%/adapters/log/ODataAdapter_trace.txt and post the error here for follow up.

Thanks,

Jeff

[former_member608605]

HI Jeff,

Thanks for your reply.

Please confirm if the below changes are to be made in DSconfig file OR ODATA Adapter config file in the agent configuration?

1. Set trace mode to True for OData Adapter

2. Increase the JVM max memory size (-Xmx256m to -Xmx1024m).

Thanks,

Shubham

[jeffrey_kresse]

Both changes are made in the adapter configuration using the agent config tool.

Thanks,

Jeff