Self Tuning in Data Services 4.2

[Former Member]

Dear experts,

Regarding self tuning feature of Data Services which can be enable by turning on Collect Statistics for Optimization and Use Collected Statistics, I want to ask the procedure to clear all the statistics collected. Could you kindly help me?

Thank you very much.

Best regards,
Theo Tando

[former_member187605 · Accepted solution]

Disclaimer: officially unsupported!

Statistics are kept in the AL_CACHE and AL_CACHE_STATS repository tables. You can get rid of them by deleting all records from those tables.

[Former Member · Accepted solution]

Dear Venken,

Thanks for the answer, I have also just found a link that states the same:

Painlessly Changing ETL Jobs to Gather Statistics | IT Services | Mobile Technology | Enterprise Sol...

Okay, I will check the external factors. By the way, is it possible that the self tuning feature can improve the performance that much?

Thank you.

Best regards,

Theo Tando

[former_member187605 · Accepted solution]

It's only 25%, isn't it? Caching in memory vs. paging. Could be.

Anyway, what are your data flows' Cache Type property settings? The statistics are only taken into account when set to Pageable. You can always override by switching to In-Memory.

[Former Member · Accepted solution]

Dear Venken,

The dataflows are set to In-Memory Cache.

So, if the statistics is only used when the dataflows are pageable, meaning that the different performance are not caused by self tuning, since it is overridden by in-memory setting applied.

Let me check the trace log first then tell you the result.

Thanks Venken.

Best regards,

Theo Tando

[Former Member]

Dear experts,

I also want to ask about Data Services performance.

I have a job which has statistics like this:

1st run: 16 minutes 19 seconds

2nd run: 16 minutes 53 seconds

3rd run: 16 minutes 23 seconds

4th to x-th runs: approx 12 minutes

I want to ask why this can happen. I run the job in a server which has no other process running, and there is no performance tuning conducted.

I suspect that the self tuning feature is the reason behind this. But why did the improvement only occur at the 4-th run? Is there any other factor that can cause this performance improvement?

Thank you very much.

Best regards,

Theo Tando

[former_member187605]

What about your source and target datastores? Your network load?

I mean, there are external factors that may influence performance, too. Don't always blame it on DS .

Compare the monitor log output of 3rd and 4th run and try to narrow down, find out where the difference womes from.