Model Compare as a new feature in PA 2.2

[former_member184566]

Intro

PA 2.2 has been released, I have recently installed and seen a nice new feature called model compare that exists in the expert side of predictive.

Model Compare

Basically this feature tells you which algorithm has the strongest KI and KR. The algorithm that has the best KI and KR score should be the most accurate. We must remember that KI is a measurement of predictive power while KR is the measurement prediction confidence. The closer these are to 1 the stronger the model is.

Below is an example where data is run through three different algorithms, once run the algorithm with the highest KI and KR will get a star.

The results will display the best algorithm and the supporting KI and KR scores per algorithm.

You will also be able to review the results per algorithm if needed.

Hope you find this helpful.

Thanks.

[achab]

Concise & clear! Nice one Louis.

[former_member184566]

Thanks. Wonder who pointed out to me this new functionality :wink:

[achab]

Mystery 🙂

it is a mystery 10 hour version - YouTube

Cheers,

Antoine

PS/ still working on these email questions you sent me 🙂

[achab]

Also check this nice blog entry from hillary.bliss here: Best New Features in SAP Predictive Analytics 2.2 | SAP BI BLOG

Best regards

Antoine

[achab]

louis.degouveia2

I realized that the model statistics and model comparison cannot be used for time series algorithms, only for classification or regression scenarios.

See  our user guide for expert analytics on page 230: http://help.sap.com/businessobject/product_guides/pa22/en/pa22_expert_user_en.pdf

Model Statistics is a component that calculates performance statistics on datasets that are generated by algorithms. It can calculate statistics for two algorithm types, classification and regression.

It would make sense to update the screenshots in this blog to avoid any confusion.

Sorry for the inconvenience.

Antoine

[former_member184566]

Hi Antoine

Well spotted. KI and KR is used as a measurement for the accuracy of classification and regression algorithms. Time series we should make use of MAPE to determine time series model accuracy.

I have updated the screenshots accordingly.

I look forward to future versions of SAP PA throwing an exception or warning should a logical error occur like this, or maybe even the comparison at MAPE level :wink:

Thanks for spotting.

[achab]

+1. I raised the same ask to engineering teams 2 hours ago :-).

[achab]

Feel free to raise any product improvement idea in our idea place: Predictive Analytics: Home

I see two potential asks in this area, here we are discussing the first one mostly.

a. making the comparison of time series algorithms possible in expert - at MAPE level

b. bringing time series automated algorithms in expert analytics.

Thanks & regards

Antoine

[former_member184566]

Hi

Agree fully with point A.

Just created the idea,  https://ideas.sap.com/D27885

Please vote up.

[Former Member]

I'm new at this area and perhaps I am misunderstanding the concept of KI, but, when I tried to do an action using the SAP PA Association Rules, I got in some cases a negative KI.

How is it possible? And what does it mean?

Thanks in advance.

[achab]

Hi Jedson, can you please create a dedicated question? Ideally include screenshots and the underlying dataset if possible.

Thanks & regards

Antoine

[Former Member]

Sure.

Sorry about that.

[achab]

No problem - we just need to make sure we comply with SCN rules of engagement