Skip to Content
author's profile photo Former Member
Former Member

PAL Algorithms - Memory management

Hello,

I am trying to execute few predictive models, based on the help documentation.

Here is the scenario:

When I try to use HANA Apriori (tried APRIORILITE as well) in SAP Predictive Analytics 2.2 & 2.3 - with a table from HANA containing nearly a 100,000 records - It works fine.

The same table, when I try with a Procedure call directly using the PAL algorithm, it works in the following manner

(The minimum support and minimum confidence are the same as used in PA 2.3)

  1. When I run the Procedure with 10 entries in the view, it works.
  2. When I run the Procedure with 30 entries in the view, it works.
  3. When the number of entries is beyond 50 - it takes very long to respond, and the Procedure call would never execute successfully. I had to manually terminate using "ALTER " command.

The code for the view is as follows :

DROP VIEW PAL_DATA_V;

CREATE VIEW PAL_DATA_V AS

SELECT TOP 50 "Transaction", "Material_Description" from "SCHEMA"."TABLE";

Shouldn't it work for all the entries in the table?

I tried increasing the result table attribute sizes to the maximum - VARCHAR (5000)

I guess the issue is in setting up some memory related parameters. And how do I analyze / foresee such situations while handling large data sets?

The situation is the same for all the three algorithms that I tried (APRIORI, APRIORILITE, FPGROWTH).

Request your help to get through this.

Thank you.

Add a comment
10|10000 characters needed characters exceeded

Assigned Tags

Related questions

2 Answers

  • author's profile photo Former Member
    Former Member
    Posted on Sep 24, 2015 at 02:24 AM

    Hi Tamilnesan,

    The running time and memory consumption of these algorithms are depended on the data distribution & parameter settings.

    Actually there two parameters (MIN_SUPPORT & MIN_CONFIDENCE) are very important for association rule mining algorithms.

    So I suggest you to set them a little bigger (such as 0.5 & 0.7) firstly, and then decrease them depend on the output result.

    By the way could you please share us with the data & SQLScripts you used, maybe there are some issues in our algorithm.

    Thanks,

    Peng

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Sep 23, 2015 at 01:12 PM

    Hi,

    Maybe Former Member will be able to help with this question?

    Thanks & best regards

    Antoine

    Add a comment
    10|10000 characters needed characters exceeded

    • Former Member

      Thank you Antoine, lets wait for Xingtian Shi to reply for this.

      Today I tried the same with a different data set, with full of Integer values, and the only difference is the count increased from 50 to 100.

      This indicates that there is some "Setting" / "Parameter Configuration" to be taken care. I am not sure.

      Appreciate your help.

      Thank you.

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.