Unable to execute graph with two different tags in Python Operator of SAP DI 3.0

[ansharm71]

Hi,

We have to execute ML model in DI Graph which involves libraries like xgboost,fbprophet,sklearn,statsmodels etc.

We have successfully created two docker files using below code:

1. Installing xgboost/fbprophet as tag "fbprophet-xgboost"

FROM $com.sap.sles.ml.python USER 1972:1972

WORKDIR /home/vflow

ENV HOME=/home/vflow

RUN python3.6 -m pip --no-cache-dir install --user --upgrade pip

RUN python3.6 -m pip --no-cache-dir install --user fbprophet

RUN python3.6 -m pip --no-cache-dir install --user xgboost

2. Installing other ML libraries as tag "CP"

FROM §/com.sap.datahub.linuxx86_64/sles:15.0-sap-020

RUN groupadd -g 1972 vflow && useradd -g 1972 -u 1972 -m vflow

USER 1972:1972 WORKDIR /home/vflow

ENV HOME=/home/vflow # Install Cognitive Pricing Libraries

RUN python3.6 -m pip --no-cache-dir install 'future' --user

RUN python3.6 -m pip --no-cache-dir install 'sklearn' --user

RUN python3.6 -m pip --no-cache-dir install 'statsmodels' --user

Now we are using tags in python operator to execute graph .

But graph is Failing with above error message.

Please note when we run individually xgboost/fbprophet then graph is running fine but when we use it with other libraries its giving error.

Need help to resolve this issue.

[former_member704859 · Accepted solution]

Hi Ankit,

A single docker file has to be created for all the libraries and that has to be used as a tag while grouping python operator. Please try the following code. It worked for me.

FROM $com.sap.sles.ml.python
USER 1972:1972

WORKDIR /home/vflow
ENV HOME=/home/vflow

RUN python3.6 -m pip --no-cache-dir install --user --upgrade pip
RUN python3.6 -m pip --no-cache-dir install --user fbprophet
RUN python3.6 -m pip --no-cache-dir install --user xgboost
RUN python3.6 -m pip --no-cache-dir install 'future' --user
RUN python3.6 -m pip --no-cache-dir install 'sklearn' --user
RUN python3.6 -m pip --no-cache-dir install 'statsmodels' --user

Regards

Achin Kimtee

[ansharm71 · Accepted solution]

Thanks Achin for your response.

Yes using above code graph executed successfully.

[Vitaliy-R]

From what I can see these two tags represent two separate images. And only one image can be used to create a container in which the operator is executed.

When you put both of the two tags into the group of the operator DI tries to find a single container image that has both tags -- but such a single image with these two tags does not exist.

[ansharm71]

Thanks Witalij for your response.

You were correct i have to use single container image for both tags.

I did that and then graph executed successfully.