Python 3 Runtime in HANA XSA

[SergioG_TX]

Hello all,

I have tried installing the Python 3.x version to my HANA XSA Express edition without any luck.

I am able to create the CF Python buildpack but that apparently does not provide me the runtime and there are also instructions to create the runtime. I am following the BYOR Setup Python Runtime blog but I am only able to get to the step where the create-runtime command is executed and I am getting an error stating:

java.io.IOException: could not find ~/builds/Python-3.9.1/lb/python3.9/site-packages/pks_resources

I am not sure how to proceed from here.

in the vm, I am able to run python --version and i get the default runtime 2.7 and I can run python3 --version and I get 3.4

I am confused whether the buildpack should allow me to push a Python app to the system or not (I get an error when trying xs push app -b cf_python <-- the name of my buildpack which also shows in the xs buildpacks command)

or if I need to install the Python 3.9 version (matching the python install from the blog) but I cannot get pass through the create runtime command. Any tips are appreciated.

Thank you!

[SergioG_TX]

I have tried python 3.9.1 and 3.4.10 - I wonder if python 3 is not supported yet ?

[dvankempen]

Hi Sergio,

Python 3 is supported; the guide provides an example for 3.6.

./configure --prefix=/home/<myuser>/tmp/builds/Python-3.6.2 --exec-prefix=/home/<myuser>/tmp/builds/Python-3.6.2 --enable-optimizations
make -j4
make altinstall

• XSA Developer Guide (SPS 05)
  • Building from Sources and Deploying the Python Run Time to XS Advanced

[ansteiner]

Hi Sergio,

python 3 is supported. The most issues are caused by a incomplete Python build due to missing packages in the OS preparation step, also mentioned in the blog from Denys.

Please check the SP05 Development Guide about the required packages, as they are some added to the already mentioned packages in SP03 documentation and in Denys blog.

If you are still facing issues, please provide the whole output of the create-python-runtime.sh script.

Kind regards,

Andreas

[SergioG_TX]

thank you for your responses. I have already added the required packages shown in the building from sources documentation page. My current problem now is during the make -j4 command I am getting this error:

(I waited several minutes and the error wouldn'g go away, then I clicked on Cntl+C to exit the log and see the following:

Total duration 11 mins 56 sec

Test result: FAILURE

makefile:482: recipe for target 'run_profile_task' failed

Makefile:460: recipe for target 'profile-opt' failed

make: *** [profile-opt] interrupt

running the sudo make alt install clean afterwards runs and returns and completes.

running the create-runtime command

** the __pycache__ folder exists however

thank you again for your guidance

[ansteiner]

Hi Sergio,
there must be an failure earlier already. Before the test. Was the configure step successful? If so, was there an failure on the make earlier already?

On which OS you are trying to build the Python? If you have all the listed packages installed I would recommend to use the create-python-runtime.sh script. I unset PYTHONHOME and PYTHONPATH as <sid>adm before I run the script for version 3.9.1, otherwise I got also some make errors.

You should also uncomment the read -p lines in the script to check every step, before you proceed. Because in case the configure is not successful, the make will also fail and for sure the xs create-runtime. Perhaps we can improve the provided script with the unset of the env and also some exit code checks for every executed command.

In case you didn't get your Python runtime build successfully, please open an incident on BC-XS-PY and inform me about this. I will pick it up and we will get that resolved together.

Kind regards,

Andreas

[SergioG_TX]

my OS is suse linux on a vmworkstation 16 player.

yes, all the listed packages were installed correctly with no errors.I unset the PYTHONHOME and PYTHONPATHs -

where is the create-pythin-runtime.sh file located? I do not find it in the Python 3.9.1 version, I only see several references in the 3.6.2 folder

[ansteiner]

Hi Sergio,
I thought you followed the blog and in the section "Code Walktru" you can get everything from github. Also the scripts and run the create-python-runtime.sh with parameter e.g. 3.9.1 to get the Python 3.9.1 downloaded and created. Otherwise you have to download the Python version you want to build, extract them and run the configure and make by your self. But as mentioned you have to unset the env, if you run the script as <sid>adm and please uncomment the mentioned read -p lines in the script to stop after every step and check the output above to be sure the step was successful before you continue. As mentioned in case the configure or make step is not successful the incomplete Python runtime will be uploaded via xs create-runtime, but will not work.
You need to get a successful Python runtime build first and upload/install this in XSA via xs create-runtime command.

Kind regards
Andreas

[SergioG_TX]

hi Andreas, yes I followed the blog - I thought you were referring to a different file. When I run the create-python-runtime.sh file from the command line (uncommented the -p lines) and also unsetting the PYTHONPATH and PYTHONHOME vars, I still get an error at right after the make altinstall command (I did open the script and also included the make -j4 command suggested by Denys before the make altinstall clean

the output of the create-python-runtime.sh attached

output.txt

[ansteiner]

Hi Sergio,

thanks for the output. There is another package missing and the reason why the make failed.

Please check with zypper search libuuid-devel and as this should be missing install this via zypper install libuuid-devel.

Rerun the script. The make -j4 isn't really necessary. If all the needed packages installed the script should run without errors.

Kind regards,

Andreas

[SergioG_TX]

thank you for the response Andreas. I added the libuuid-devel and tried the steps to create the runtime but i am still getting errors on the altinstall step unfortunately. I will keep trying

[ansteiner]

Hi Sergio,

please check the github project for updates. I improved the create-python-runtime script already. It will first check the required packages, before the build will start and will do some further checks after every process call to avoid an upload of a broken or incomplete Python runtime to XSA.

Please provide the complete output again to check the latest error.

Kind regards,

Andreas

[SergioG_TX]

thank you for the updated script - I was able to run it and still getting an error during the altinstall step.

attached is the output --> outsat.txt

[ansteiner]

Hi Sergio,

you are still struggling with the Python make. There are still issues with Python packages like pip:

Can't list '/usr/sap/HXE/home/builds/Python-3.9.1/lib/python3.9/site-packages/pip' and some others.

In addition there are some strange permission issues:

Compiling '/usr/sap/HXE/home/builds/Python-3.9.1/lib/python3.9/config-3.9-x86_64-linux-gnu/python-config.py'... *** PermissionError: [Errno 13] Permission denied: '/usr/sap/HXE/home/builds/Python-3.9.1/lib/python3.9/config-3.9-x86_64-linux-gnu/__pycache__/python-config.cpython-39.opt-1.pyc.140048625805776'

I would recommend to cleanup the folders builds, source, Downloads in home of hxeadm (/usr/sap/HXE/home) and rerun the script again.

If there are still issues it must be somehow related to the vm you are running into. Do you have the possibility to make the Python on another host?

Kind regards

Andreas

[SergioG_TX]

I wish I had better answers but it is still not working.

I have tried this on 3 different VM ware versions and i get the errors before mentioned. I also tried on an Oracle Virtual box getting the same errors.

I went ahead and tried the Express edition on an EC2 instance on AWS but the issue I am now facing is that the terminal does not seem to take the root password. I assume the root password is the one I have to change initially from HXEHana1 to my new password? is there a different default password I am not aware of on HXE? I have tried installing this a couple of times and it is failing on the root password so I am not even able to install any of the packages. In all the other videos from HANA academy, they are using google cloud, is there something unique about google cloud? I feel really lost on something that seems to be fairly simple. I do not see any other post from anyone else having this error. any advice on which vm i need to try next?

[ansteiner]

Hi Sergio,

it's really strange that the python make is not working. Did the error change after you installed the libuuid-devel or was it really still the same error? Can you provide the whole output again? Perhaps you can also ask the python community for some support with the make.

I'm not an expert about the HXE. Perhaps denys.kempen can help with this.

Kind regards,

Andreas

[dvankempen]

Will check, to be continued...

[SergioG_TX]

Andreas, thank you for your continued support on this question. I was able to rebuilt my vm (vm ware) and install, import, etc... however, the create runtime file is still throwing the error related to no acceptable C compiler.

see attached file.output.txt

[ansteiner]

Hi Sergio,

for sure, you need a compiler installed to build python. See also https://devguide.python.org/setup/#build-dependencies.

So install the gcc and rerun the script.

Kind regards,

Andreas

[draschke]

Hi Sergio,

I know it's been a while since you posted this thread. However, since you mentioned that you built your own python_buildpack, I'm wondering if we need to do the same.

We were able to build the Python Runtime for multiple versions using a slightly modified [repo](https://github.com/draschke/rhel-create-xsa-python-runtime.sh) from [Andreas](https://github.com/saphanaacademy/python) and push it to XSA.

vscode ➜ /workspaces/python/pyapp $ xs runtimes | grep pythonpython3 3.10.13 23 true true Python 3.10.13 0python3 3.12.0 22 true true Python 3.12.0 0python3 3.9.17 24 true true Python 3.9.17 0
vscode ➜ /workspaces/python/pyapp $ xs buildpacksbuildpacks version position enabledsap_python_buildpack 0.5.3 4 true

However, we are now getting the following error when pushing the sample app [pyapp](https://help.sap.com/docs/SAP_HANA_PLATFORM/4505d0bdaf4948449b7f7379d24d0f0d/5fdcf7f11e9c4979a83d00e9ca44b67a.html?version=2.0.05&locale=en-US)

xs push --verbose pyapp

OUT Detect: BUILD_DIR [/hana/shared/sid/xs/app_working/stager/stager-sidxsaspace2xsa/work/64d8b540-5b83-488f-9efb-231f9535b6a5-6/droplet/app] OUT Detect: BUILDPACK_DIR [/hana/shared/sid/xs/app_working/stager/stager-sidxsaspace2xsa/cache/264513e6-7dc1-442d-8b01-17123507f88e/bin/..] OUT Python Buildpack version: 0.5.3 ERR compile: python3 detected at /bin/python3 OUT Logging was set to INFO OUT Compile: started, running with python version: 3.6.8 OUT Compile: BUILD_DIR [/hana/shared/sid/xs/app_working/stager/stager-sidxsaspace2xsa/work/64d8b540-5b83-488f-9efb-231f9535b6a5-6/droplet/app] OUT Compile: CACHE_DIR [/hana/shared/sid/xs/app_working/stager/stager-sidxsaspace2xsa/cache/20e5c823-654f-4cad-a749-e108192e40e3-264513e6-7dc1-442d-8b01-17123507f8 8e] OUT Compile: BUILDPACK_DIR [/hana/shared/sid/xs/app_working/stager/stager-sidxsaspace2xsa/cache/264513e6-7dc1-442d-8b01-17123507f88e] OUT Compile: running on [Linux exvmdsapsidrdb 4.18.0-305.91.1.el8_4.x86_64 #1 SMP Thu May 18 15:06:49 EDT 2023 x86_64] OUT Compile: application requires python version 3.12.0
ERR Execution of bin/compile failed (exit value 1) ERR Failed to stage app 'pyapp' [Org 'XSA', Space 'DEV_SANDBOX']: Execution of bin/compile failed (exit value 1).

The system is trying to build the app with Python 3.6.8, even though we have the runtime 3.12.0 in XSA. Do we need to build the Python buildpack ourselves?

Thanks, Dirk