Accesing Hana from spark via jdbc

[Former Member]

Hi there,

I am trying to access a Hana table from spark using sparksql but when  building the dataframe and doing .show() i get an error saying one of the classes(Host)  in the hana driver does not support serializable, any idea how to fix this?

this is what I do to generate the problem:

1. Start spark-shell

2. val df = sqlContext.load("jdbc", Map("url" -> "jdbc:sap://xXXXXXX:30015/?databaseName=mydb&user=SYSTEM&password=xxxxx", "dbtable" -> "SYSTEM.TEST1"));

3. df.show();

I get below exception on calling any action on dataframe object.

org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: com.sap.db.jdbc.topology.Host

Serialization stack:

        - object not serializable (class: com.sap.db.jdbc.topology.Host, value: xxxxxx:30015)

        - writeObject data (class: java.util.ArrayList)

        - object (class java.util.ArrayList, [xxxxxx:30015])

        - writeObject data (class: java.util.Hashtable)

        - object (class java.util.Properties, {dburl=jdbc:sap://xxxxx:30015, user=SYSTEM, password=Saphana123, url=jdbc:sap://xxxxxxxx:30015/?system&user=SYSTEM&password=Saphana123, dbtable=SYSTEM.TEST1, hostlist=[xxxxxxx:30015]})

any ideas how to fix this?

regards,

Miguel

[Former Member]

Hi Miguel,

I saw the same issue with the SPS09 version of the Hana hdbc driver (ngdbc.jar) , but the issue appears to resolved with the latest SPS12 driver.  You can see examples running in this blog

I didn't bother testing the SPS10 or SPS11 versions of the jar, so not sure in which version it was resolved.

Regards

Aron