Skip to Content
avatar image
Former Member

Accesing Hana from spark via jdbc

Hi there,

I am trying to access a Hana table from spark using sparksql but when building the dataframe and doing .show() i get an error saying one of the classes(Host) in the hana driver does not support serializable, any idea how to fix this?

this is what I do to generate the problem:

1. Start spark-shell

2. val df = sqlContext.load("jdbc", Map("url" -> "jdbc:sap://xXXXXXX:30015/?databaseName=mydb&user=SYSTEM&password=xxxxx", "dbtable" -> "SYSTEM.TEST1"));

3. df.show();

I get below exception on calling any action on dataframe object.

org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: com.sap.db.jdbc.topology.Host

Serialization stack:

- object not serializable (class: com.sap.db.jdbc.topology.Host, value: xxxxxx:30015)

- writeObject data (class: java.util.ArrayList)

- object (class java.util.ArrayList, [xxxxxx:30015])

- writeObject data (class: java.util.Hashtable)

- object (class java.util.Properties, {dburl=jdbc:sap://xxxxx:30015, user=SYSTEM, password=Saphana123, url=jdbc:sap://xxxxxxxx:30015/?system&user=SYSTEM&password=Saphana123, dbtable=SYSTEM.TEST1, hostlist=[xxxxxxx:30015]})

any ideas how to fix this?

regards,

Miguel

Add comment
10|10000 characters needed characters exceeded

  • Follow
  • Get RSS Feed

1 Answer

  • avatar image
    Former Member
    Sep 09, 2016 at 10:40 PM

    Hi Miguel,

    I saw the same issue with the SPS09 version of the Hana hdbc driver (ngdbc.jar) , but the issue appears to resolved with the latest SPS12 driver. You can see examples running in this blog Calling HANA Views from Apache Spark

    I didn't bother testing the SPS10 or SPS11 versions of the jar, so not sure in which version it was resolved.

    Regards

    Aron

    Add comment
    10|10000 characters needed characters exceeded