Skip to Content
avatar image
Former Member

SAP Vora table auto refresh

Hi Community,

We have 2 queries regarding SAP Vora:

1. We came across a scenario where we need to refresh Vora tables with original data source files (present in HDFS) automatically. is there a way this can be achieved?

Can we create and schedule a job in Vora that would integrate with original data source files(located in HDFS) and update and(or) append the required changes into existing Vora tables?

OR

Is it possible to update Vora tables with recent data from HDFS while executing them from Vora?

2. Relational Engine tables created on top of HDFS files gets memory allocated in Vora, is there a way to find out if the memory allocated is on disk or in Memory without restarting Vora instance.

3. Need to know the directory path in Vora that can be accessed via command line (CLI) to know location of tables stored in Vora (across different engines)

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Feb 05 at 04:02 PM

    Ravi,

    1. You can always append new data files from HDFS to an existing table using SQL. However, Vora does not automatically pick-up changes to existing files it has already loaded. If you update/append HDFS files with new data, you must either drop and recreate the table in Vora, or restart the Vora in-memory engine.

    2. I am not sure I understand you here. Generally, the in-memory relational engine always uses RAM to store table data, and the disk engine always uses disk. The disk engine uses some RAM for processing, but this is limited by the disk engine deployment parameters. You might consider playing around with the Grafana UI and looking at the statistics that are measured/monitored there to see if there is something you can use (see section 3.3 of the Install and Admin. guide).

    3. Again, I am not exactly sure what you are looking for, but the location of data files used by Vora is defined in the deployment parameters for the various services (see section 2.5 of the Install and Admin. guide).

    --Jason

    Add comment
    10|10000 characters needed characters exceeded

    • If Vora is restarted, the engines will reload any hdfs files you have previously added to the table when you query them.

      Table metadata is kept by the Vora Catalog service and durability is provided by the Distributed Log (DLog) service. The location of your distributed log files is defined in your DLog service definition parameters, as I mentioned in my original answer.

      --Jason