Skip to main content Link Menu Expand (external link) Document Search Copy Copied

Example Impala data source

This example is intended for use with the fbsql loader command and provides:

  • valid SQL to create a table then insert data
  • TOML key/values to save as a configuration file

Before you begin

Step 1 - Create data source

  • Connect to your Impala database.
  • Create the source table:
    CREATE TABLE testdb.impala_table (
      idkey int,
      intf int,
      stringf string,
      idf int,
      stringsetf string,
      idsetf string);
  • Insert data to the source table:
    INSERT INTO testdb.impala_table VALUES
     (0, 0, 'a', 0, 'a', '3'),
     (1, 0, 'a', 0, 'c', '4'),
     (2, 0, 'a', 0, 'd', '5');

Step 2 - Create TOML configuration file

  • Open a terminal then run nano impala-import.toml
  • Add the following keys and values and substitute your Impala connection string as directed:
table = "loader-target"
query = "select idkey, intf, stringf, idf, stringsetf, idsetf from <impaladb>.impala_table;"
driver = "impala"
connection-string = "impala://<connection-string>/database=<source-db-name>"
batch-size = 1

The following [[fields]] keys and values are optional

name = "idkey"
source-type = "id"
primary-key = true

name = "intf"
source-type = "int"

name = "stringf"
source-type = "string"

name = "idf"
source-type = "id"

name = "stringsetf"
source-type = "stringset"

name = "idsetf"
source-type = "idset"
  • Save then exit nano.

Next step

Further information