cannot feed a table in hive from hbase

Question

on HDP 3.1.x I created a table linked to Hbase with the option STORED BY org.apache.hadoop.hive.hbase.HBaseStorageHandler. When executing a select, it works fine. When I try to populate a table from this, it crashes with the error

create table test as select * from hbase_xxx;

INFO  : Completed executing command(queryId=hive_20210205161427_a49ca7bc-0637-4c19-9a62-6657376373a1); Time taken: 74.951 seconds

Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. 
Vertex failed, vertexName=Map 1, vertexId=vertex_1611574680060_3923_1_00, diagnostics=[Vertex vertex_1611574680060_3923_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: raw_eff_ann_ent initializer failed, vertex=vertex_1611574680060_3923_1_00 [Map 1], 
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location for replica 0

When having a look to YARN logs, it appears that it tries to connect to zookeeper from a datanode with localhost:2181 ... and failed

2021-02-05 11:22:41,921 [WARN] [ReadOnlyZKClient-localhost:2181@0x48730f2c] |zookeeper.ReadOnlyZKClient|: 0x48730f2c to localhost:2181 failed for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 1

The same log on a select show the zookeeper_quorum connection string to zookeeper and succeed

Any ideas?

question from:https://stackoverflow.com/questions/66066955/cannot-feed-a-table-in-hive-from-hbase

Answer 1

I had a return from the support; you have to force hbase-site.xml into the template of hive env This is the solution

    Add the below:
export HIVE_AUX_JARS_PATH=${HIVE_AUX_JARS_PATH}:/etc/hbase/conf/hbase-site.xml 
to your Advanced hive-env->hive-env template, before this statement export METASTORE_PORT={{hive_metastore_port}}