onefoursix/Cloudera-Impala-JDBC-Example: A Maven-based example of using Cloudera ...

原作者: [db:作者] 来自: 网络收藏邀请

开源软件名称（OpenSource Name）：

onefoursix/Cloudera-Impala-JDBC-Example

开源软件地址(OpenSource Url)：

https://github.com/onefoursix/Cloudera-Impala-JDBC-Example

开源编程语言(OpenSource Language)：

Java 92.2%

开源软件介绍(OpenSource Introduction)：

###Cloudera Impala JDBC Example

Apache Impala (Incubating) is an open source, analytic MPP database for Apache Hadoop.

This example shows how to build and run a Maven-based project to execute SQL queries on Impala using JDBC

This example was tested using Impala 2.3 included with CDH 5.5.2 and the Impala JDBC Driver v2.5.30

When you download the Impala JDBC Driver from the link above, it is packaged as a zip file with separate distributions for JDBC3, JDBC4 and JDBC4.1. This example uses the distribution for JDBC4.1 on RHEL6 x86_64. The downloaded zip file contains the following eleven jar files:

(1)  ImpalaJDBC41.jar
(2)  TCLIServiceClient.jar
(3)  hive_metastore.jar
(4)  hive_service.jar
(5)  ql.jar
(6)  libfb303-0.9.0.jar
(7)  libthrift-0.9.0.jar
(8)  log4j-1.2.14.jar
(9)  slf4j-api-1.5.11.jar
(10) slf4j-log4j12-1.5.11.jar
(11) zookeeper-3.4.6.jar

The JDBC driver's installation instructions say only that "...you must set the class path to include all the JAR files from the ZIP archive containing the driver that you are using..."

While this works fine for one-off projects, it's a little loose for shops that would rather manage their dependencies using Maven or other build systems.

Part of the challenge in building a project using those jars with Maven is that some of the jars are not available in public repos and some of them do not have obvious version numbers. My approach in this example will be to use a local Maven repo to manage the first five jars in the list above and to rely on publicly available Maven repos for jars 6 - 11 (as they have version numbers in their name). I will use the community version of the Nexus Repository Manager OSS as a local Maven repo

I downloaded Nexus Repository Manager OSS v2.12 from the link here and followed the installation instructions here

Here is the view of my local Nexus repo available after launching it for the first time. Note there is already a repo named "3rd party" which I will use to manage the first five JDBC driver jars:

To add jars to the repo, login to the local Nexus repo, go to the 3rd party repo's "upload artifacts" tab and select the desired jar to upload. I specified a group of "com.cloudera.impala.jdbc" and a version number of "2.5.30" for each of the five jars I uploaded, like this:

Click on the 3rd party repo's URL link and you can browse the uploaded artifacts:

Drill into any of the links and you can see the version number has been appended to each jar:

Now that we have a local repo available hosting the JDBC jars, all we need to do is add that repo to our pom with an entry like this:

<repository>
  <id>YOUR.LOCAL.REPO.ID</id>
  <url><YOUR LOCAL REPO URL></url>
  <name>YOUR.LOCAL.REPO.NAME</name>
  <snapshots>
    <enabled>false</enabled>
  </snapshots>
</repository>

For example, in my case my local repo entry looks like this:

<repository>
  <id>nexus.local</id>
  <url>http://10.10.10.7:8081/nexus/content/repositories/thirdparty</url>
  <name>Nexus Local</name>
  <snapshots>
    <enabled>false</enabled>
  </snapshots>
</repository>

And you can refer to the JDBC artifacts with entries like this:

<dependency>
  <groupId>com.cloudera.impala.jdbc</groupId>
  <artifactId>ImpalaJDBC41</artifactId>
  <version>2.5.30</version>
</dependency>

Jars 6 - 11 will be retrieved from the Cloudera and Maven Central repos and will have traditional dependency elements like this:

<dependency>
  <groupId>org.apache.thrift</groupId>
  <artifactId>libfb303</artifactId>
  <version>0.9.0</version>
</dependency>

See the pom.xml for details

####Dependencies To build the project you must have Maven 2.x or higher installed. Maven info is here.

To run the project you must have access to a Hadoop cluster running Impala with at least one populated table defined in the Hive Metastore.

Configuring the example

Make sure to set your local repo in pom.xml as described above

Edit the file src/main/resources/ClouderaImpalaJdbcExample.conf and set an Impala daemon's host and port in the connection.url (Impala's default JDBC port is 21050) and set the appropriate JDBC driver class. I am using JDBC4.1 so my conf file looks like this:

# ClouderaImpalaJdbcExample.conf
connection.url = jdbc:impala://chicago.onefoursix.com:21050
jdbc.driver.class.name = com.cloudera.impala.jdbc41.Driver

See the JDBC driver's docs for more details.

Building the example

Build the project like this:

$ mvn clean package

If this is the first time you are building the project you should see messages like this showing that Maven is retrieving the JDBC jars from your local repo:

Downloading: http://10.10.10.7:8081/nexus/content/repositories/thirdparty/com/cloudera/impala/jdbc/hive_metastore/2.5.30/hive_metastore-2.5.30.jar
Downloading: http://10.10.10.7:8081/nexus/content/repositories/thirdparty/com/cloudera/impala/jdbc/hive_service/2.5.30/hive_service-2.5.30.jar
Downloading: http://10.10.10.7:8081/nexus/content/repositories/thirdparty/com/cloudera/impala/jdbc/ImpalaJDBC41/2.5.30/ImpalaJDBC41-2.5.30.jar
Downloading: http://10.10.10.7:8081/nexus/content/repositories/thirdparty/com/cloudera/impala/jdbc/ql/2.5.30/ql-2.5.30.jar
Downloading: http://10.10.10.7:8081/nexus/content/repositories/thirdparty/com/cloudera/impala/jdbc/TCLIServiceClient/2.5.30/TCLIServiceClient-2.5.30.jar

Whereas the other jars (and their dependencies) are downloaded from the public repos:

Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/thrift/libfb303/0.9.0/libfb303-0.9.0.jar
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/thrift/libthrift/0.9.0/libthrift-0.9.0.jar
...

If your build is successful you should see messages like this:

[INFO] Building jar: /home/mark/a/Cloudera-Impala-JDBC-Example-impala-cdh-5.5.2/cloudera-impala-jdbc-example-1.0.jar
[INFO] 
[INFO] --- maven-shade-plugin:2.2:shade (default) @ cloudera-impala-jdbc-example ---
[INFO] Including com.cloudera.impala.jdbc:hive_metastore:jar:2.5.30 in the shaded jar.
[INFO] Including com.cloudera.impala.jdbc:hive_service:jar:2.5.30 in the shaded jar.
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.108 s
[INFO] Finished at: 2016-02-21T11:24:56-08:00
[INFO] Final Memory: 32M/476M
[INFO] ------------------------------------------------------------------------

Note that pom.xml is configured to have Maven build an "uber jar" will all dependencies packaged in a single jar and with the main class set

The uber jar will be located at target/cloudera-impala-jdbc-example-uber.jar

Running the example using the uber jar

One can run the example using the uber jar with a "java -jar" command with a SQL statement as an argument like this:

$ java -jar target/cloudera-impala-jdbc-example-uber.jar "SELECT description FROM sample_07 limit 10"

=============================================
Cloudera Impala JDBC Example
Using Connection URL: jdbc:impala://chicago.onefoursix.com:21050
Running Query: SELECT description FROM sample_07 limit 10

== Begin Query Results ======================
All Occupations
Management occupations
Chief executives
General and operations managers
Legislators
Advertising and promotions managers
Marketing managers
Sales managers
Public relations managers
Administrative services managers
== End Query Results =======================

There is a "run.sh" script provided with that command

Running the example using Maven

One can also run the example using Maven using the run-with-maven.sh script which by default passes a SQL statement as an argument:

mvn exec:java -Dexec.mainClass=com.cloudera.example.ClouderaImpalaJdbcExample -Dexec.arguments="SELECT description FROM sample_07 limit 10"

Your output should look like this:

$ ./run-with-maven.sh
[INFO] Scanning for projects...
...                                                                        
[INFO] ------------------------------------------------------------------------
[INFO] Building cloudera-impala-jdbc-example 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) > validate @ cloudera-impala-jdbc-example >>>
[INFO] 
[INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) < validate @ cloudera-impala-jdbc-example <<<
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ cloudera-impala-jdbc-example ---

Cloudera Impala JDBC Example
Using Connection URL: jdbc:impala://chicago.onefoursix.com:21050
Running Query: SELECT description FROM sample_07 limit 10

== Begin Query Results ======================
All Occupations
Management occupations
Chief executives
General and operations managers
Legislators
Advertising and promotions managers
Marketing managers
Sales managers
Public relations managers
Administrative services managers
== End Query Results =======================

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

鲜花

握手

雷人

路过

鸡蛋

该文章已有0人参与评论

请发表评论

全部评论

专题导读

More+

10-27 六六分期app的软件客服如何联系？(六六分期

11-06 可心卡盟:win10系统火狐flash插件崩溃怎么

11-06 亲亲特价:怎么删除回收站图标

11-06 济南大学虚拟社区:鲁大师节能降温的具体办

11-06 xlueops.exe:无线网络安装向导

11-06 女斗合众国:win7系统cf与主机连接不稳定怎

11-06 0xc000022-[cf烟雾头]cf怎么调烟雾头

11-06 qizideyouhuo:应用程序无法正常启动0xc0000

11-06 ipz-185:win7系统vcf文件怎么打开

11-06 傻哥蹦迪:win10系统s4怎么打开usb调试

11-06 八神浩树gtaste:回收站清空了怎么恢复

11-06 妖尾之黑色守护:win10系统电脑没有1440x900

11-06 校园至尊魔王小说:win7系统浏览网页时字体

11-06 女斗合众国:win10系统访问共享文件夹提示请

11-06 tokyo hot n0654:恢复win7系统默认字体一招

11-06 雨酷仙境:设置win7系统转移临时文件夹腾出

11-06 阿穆纳伊之杖:win7系统开始菜单在右边还原

11-06 tunespotting:win10系统火狐flash插件总是

11-06 甘尔葛分析师：计谋网站seo关键词暴涨有什

11-06 蔡贵霖: 计谋网站seo关键词暴涨有什么秘密

11-06 博益网首页:ao3网页版进入不了解决方法

11-06 漏斗子专栏: 网站数据分析小白易懂精华篇

11-06 见证双虹怎么做:win7系统开启telnet命令的

11-06 颾狐蝶蜋:系统资源不足无法完成请求的服务

11-06 国光中学校歌:提交网站到alexa查询详细步骤

11-06 西安有情天:静态网页和动态网页的区别

11-06 红木雅尚斋:外部链接构造对网站的好处

11-06 前官礼遇：防止域名劫持–增强域安全性的10

11-06 密传二转答案: 中文分词算法有哪些

11-06 金泉家园邮编:百度快照劫持的表现及应对方

jenkinsci/archetypes: Collection of Maven archetypes to get developers started发布时间：2022-08-17

fenniless/Maven.Spring-BeansLearnerLab发布时间：2022-08-17

剪的笔顺,诠释剪的笔画,认识剪的部首

1 六六分期app的软件客服如何联系？(六六分期

六六分期app的软件客服如何联系？不知道吗？加qq群【895510560】即可！标题：六六分期

阅读：19085|2023-10-27

2 可心卡盟:win10系统火狐flash插件崩溃怎么

今天小编告诉大家如何处理win10系统火狐flash插件总是崩溃的问题，可能很多用户都不知

阅读：9959|2022-11-06

3 亲亲特价:怎么删除回收站图标

今天小编告诉大家如何对win10系统删除桌面回收站图标进行设置，可能很多用户都不知道

阅读：8312|2022-11-06

4 济南大学虚拟社区:鲁大师节能降温的具体办

今天小编告诉大家如何对win10系统电脑设置节能降温的设置方法，想必大家都遇到过需要

阅读：8682|2022-11-06

5 xlueops.exe:无线网络安装向导

我们在使用xp系统的过程中,经常需要对xp系统无线网络安装向导设置进行设置，可能很多

阅读：8620|2022-11-06

6 女斗合众国:win7系统cf与主机连接不稳定怎

今天小编告诉大家如何处理win7系统玩cf老是与主机连接不稳定的问题，可能很多用户都不

阅读：9634|2022-11-06

7 0xc000022-[cf烟雾头]cf怎么调烟雾头

电脑对日常生活的重要性小编就不多说了，可是一旦碰到win7系统设置cf烟雾头的问题，很

阅读：8607|2022-11-06

8 qizideyouhuo:应用程序无法正常启动0xc0000

我们在日常使用电脑的时候，有的小伙伴们可能在打开应用的时候会遇见提示应用程序无法

阅读：7985|2022-11-06

9 ipz-185:win7系统vcf文件怎么打开

今天小编告诉大家如何对win7系统打开vcf文件进行设置，可能很多用户都不知道怎么对win

阅读：8631|2022-11-06

10 傻哥蹦迪:win10系统s4怎么打开usb调试

今天小编告诉大家如何对win10系统s4开启USB调试模式进行设置，可能很多用户都不知道怎

阅读：7524|2022-11-06

客服电话

电子邮件

onefoursix/Cloudera-Impala-JDBC-Example: A Maven-based example of using Cloudera ...

开源软件名称（OpenSource Name）：

开源软件地址(OpenSource Url)：

开源编程语言(OpenSource Language)：

开源软件介绍(OpenSource Introduction)：

Configuring the example

Building the example

Running the example using the uber jar

Running the example using Maven

请发表评论

全部评论

上一篇：

下一篇：

CVE-2022-20877

chasinginfinity/ml-from-scratch: Machine

平板电脑性价比排行

mkyong/spring3-mvc-maven-annotation-hell

床的笔顺,关于床的笔画,体会床的部首

剪的笔顺,诠释剪的笔画,认识剪的部首

六六分期app的软件客服如何联系？(六六分期

florent37/ViewAnimator: A fluent Android

florent37/Shrine-MaterialDesign2: implem

CVE-2020-36276

SimpleSoftwareIO/simple-sms: Send and re

关于我们

产品与服务

解决方案

139-2527-9053