hive2.1.0安装

这里是使用mysql作为hive的元数据存储所以要先启动mysql的服务,MySQL的安装可以参考下面连接

MySQL安装:http://www.cnblogs.com/zhangXingSheng/p/6941422.html

下载hive(http://mirrors.cnnic.cn/apache/hive/)

或者 http://archive.apache.org/dist/hive/(hive历史版本)
在本地进行解压
 
下载mysql的jdbc,将下载后的jdbc放到hive安装包的lib目录下
http://dev.mysql.com/downloads/connector/j/
 
配置环境变量
#########hive####################
export HIVE_HOME=/usr/local/development/hive-2.1.0
export PATH=$HIVE_HOME/bin:$PATH
在hdfs上创建hive存储数据的相关目录
[root@node4 bin]# hdfs dfs -mkdir -p /opt/hive/warehouse
[root@node4 bin]# hdfs dfs -mkdir -p /opt/hive/log
修改hive-site.xml文件
cp hive-default.xml.template hive-site.xml
<name>hive.metastore.warehouse.dir</name>
<value>/opt/hive/warehouse</value>//对应以上在hdfs创建的文件目录
<description>location of default database for the warehouse</description>
 
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.177.124:3306/hivedb</value>//mysql的链接地址,这里的hivedb是我已经在mysql中创建的数据库,如果不写就是default
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/dbName?ssl=true for postgres database.
</description>
 
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>//mysql的驱动
<description>Driver class name for a JDBC metastore</description>
 
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>//链接mysql的用户名
<description>Username to use against metastore database</description>
 
 
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>////链接mysql的密码
<description>password to use against metastore database</description>
 
<name>hive.metastore.uris</name>
<value>thrift://192.168.177.124:9083</value>
 <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
 
 
修改hive-log4j2.properties和hive-exec-log4j2.properties//hive的日志文件
在 新建日志存储目录/home/zhangxs/hive/log  和  /home/zhangxs/hive/execlog
 
(1)hive-log4j2.properties
property.hive.log.dir = ${mine:/home/zhangxs/hive/log}/${user.name}
(2)hive-exec-log4j2.properties
property.hive.log.dir = ${mine:/home/zhangxs/hive/execlog}/${user.name}
 
启动步骤
1:首次启动hive需要初始化
使用schematool初始(要保证hadoop集群已经启动)
schematool -dbType mysql -initSchema
 
2:启动hive服务
[ hive --service metastore]
 
3:新打开一个终端 输入
hive
 
 
 
遇到问题
1:这个错误是因为,我在hive-site.xml文件里配置的是以root用户登录的,mysql不予许hive远程root登录,你可以配置一下权限,或者换一个普通的用户登录
Caused by: java.sql.SQLException: Access denied for user 'root'@'hadoop-node4.com' (using password: YES)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:964)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3970)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3906)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:873)
at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1710)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2253)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2284)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2083)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:806)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:410)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:328)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:190)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
问题2:hive在第一次登录的时候使用 schematool -dbType mysql -initSchema命令初始化,以后则不用。使用
hive --service metastore启动服务即可
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
问题3:这个是因为我修改hive-site.xml文件时候,值替换了system:java.io.tmpdir一部分,因该将
这整个value里面的值都替换掉
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
hive> show tables;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$%7B/home/zhangxs/data/hive/tmp%7D/$%7Bsystem:user.name%7D/dfb16330-2bbc-481a-bbdc-e4503c4d1a5f/hive_2016-12-12_05-12-52_963_7246570582065346726-1
hive> show tables;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$%7B/home/zhangxs/data/hive/tmp%7D/$%7Bsystem:user.name%7D/dfb16330-2bbc-481a-bbdc-e4503c4d1a5f/hive_2016-12-12_05-14-22_015_4353303889983385851-1
 
 
在启动schematool -dbType mysql -initSchema或者hive --service metastore的时候命令后,应该在重新打开一个窗口执行hive命令,要不然,会卡在
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
 
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
^Z
[2]+ Stopped hive
 
原文地址:https://www.cnblogs.com/zhangXingSheng/p/6223354.html