数据仓库Hive配置

1.下载Hive

1、Hive下载:

执行命令:wget http://mirror.bit.edu.cn/apache/hive/hive-2.3.0/apache-hive-2.3.0-bin.tar.gz

 

 

2、解压Hive包

bin:包含hive的命令shell脚本

binary-package-licenses:包含了LICENSE说明文件

conf:包含hive配置文件

examples:包含了示例

hcatalog:Metastore操作的元数据目录

jdbc:提供了hive-jdbc-2.3.0-standalone.jar包

scripts:提供了sql脚本

 

 

3、修改环境变量:

执行命令:vi /etc/profile

export JAVA_HOME=/usr/local/software/jdk1.8.0_66

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export HADOOP_HOME=/usr/local/software/hadoop_2.7.1

export HBASE_HOME=/usr/local/software/hbase_1.2.2

export HIVE_HOME=/usr/local/software/apache-hive-2.3.0-bin

export PATH=.:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$PATH

执行命令:source /etc/profile   刷新环境变量

2. 修改Hive配置文件

1、修改hive-site.xml配置文件:

执行命令:

cd/usr/local/software/apache-hive-2.3.0-bin/conf/

mv hive-default.xml.templatehive-site.xml

 

2、新建 hdfs目录

使用 hadoop新建 hdfs目录,因为在 hive-site.xml中有默认如下配置:

 

  1.  
    <property>
  2.  
    <name>hive.metastore.warehouse.dir</name>
  3.  
    <value>/user/hive/warehouse</value>
  4.  
    <description>location of defaultdatabase for the warehouse</description>
  5.  
    </property>

3、进入 hadoop安装目录执行hadoop命令新建/user/hive/warehouse目录,并授权,用于存储文件

hadoop fs -mkdir -p /user/hive/warehouse

hadoop fs -mkdir -p /user/hive/tmp

hadoop fs -mkdir -p /user/hive/log

hadoop fs -chmod -R 777 /user/hive/warehouse

hadoop fs -chmod -R 777 /user/hive/tmp

hadoop fs -chmod -R 777 /user/hive/log

 

# 用以下命令检查目录是否创建成功

hadoop fs -ls /user/hive

 

4、修改 hive-site.xml

搜索hive.exec.scratchdir,将该name对应的value修改为/user/hive/tmp

 

  1.  
    <property>
  2.  
    <name>hive.exec.scratchdir</name>
  3.  
    <value>/user/hive/tmp</value>
  4.  
    </property>

搜索hive.querylog.location,将该name对应的value修改为/user/hive/log/hadoop

 

  1.  
    <property>
  2.  
    <name>hive.querylog.location</name>
  3.  
    <value>/user/hive/log/hadoop</value>
  4.  
    <description>Location of Hive run time structured logfile</description>
  5.  
    </property>

搜索javax.jdo.option.connectionURL,将该name对应的value修改为MySQL的地址

 

  1.  
    <property>
  2.  
    <name>javax.jdo.option.ConnectionURL</name>
  3.  
    <value>jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true</value>
  4.  
    <description>
  5.  
    JDBC connectstring for a JDBC metastore.
  6.  
    To use SSL toencrypt/authenticate the connection, provide
  7.  
    database-specific SSL flag in theconnection URL.
  8.  
    For example,jdbc:postgresql://myhost/db?ssl=true for postgres database.
  9.  
    </description>
  10.  
    </property>

搜索javax.jdo.option.ConnectionDriverName,将该name对应的value修改为MySQL驱动类路径

 

  1.  
    <property>
  2.  
    <name>javax.jdo.option.ConnectionDriverName</name>
  3.  
    <value>com.mysql.jdbc.Driver</value>
  4.  
    <description>Driverclass name for a JDBC metastore</description>
  5.  
    </property>

搜索javax.jdo.option.ConnectionUserName,将对应的value修改为MySQL数据库登录名

 

  1.  
    <property>
  2.  
    <name>javax.jdo.option.ConnectionUserName</name>
  3.  
    <value>root</value>
  4.  
    <description>Username touse against metastore database</description>
  5.  
    </property>

搜索javax.jdo.option.ConnectionPassword,将对应的value修改为MySQL数据库的登录密码

 

  1.  
    <property>
  2.  
    <name>javax.jdo.option.ConnectionPassword</name>
  3.  
    <value>root</value>
  4.  
    <description>password to useagainst metastore database</description>
  5.  
    </property>

创建tmp目录

执行命令:mkdir -p/usr/local/software/apache-hive-2.3.0-bin/tmp

 

5、修改 hive-site.xml

把${system:java.io.tmpdir}改成/usr/local/software/apache-hive-2.3.0-bin/tmp

把${system:user.name}改成${user.name}

 

6、修改hive-env.sh

mv hive-env.sh.template hive-env.sh

HADOOP_HOME=/usr/local/software/hadoop_2.7.1

export HIVE_CONF_DIR=/usr/local/software/apache-hive-2.3.0-bin/conf

 

 

7、下载 mysql驱动包

执行命令:

cd/usr/local/software/apache-hive-2.3.0-bin/lib/

wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.38/mysql-connector-java-5.1.38.jar

 

 

3. 初始化MySQL

1、MySQL数据库进行初始化,首先确保 mysql 中已经创建 hive库:

执行命令:

cd /usr/local/software/apache-hive-2.3.0-bin/bin

 ./schematool-initSchema -dbType mysql

 

              看到红框的内容表示初始化成功。

2、MySQL数据库进行初始化,首先确保 mysql 中已经创建 hive库:

 

3、查看hive库中所有的表

 

14.启动Hive

1、启动Hive

执行命令:/usr/local/software/apache-hive-2.3.0-bin/bin/hive

 

              在命令行显示:OK表示已经查看数据库成功。

原文地址:https://www.cnblogs.com/xiatian21/p/13905153.html