简介
#

本文将介绍分布式部署hadoop方式，各角色部署使用独立的用户

HDFS 部署
httpfs 部署
yarn 部署
hive 部署
HBase 部署

环境准备
#

CentOS-7(Ubuntu亦可，或使用docker容器部署)
hadoop-2.6.5.tar.gz
zookeeper-3.4.12.tar.gz

计划
#

hostname	作用
node1	namenode, datanode, JournalNode, zookeeper
node2	namenode, datanode, JournalNode, zookeeper
node3	datanode, JournalNode, zookeeper, httpfs

安装步骤
#

配置zookeeper
#

# 添加zookeeper用户
useradd zookeeper

# 解压并安装zookeeper包
tar -xvf zookeeper-3.4.12.tar.gz
mv zookeeper-3.4.12 /opt/
chown root:root /opt/zookeeper-3.4.12/ -R
ln -vs /opt/zookeeper-3.4.12/ /opt/zookeeper

# 创建配置文件
cp /opt/zookeeper/conf/zoo_sample.cfg /opt/zookeeper/conf/zoo.cfg

# 创建日志和数据目录
mkdir /var/log/zookeeper
chown zookeeper:zookeeper /var/log/zookeeper/

mkdir /var/lib/zookeeper
chown zookeeper:zookeeper /var/lib/zookeeper/

在node1上执行：echo 1 > /var/lib/zookeeper/myid 在node2上执行：echo 2 > /var/lib/zookeeper/myid 在node3上执行：echo 3 > /var/lib/zookeeper/myid

执行vim /opt/zookeeper/conf/zoo.cfg 底部添加如下内容

dataDir=/var/lib/zookeeper
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888

执行vim /opt/zookeeper/bin/zkEnv.sh 修改如下内容

if [ "x${ZOO_LOG_DIR}" = "x" ]
then
    ZOO_LOG_DIR="/var/log/zookeeper"
fi

在node1, node2, node3上启动zookeeper

# 启动
su zookeeper -c "/opt/zookeeper/bin/zkServer.sh start"
# 停止
su zookeeper -c "/opt/zookeeper/bin/zkServer.sh stop"

SSH 配置
#

# 创建用户组hadoop
groupadd hadoop

# 创建用户hdfs并设置用户组为hadoop
useradd hdfs -g hadoop

# 设置hdfs密码
passwd hdfs

su hdfs

# 一路回车
ssh-keygen

# 将本主机的公钥分发至node1
ssh-copy-id node1

# 使用ssh-copy-id将：
# node1, node2的公钥分发至node1, node2, node3
# node3的公钥分发至node3

配置HDFS
#

切换至root用户

安装和配置

# 解压hadoop安装包
tar -xvf hadoop-2.6.5.tar.gz

mv hadoop-2.6.5 /opt/

chown root:root /opt/hadoop-2.6.5 -R

ln -vs /opt/hadoop-2.6.5/ /opt/hadoop

mkdir /var/log/hadoop-hdfs
chown hdfs:hadoop /var/log/hadoop-hdfs

mkdir /var/lib/hadoop-hdfs
chown hdfs:hadoop /var/lib/hadoop-hdfs

以下修改hdfs相关配置，配置文件位于目录/opt/hadoop/ect/hadoop

执行vim hadoop-env.sh 修改如下对应配置

export HADOOP_LOG_DIR=/var/log/hadoop-hdfs
# 此处注意修改为环境中对应的JAVA_HOME路径
export JAVA_HOME=/usr/java/jdk1.8.0_171

执行vim core-site.xml，其中<configuration>节点修改为如下内容

<configuration>

    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/var/lib/hadoop-hdfs</value>
    </property>

    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://mycluster</value>
    </property>

    <property>
        <name>ha.zookeeper.quorum</name>
        <value>node1:2181,node2:2181,node3:2181</value>
    </property>

</configuration>

执行vim hdfs-site.xml，其中<configuration>节点修改为如下内容

<configuration>

    <property>
        <name>dfs.nameservices</name>
        <value>mycluster</value>
    </property>

    <property>
        <name>dfs.ha.namenodes.mycluster</name>
        <value>node1,node2</value>
    </property>

    <property>
        <name>dfs.namenode.rpc-address.mycluster.node1</name>
        <value>node1:8020</value>
    </property>

    <property>
        <name>dfs.namenode.http-address.mycluster.node1</name>
        <value>node1:50070</value>
    </property>

    <property>
        <name>dfs.namenode.rpc-address.mycluster.node2</name>
        <value>node2:8020</value>
    </property>

    <property>
        <name>dfs.namenode.http-address.mycluster.node2</name>
        <value>node2:50070</value>
    </property>

    <property>
        <name>dfs.namenode.shared.edits.dir</name>
        <value>qjournal://node1:8485;node2:8485;node3:8485/journal</value>
    </property>

    <property>
        <name>dfs.journalnode.edits.dir</name>
        <value>/var/lib/hadoop-hdfs</value>
    </property>

    <property>
        <name>dfs.ha.automatic-failover.enabled</name>
        <value>true</value>
    </property>

    <property>
        <name>dfs.client.failover.proxy.provider.mycluster</name>
        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>

    <property>
        <name>dfs.ha.fencing.methods</name>
        <value>
            sshfence
            shell(/bin/true)
        </value>
    </property>

    <property>
        <name>dfs.ha.fencing.ssh.private-key-files</name>
        <value>/root/.ssh/id_rsa</value>
    </property>

    <property>
        <name>dfs.ha.fencing.ssh.connect-timeout</name>
        <value>30000</value>
    </property>

    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/var/lib/hadoop-hdfs/nn</value>
    </property>

    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/var/lib/hadoop-hdfs/dn</value>
    </property>

</configuration>

执行vim slaves 替换为如下内容

node1
node2
node3

执行cp mapred-site.xml.template mapred-site.xml 复制mapred-site.xml.template为mapred-site.xml
执行vim mapred-site.xml，其中<configuration>节点修改为如下内容

<configuration>

    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

</configuration>

启动journalnode

su hdfs -c "/opt/hadoop/sbin/hadoop-daemon.sh start journalnode"
# 对应停止命令如下
# su hdfs -c "/opt/hadoop/sbin/hadoop-daemon.sh stop journalnode"

使用jps可以查到JournalNode进程
如果无法使用jps则需要先导入java环境变量，例如使用source /etc/profile

初始化namenode

# node1上执行
su hdfs -c "/opt/hadoop/bin/hdfs namenode -format"
su hdfs -c "scp -r /var/lib/hadoop-hdfs/nn hdfs@node2:/var/lib/hadoop-hdfs"
su hdfs -c "/opt/hadoop/bin/hdfs zkfc -formatZK"
su hdfs -c "/opt/hadoop/sbin/start-dfs.sh"

打开http://node1:50070或http://node2:50070查看hdfs信息

配置HttpFs
#

```sh
# 创建用户
useradd httpfs

# 创建目录
mkdir /var/log/hadoop-httpfs
chown httpfs:httpfs /var/log/hadoop-httpfs/

mkdir /var/tmp/hadoop-httpfs
chown httpfs:httpfs /var/tmp/hadoop-httpfs/

# 设置权限
chmod +r /opt/hadoop/share/hadoop/httpfs/tomcat/conf -R
```

vim httpfs-site.xml
```xml
<configuration>

    <property>
        <name>httpfs.proxyuser.hue.hosts</name>
        <value>*</value>
    </property>

    <property>
        <name>httpfs.proxyuser.hue.groups</name>
        <value>*</value>
    </property>

</configuration>
```

vim core-site.xml 添加
```xml
<property>
    <name>hadoop.proxyuser.httpfs.hosts</name>
    <value>*</value>
</property>

<property>
    <name>hadoop.proxyuser.httpfs.groups</name>
    <value>*</value>
</property>
```

vim httpfs-env.sh
```sh
export HTTPFS_LOG=/var/log/hadoop-httpfs
export HTTPFS_TEMP=/var/tmp/hadoop-httpfs
```

启动或停止
```bash
su httpfs -c "/opt/hadoop/sbin/httpfs.sh start"
su httpfs -c "/opt/hadoop/sbin/httpfs.sh stop"
```

附
#

以下为本文档中所使用的docker容器构建和编排相关的文档，如果需要使用容器部署hadoop，可以参考使用

Dockerfile
#

构建centos7-base镜像所使用的相关文件如下

目录结构

├── Dockerfile
└── files
    └── jdk-8u171-linux-x64.tar.gz

jdk下载地址：
jdk-8u171-linux-x64.tar.gz

Dockerfile内容

FROM centos:7.2.1511

MAINTAINER yancai
ENV TZ Asia/Shanghai

# 设置yum源 安装常用工具
RUN curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo \
    && yum makecache \
    && yum install -y vim mlocate telnet zlib-devel openssh-server openssh-clients net-tools \
    && mkdir /usr/java

WORKDIR /root

# 安装jdk
ADD files/jdk-8u171-linux-x64.tar.gz /usr/java
RUN echo "export JAVA_HOME=/usr/java/jdk1.8.0_171" >> /etc/profile \
    && echo "export PATH=\$JAVA_HOME/bin:\$PATH" >> /etc/profile \
    && echo "export CLASSPATH=.:\$JAVA_HOME/lib/dt.jar:\$JAVA_HOME/lib/tools.jar" >> /etc/profile \
    && ln -vs /usr/java/jdk1.8.0_171/bin/java /usr/bin/java

在Dockerfile所在目录使用sudo docker build --tag="centos7-base" .构建镜像

docker-compose.yml
#

以下为docker-compose.yml内容：

version: '2.2'
services:
    node1:
        image: centos7-base
        container_name: node1
        hostname: node1
        environment:
            - TZ=Asia/Shanghai
        command: tail -f /dev/null
        volumes:
            - ~/share:/root/share

    node2:
        image: centos7-base
        container_name: node2
        hostname: node2
        environment:
            - TZ=Asia/Shanghai
        command: tail -f /dev/null
        volumes:
            - ~/share:/root/share

    node3:
        image: centos7-base
        container_name: node3
        hostname: node3
        environment:
            - TZ=Asia/Shanghai
        command: tail -f /dev/null
        volumes:
            - ~/share:/root/share

在docker-compose.yml所在目录使用命令sudo docker-compose up -d启动容器

使用docker容器部署时，需要开启SSH服务，手动开启方式如下：

# 在容器内执行
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
/usr/sbin/sshd

简介#

环境准备#

计划#

安装步骤#

配置zookeeper#

SSH 配置#

配置HDFS#

配置HttpFs#

附#

Dockerfile#

docker-compose.yml#