官方安装文档
在线安装文档
Installing Cloudera Manager, CDH, and Managed Services
离线安装参考
版本：CDH 6.3.1
安装指引：<https://docs.cloudera.com/documentation/enterprise/latest/topics/install_cm_cdh.html#cmig_topic_6_6>
-------------------------------------------------------------------------------------------------------
》安装前准备
1. 集群角色规划
- 集群网络
cdh-test => bridge
subnet 172.20.0.0/16
gateway 172.20.10.11
ip range 172.20.10.128/25
- 网路规划
cdn00 172.20.0.3 cm、server、db、agent
cdh01 172.20.0.7 agent
cdh02 172.20.0.12 agent
cdh03 172.20.0.16 agent
[hosts]
172.20.0.3 cdn00
172.20.0.7 cdh01
172.20.0.12 cdh02
172.20.0.16 cdh03
2. 静态 ip 设置，指定 hostname，修改 host 映射 -all node
容器启动
docker run -itd -h cdh00 --name cdh00 \\
-p 7180:7180 \\
--net cdh-test --ip 172.20.0.3 \\
--add-host=cdn01:172.20.0.7 \\
--add-host=cdn02:172.20.0.12 \\
--add-host=cdn03:172.20.0.16 \\
--privileged \\
yulinying/centos7 /usr/sbin/init
docker run -itd -h cdh01 --name cdh01 \\
--net cdh-test --ip 172.20.0.7 \\
--add-host=cdn00:172.20.0.3 \\
--add-host=cdn02:172.20.0.12 \\
--add-host=cdn03:172.20.0.16 \\
--privileged \\
yulinying/centos7 /usr/sbin/init
docker run -itd -h cdh02 --name cdh02 \\
--net cdh-test --ip 172.20.0.12 \\
--add-host=cdn00:172.20.0.3 \\
--add-host=cdn01:172.20.0.7 \\
--add-host=cdn03:172.20.0.16 \\
--privileged \\
yulinying/centos7 /usr/sbin/init
docker run -itd -h cdh03 --name cdh03 \\
--net cdh-test --ip 172.20.0.16 \\
--add-host=cdn00:172.20.0.3 \\
--add-host=cdn01:172.20.0.7 \\
--add-host=cdn02:172.20.0.12 \\
--privileged \\
yulinying/centos7 /usr/sbin/init
3。禁用防火墙 - 容器省略 -all node
# 禁止防火墙开启自启
systemctl disable firewalld
# 关闭防火墙
systemctl stop firewalld
4. 关闭 SELinux - 容器省略 -all node
# 检查 selinux 状态
getenforce
# 关闭 selinux
vim /etc/selinux/config —> SELINUX=permissive
# 重启并强制更新 selinux 状态
setenforce 0
# 安装 cdh 后可以重新恢复 selinux 状态
vim /etc/selinux/config —> SELINUX=enforcing
setenforce 1
5. 配置各个节点 ssh 免密登录 -all node
manager 节点：cdh1
# 生成公钥文件，默认在 /root/.ssh/下面会生成一个公钥文件id_rsa.pub
ssh-keygen -t rsa
# 公钥追加到 authorized_keys
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# 修改可读权限
chmod 600 ~/.ssh/authorized_keys
# 分发 ~/.ssh 到各个 agent 节点
scp -r ~/.ssh/ root@cdh01:~/.ssh/
scp -r ~/.ssh/ root@cdh02:~/.ssh/
scp -r ~/.ssh/ root@cdh03:~/.ssh/
6. 安装并启动 ntp 时间校准服务 -all node
yum -y install ntp
ntpdate -u time.pool.aliyun.com
systemctl start ntpd
systemctl enable ntpd
7. 确认 python 版本是否为 2.7 -all node
-------------------------------------------------------------------------------------------------------
》安装 CM
1. 配置 CM 仓库 -cm node
wget <https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo> -P /etc/yum.repos.d/
2. 安装 jdk -cm node
# 删除原有jdk版本
# 安装 cloudera 兼容 oracle-jdk
yum install oracle-j2sdk1.8
3. 安装cm服务 -cm node
yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
4. 安装配置数据库 -db node
这里选用 MariaDB
# 安装 mariaDB
yum install mariadb-server
# 停止 mariaDB 服务
systemctl stop mariadb
# 备份原 my.cnf 配置文件
mv /etc/my.cnf /etc/my.cnf.backup
# 修改 my.cnf 配置文件 ：<https://docs.cloudera.com/documentation/enterprise/latest/topics/install_cm_mariadb.html#install_cm_mariadb>
# mariaDB 服务设置开机启动
systemctl enable mariadb
# 启动 mariaDB
systemctl enable mariadb
# 配置 mariadb 安全相关设置
/usr/bin/mysql_secure_installation
password：root
---
Set root password? [Y/n] Y
New password:
Re-enter new password:
[...]
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
---
5. 下载配置 MySQL JDBC Driver
# 下载 JDBC Driver、解压
wget <https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz>
tar zxvf mysql-connector-java-5.1.46.tar.gz
# 移动到 Java 指定目录
cd mysql-connector-java-5.1.46
cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
6. 创建 cloudera 相关数据库和账户权限
<https://docs.cloudera.com/documentation/enterprise/latest/topics/install_cm_mariadb.html#install_cm_mariadb>
mysql -u root -p
# cloudera 相关数据库
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
# 用户权限授予
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'root';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'root';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'root';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'root';
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'root';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'root';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'root';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'root';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'root';
7. 配置 CM 数据库 -cm node
# 执行数据库配置脚本 -scm
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm root
8. 启动 CDH
# 启动 scm 服务
systemctl start cloudera-scm-server
# 启动过程中查看启动日志
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
# 启动成功标志：INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
# 浏览器中访问
http:<hostname>:7180
# 默认账户密码： admin | admin
# 进入启动导向 <https://docs.cloudera.com/documentation/enterprise/latest/topics/install_software_cm_wizard.html>
Edition 选择 Cloudera Express
# 集群安装
Cluster Basics：填写集群名称
Specify Hosts：主机名称：cdh1,cdh2,cdh3 SSH端口：22
# 选择储存库
# JDK 安装：勾选
# SSH 登录凭证：接受相同密码
# install Agent：等待安装