Big Data Cluster Ansible Automated Deployment Tutorial

pblh123@126.com (John Lee) — Fri, 29 May 2026 00:00:00 +0000

Big Data Cluster Ansible Automated Deployment Tutorial
#

An Ansible-based declarative automation deployment solution, coexisting with existing Shell/Python scripts, for 3-10 node clusters.

1. Prerequisites
#

1.1 Control Node Requirements
#

Requirement	Details
OS	Linux / macOS (Windows WSL compatible)
Ansible	>= 2.15
sshpass	`yum install sshpass` (CentOS) or `apt install sshpass` (Ubuntu)
Network	SSH access to all cluster nodes

1.2 Cluster Node Requirements
#

Requirement	Details
OS	CentOS 7.9+ / Rocky Linux 8+
SSH	Enabled with password or key authentication
Sudo	hadoop user has sudo privileges
Disk	`/opt/module` at least 10GB free

1.3 Install Ansible
#

# CentOS / Rocky
sudo yum install -y epel-release && sudo yum install -y ansible

# Ubuntu / Debian
sudo apt update && sudo apt install -y ansible

# macOS
brew install ansible

# Verify
ansible --version # Should show >= 2.15

1.4 Pre-cache Component Archives (Recommended)
#

cd ansible/files/cache/

# Download all components (~1.8 GB total)
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.9/apache-zookeeper-3.5.9-bin.tar.gz
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/2.6.4/hbase-2.6.4-hadoop3-bin.tar.gz
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.8.0/kafka_2.13-2.8.0.tgz
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/spark/spark-3.5.8/spark-3.5.8-bin-hadoop3-scala2.13.tgz
wget -c https://maven.aliyun.com/repository/public/mysql/mysql-connector-java/5.1.49/mysql-connector-java-5.1.49.jar

JDK should be pre-installed to /opt/module/jdk1.8.0_471 on all nodes, or can be auto-downloaded via the common role.

Automation on 李先生的树洞

Big Data Cluster Ansible Automated Deployment Tutorial

Big Data Cluster Ansible Automated Deployment Tutorial #

1. Prerequisites #

1.1 Control Node Requirements #

1.2 Cluster Node Requirements #

1.3 Install Ansible #

1.4 Pre-cache Component Archives (Recommended) #

Big Data Cluster Ansible Automated Deployment Tutorial
#

1. Prerequisites
#

1.1 Control Node Requirements
#

1.2 Cluster Node Requirements
#

1.3 Install Ansible
#

1.4 Pre-cache Component Archives (Recommended)
#