Centos7搭建kafka集群

1. 环境

环境	ip	软件
Centos7	192.168.2.5(node01)	jdk,zookeeper,kafka
Centos7	192.168.2.6(node02)	jdk,zookeeper,kafka
Centos7	192.168.2.7(node03)	jdk,zookeeper,kafka

已经搭建好的zookeeper集群

kafka下载地址

2. 安装

这里从192.168.2.5开始

# 下载解压
cd /usr/local/
wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.6.0/kafka_2.12-2.6.0.tgz
tar -zxvf kafka_2.12-2.6.0.tgz
mv kafka_2.12-2.6.0 kafka

配置kafka

vi /usr/local/kafka/config/server.properties
// 修改如下内容
// 依次增长的整数，0、1、2，集群中Broker的唯一id
broker.id：0
zookeeper.connect=192.168.2.5:2181,192.168.2.6:2181,192.168.2.7:2181
advertised.listeners=PLAINTEXT://192.168.2.5:9092

#分发到node02节点：　　
    scp -r /usr/local/kafka/ 192.168.2.6:/usr/local/kafka/

#分发到node03节点： 
　　 scp -r /usr/local/kafka/ 192.168.2.7:/usr/local/kafka/

修改其它节点配置文件

#node02节点
vim /usr/local/kafka/config/server.properties

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=1
advertised.listeners=PLAINTEXT://192.168.2.6:9092

#node03节点
vim /usr/local/kafka/config/server.properties

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=2
advertised.listeners=PLAINTEXT://192.168.2.7:9092

启动kafka集群

#分别在三台节点执行：node01/node02/node03

#进入到kafka的bin目录 
cd /usr/local/kafka/bin 
./kafka-server-start.sh -daemon ../config/server.properties

kafka的基本使用

#查看topic 列表：
/usr/local/kafka/bin/kafka-topics.sh --list --zookeeper 192.168.2.5:2181,192.168.2.6:2181,192.168.2.7:2181
    
#查看指定topic：
/usr/local/kafka/bin/kafka-topics.sh  --describe --zookeeper 192.168.2.5:2181,192.168.2.6:2181,192.168.2.7:2181 --topic topic_test
    
#创建topic
# --create：表示创建
# --zookeeper 后面的参数是zk的集群节点
# --replication-factor 1 ：表示复本数
# --partitions 1：表示分区数
# --topic topic_test：表示topic的主题名称

/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 192.168.2.5:2181,192.168.2.6:2181,192.168.2.7:2181 --replication-factor 1 --partitions 1 --topic topic_test

#删除topic
/usr/local/kafka/bin/kafka-topics.sh --delete --zookeeper 192.168.2.5:2181,192.168.2.6:2181,192.168.2.7:2181 --topic topic_test

# 创建生产者
/usr/local/kafka/bin/kafka-console-producer.sh --broker-list 192.2.5.181:9092,192.168.2.6:9092,192.168.2.7:9092 --topic topic_test

# 创建消费者
/usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server 192.2.5.181:9092,192.168.2.6:9092,192.168.2.7:9092 --topic topic_test --consumer-property  --offset 0