kafka的使用

kafka:

  Broker

  Producer

  Consumer

  Topic

  Partition

  Consumer group

一、搭建kafka集群

(1)准备zookeeper集群环境

  准备3台服务器

  pwd查看目录

  启动zookeeper集群:    sh ../zookeeper-3.4.10/bin/zkServer.sh start

  sh ../zookeeper-3.4.10/bin/zkCli.sh

  下载kafka_2.12-0.11.0.1.tgz

  解压:tar -zxvf kafka_2.12-0.11.0.1.tgz

  cd kafka_2.12-0.11.0.1

  (bin,config,libs,site-docs)

  cd config/

  vim server.properties

  1)修改唯一的broker.id

  2)修改listeners=PLAINTEXT://192.168.11.140:9092

  3)zookeeper.connect=192.168.11.140:2181,192.168.11.137:2181,192.168.11.138:2181

  4)启动服务

 

  启动命令:bin/kafka-server-start -daemon ../config/server.properties

 

  cd zookeeper bin/执行命令

 

  sh kafka-server-start.sh -daemon ../config/server.properties

 

  此处不属于启动命令(查看启动日志:

 

  tail  -100 ../logs/kafkaServer.out

 

  关闭防火墙:

 

  systemctl stop firewalld

 

  查看服务是否启动

 

  新打开一个shell窗口

 

  cd ../zookeeper-3.4.10/bin/

 

  sh zkCli.sh

 

  cd /brokers/ids/

 

  查看当前kafka的leader:

 

  get /controller

 

  )

 

  打开leader shell

(2)通过命令实现消息的发送和接收

  1)创建topic

  sh -kafka-topics.sh --Create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic first

  cd /brokers/topics

  sh kafka-console-customer.sh --broker-list 192.168.11.140:9092,192.168.11.140:9092,192.168.11.140:9092 --topic first --from-beginning(错误,应该连接zookeeper)

  sh kafka-console-customer.sh --zookeeper 192.168.11.137:2181 --topic first --from-beginning

  2)发送消息:

  sh kafka-console-producer.sh --broker-list 192.168.11.140:9092 --topic first

  输入消息即可发送消息

原文地址:https://www.cnblogs.com/wzb0228/p/10368618.html