flume sink kafka

agent.sources = s1   
agent.channels = c1 
agent.sinks = sk1 
 
agent.sources.s1.type = netcat 
agent.sources.s1.bind = localhost 
agent.sources.s1.port = 5678 
agent.sources.s1.channels = c1 

agent.sinks.sk1.type = org.apache.flume.sink.kafka.KafkaSink
agent.sinks.sk1.topic = hellotest
agent.sinks.sk1.brokerList = localhost:9092
agent.sinks.sk1.requiredAcks = 1
agent.sinks.sk1.batchSize = 20
agent.sinks.sk1.channel = c1
 
agent.channels.c1.type = memory
agent.channels.c1.capacity = 1000    
agent.channels.c1.transactionCapacity = 100

 开启zookeeper和kafka,创建一个hellotest的topic

Last login: Tue Mar 21 10:57:36 on ttys004
lihudeMacBook-Pro:~ SunAndLi$  telnet  localhost 5678
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
lihuhu 
OK
lihuhu is my husband
OK

 查看

Last login: Tue Mar 21 10:56:50 on ttys003
lihudeMacBook-Pro:~ SunAndLi$ cd kafka_2.11-0.10.1.0/
lihudeMacBook-Pro:kafka_2.11-0.10.1.0 SunAndLi$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic hellotest --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
lihuhu
lihuhu is my husband
原文地址:https://www.cnblogs.com/sunyaxue/p/6593992.html