0. 실행 환경

AWS EC2 t2.xlarge
OS : Red Hat 9.1
Kafka :3.3.1
Scala : 2.13

1. Python으로 Kafka producer, consumer 만들기

producer.py

from kafka import KafkaProducer

# create kafka producer instance
producer = KafkaProducer(bootstrap_servers = ['localhost:9092'])

# set topic name
producer.send('first-topic', b'hello world')
# reset buffer
producer.flush()

comsumer.py

from kafka import KafkaConsumer

# creae kafka consumer instance
consumer = KafkaConsumer('first-topic', bootstrap_servers=['localhost:9092'])

# print message
for msg in consumer:
    print(msg)

python consumer.py
python producer.py

consumer.py 실행해 놓고 다른 창에서 producer.py 실행

토픽 이름, offset(메세지 온 순서), timestamp, 내용 등이 전달된다.