Python client for publishing and consuming message from Apache Kafka

In the Java Client for publishing and consuming messages from Apache Kafka i talked about how to create a Java Client for publishing and consuming messages from Kafka. I wanted to try same thing using Python so i followed these steps
  1. Follow steps 1 through 4 of Java Client for publishing and consuming messages from Apache Kafka entry to start Zookeeper and Kafka server
  2. Follow these steps to install Kafka python client
    
    git clone https://github.com/mumrah/kafka-python
    pip install ./kafka-python
    
  3. Next create a Producer.py python script to publish message to pythontest topic, the basic concept here is you connect to Kafka server on localhost:9092 port and then publish a message to a particular topic
  4. Now create Consumer.py script that consumes messages from pythontest topic and writes them to console.

5 comments:

Anonymous said...

the consumer and producer aren't using the same topic

Nirmal Jose said...

Change "pythontest" to "test" in producer. :)

Anonymous said...

what is KafkaClient port should be

Anonymous said...

Iam getting this error while running producer.py code

kafka = KafkaClient("localhost:9092")
TypeError: __init__() takes exactly 1 argument (2 given)

Anonymous said...

SimpleConsumer and SimpleProducer is deprecated, you should update to KafkaProducer and KafkaConsumer.