Python client for publishing and consuming message from Apache Kafka

In the Java Client for publishing and consuming messages from Apache Kafka i talked about how to create a Java Client for publishing and consuming messages from Kafka. I wanted to try same thing using Python so i followed these steps
  1. Follow steps 1 through 4 of Java Client for publishing and consuming messages from Apache Kafka entry to start Zookeeper and Kafka server
  2. Follow these steps to install Kafka python client 
    
git clone https://github.com/mumrah/kafka-python
pip install ./kafka-python
  3. Next create a Producer.py python script to publish message to pythontest topic, the basic concept here is you connect to Kafka server on localhost:9092 port and then publish a message to a particular topic
  4. Now create Consumer.py script that consumes messages from pythontest topic and writes them to console.

5 comments:

  1. AnonymousMay 29, 2015 at 9:39 AM

    the consumer and producer aren't using the same topic

    ReplyDelete
  2. Nirmal JoseSeptember 10, 2015 at 9:51 AM

    Change "pythontest" to "test" in producer. :)

    ReplyDelete
  3. AnonymousJuly 14, 2016 at 10:22 PM

    what is KafkaClient port should be

    ReplyDelete
  4. AnonymousNovember 3, 2016 at 12:50 AM

    Iam getting this error while running producer.py code

    kafka = KafkaClient("localhost:9092")
    TypeError: __init__() takes exactly 1 argument (2 given)

    ReplyDelete
  5. AnonymousNovember 7, 2016 at 10:37 AM

    SimpleConsumer and SimpleProducer is deprecated, you should update to KafkaProducer and KafkaConsumer.

    ReplyDelete