Spark Kafka Streaming Java program Word Count using Kafka 0.10 API

Kafka API went through a lot of changes starting Kafka 0.9. Spark Kafka Streaming API also was changed to better support Kafka 0.9. i wanted to try that out so i built this simple Word Count application using Kafka 0.10 API. This blog entry does the same thing but using Scala. You can download the complete application from github
You can run this sample by first downloading Kafka 0.10.* from Apache Kafka WebSite, then you can create and start a test topic and send messages to it by following this Kafka Quick start document First thing i did was to include Kafka 0.10 API dependencies for the Spark Project. As you can see i am using Spark 2.1 version Then i did create a SparkKafka10.java file that looks like this. Please take a look at comments inside the code for what i am doing. Now if you create test topic and send messages to it, you should be able to see the wordcount on console

26 comments:

Robert Welain said...

Check out new service that helped me to write my research paper!

Academic Essay Writing Tips said...

Thank you. Got a lot of useful information for yourself.

Heena Madan said...

Hi,

Great post thanks.

I have a small query. If my kafka topic contains json data then how we can fetch data from topic using spark java?

4 Square Logic said...

If you are thinking to create the latest Android apps, likes games, music, movies, TV, books, magazines & more then join us 4Square Logic IT Solutions we are providing you Android app Development services within your budget. Our team of developers making the best attractive apps which is easy to use and enhance your business.

Delta Airlines Phone Number said...

Hey, I must say thank you to share such fantastic post with us. Here I will tell you about Delta Airlines Phone Number. This number will help those people who is looking for best discount on flights. So if you stuck in future while booking Delta Airlines flights, must contact us at Delta Airlines Phone Number.

Frank Castle said...

The high standards like the height of the magnificent sky but the prices are low like beautiful land. Every year more than 100 million passengers reach the clouds with Delta Airlines. Be a part of this ever-growing family, book tickets with Delta Airlines Phone Number now.

Stella Mills said...

Outstanding blog, thank you for sharing such a knowledgeable blog. keep sharing the subsequent posts.
Thanks and Regards,
American Airlines Phone Number

customersupport said...

Its a very nice and kind thought. I appreciate this post. Please check out my blog also. In which i have written about Delta Airlines services. To know more about its services reach us at our; Delta Airlines Phone Number. Thank You! Visit site - https://delta-airlinesnumber.com/

Banu Ravi said...

Great blog, I was searching this for a while. Do post more like this.
AngularJS Training in Chennai
Angular 4 Training in Chennai
Angular 5 Training in Chennai
Angular Training in Chennai
ReactJS Training in Chennai
PHP course in Chennai
Web Designing Training in Chennai
AngularJS Training in Anna Nagar
AngularJS Training in Vadapalani
AngularJS Training in Velachery

Danishsingh said...

Thanks for sharing such a great blog Keep posting.
marketing automation software
marketing automation tools India
sales automation
relationship intelligence
business directory
b2b database providers in India

RICK HARRISON said...

Hello, I’m Rick from Airlines Reservations and offering you valuable information regarding the United Airlines Flights Reservations. I am enthusiastic about travelling and would like to share some information about it. So if you want to would like some travelling tips and information about
United Airlines Reservations
, start following my blogs.

steve said...

Top Web development service company in Brampton
Best Web design company in Toronto
Developing from a group of specialists to an undeniable top SEO office in Canada, we have confidence in persistently redesigning ourselves with the goal that we can offer the most recent types of assistance to our customers

Emirates Airways Low Fare Calendar said...

Very good points you wrote here..Great stuffGreat work! This is the kind of info that should be shared around the internet..The best way to earn experience is traveling. But, the reality is traveling is not the cheap and expensive fare of transportation, especially air-transportation, often spoil plans. Not anymore So, Book you tickets from Emirates Airways Low Fare Calendar from where You will get to know about the lowest airfares available for each date

Qatar Customer Service said...


I visited a lot of website but I believe this one has something special in it thanks.If you are a family person and looking for an flight tickets in your budget,So without thinking too much just book your tickets from Qatar Customer Service and get ready to avail amazing deals and discounted tickets with best facilities.

Southwest Toll Free said...

I visited a lot of website but I believe this one has something special in it thanks.If you Standing clueless about the change, cancellation and Best offer and delas of Southwest Airlines? Don’t worry, we got you! Just call us at Southwest Toll Free , and get the most simplified yet effective briefing about all these processes. Our experts will remove your every doubt by their intelligence and knowledge.

Qatar Customer Service said...

I visited a lot of website but I believe this one has something special in it thanks.If you are a family person and looking for an flight tickets in your budget,So without thinking too much just book your tickets from Qatar Customer Service and get ready to avail amazing deals and discounted tickets with best facilities.

Turkish Airlines Fare Calendar said...

If you are thinking of going to somewhere, but confused about when to go, then you can even toggle between the months to finally decide when the least expensive and best time to book airfare is ,So just visit at Turkish Airlines Fare Calendar and pick your date with our special offers, and be ready to feel the joy of traveling at budget-friendly prices!

Smith said...

Word count is an usefull tool to count Word, Line, Page and Character in multiple files and also you can calculate amount and generate reports. word count tool

American Airlines Phone Number said...

Add the touch of comfort and convenience in your flight experience by choosing American Airlines as your flight partner. Purchase your flight tickets via American Airlines Phone Number to get the best deals, offers, and discounts. You can also purchase extra flight services easily including special dietary meal and excess baggage.

youtubecomactivate said...

Amazing Post, I really like your post. If any YouTube users need help regarding youtube.com/activate then they can contact us.

Techsolve Solutions said...

Thanks for sharing useful information. Keep on sharing.

Web design company toronto
Digital marketing agency toronto

Airlines Reservations said...

eva air flight change policy Passengers can request for the same day flight change through many available options provided by Eva Air. The passenger requires to pay Eva Air Ticket Change Fee for that. The change fee maybe 50 USD to 300 USD as per the ticket type and route of the passenger.

Airlines Rservations Helpdesk said...

For most trips, airfare is the most expensive part of the trip. While prices for transatlantic flights have gone down in recent years, they can still put a sizeable dent in any travel budget. Whether you’re a budget solo traveler or a family looking to vacation abroad, finding a cheap flight deal can be what makes or breaks your trip.
allegiant flight change policy

e said...


Hi, I'm Adrian, a passionate travel blogger. I have 2+ experience in travel blogging. I love adventures and hiking. With a passion for exploring the planet and bringing new lives into the world. Follow my blogs to know all my travel experiences and get amazing tips & tricks to make your travel experience amazing.
American Airlines Cancellation Policy

American airlines flight cancellation

Rihanna jones said...

Thanks for your article. It was interesting and informative. Here I also want to suggest to your readers who are usually travelling from one place to another that they must visit airlines-gethuman.org that offers best deals to book your seat on Delta airlines. Also check out Delta Airlines Flights before booking your travel vacation.
Delta Airlines Flight
Delta Airlines

volaris carry on cost said...

Many thanks for the efforts you have put into writing this site.