Ubuntu + Apache Kafka Installation

http://vulab.com/blog/?p=576

Aim of this Kafka Tutorial:

Install Apache Kafka, Create a topic and use a producer to send test messages to Kafka Topics. Also launch a consumer to consume messages from Kafka Topics.

Step 1:

Our aim is to install a Ubuntu 64bit server using Oracle Virtual box.

Please download Ubuntu server from this link Download Ubuntu Server

Please download Oracle Virtual Box from this link Download Virtual Box

Follow the instructions online to get your Virtual Box started with Ubuntu 64 bit server. Google is your best bet to get the correct instructions for this step.

Note: Make sure you have JDK installed on the OS.

On my machine java -version is displaying the following version
vulab-build-system:~$ java -version
java version “1.8.0_11″
Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)

Step 2:

Download Apache Kafka version 0.8.1.1 from Download Apache Kafka

You can use wget http://mirror.sdunix.com/apache/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz

Step 3:

Unzip or expand the zip file downloaded at previous step.

tar -xzf kafka_2.9.2-0.8.1.1.tgzCD to the folder and explore the folder structure.

Step 4:

Apache Kafka requires zookeeper to be available and running.

Start zookeeper with the following command

bin/zookeeper-server-start.sh config/zookeeper.properties

Step 5:

Start Kafka Server using the following command

bin/kafka-server-start.sh config/server.properties

Step 6:

Create a topic in Apache Kafka using the following command

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testOutput will be displayed as Created topic "test"

Step 7:

Ask Zookeeper to list available topics on Apache Kafka.

Command is as below:

bin/kafka-topics.sh --list --zookeeper localhost:2181Output will be displayed as test

Step 8:

Send test messages to Apache Kafka topic called Test using command line producer

Please use the below command

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testPlease enter some messages like "Hello vulab" press enterEnter another message "How are you kafka?" and press enter.

Step 9:

Use command line consumer to check for messages on Apache Kafka Topic called testPlease use the following command

bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic test –from-beginning
You will see messages printed on the console like “Hello vulab”
With this step you have successfully verified that you have a valid Apache Kafka setup with Apache Zookeeper.

We had Success and what is next?

You did good. In the next step we are going to setup an Apache Maven project in Eclipse IDE. We will use Apache Kafka Producer API and Apache Consumer API to publish and subscribe messages using Apache Kafka. Get ready for the next tutorial in this Apache Kafka learning series by Vulab.

- See more at: http://vulab.com/blog/?p=576#sthash.CijJE2Da.dpuf