四、kafka+maven+springMVC实例

　　本文借助主流SpringMVC框架向大家介绍如何在具体应用中简单快捷的使用kafka。kafka、maven以及SpringMVC在现在的企业级应用中都占据着非常重要的地位，所以本文将三者结合起来也可以方便大家进一步熟悉基于Maven的SpringMVC框架搭建。

项目展示

　　国际惯例，首先先向大家展示一下项目最终的运行效果：

　　当项目正常启动后，在浏览器中输入：http://127.0.0.1:8080/kafkaSpringMVC/welcome 进入欢迎界面：

　　然后点击Send a Message 进入消息发送页面：

　　从上面可以看出，发送的消息是当前系统的时间（当然你也可以修改成为自己感冒的消息），点击Submit后将消息发送到kafka集群服务器，然后自动返回到Welcome欢迎界面。在欢迎界面点击Get a Message：

　　从上述界面中我们可以看见页面中已经获取到了刚才发送的消息，点击RETURN HOME，返回欢迎界面，好啦项目展示就这么简单。

开发环境

• 操作系统：MacOS 10.12.3（同样适用于Linux系统和Windows系统）
• JDK: java version "1.8.0_121"
• 开发平台：Eclipse Neon.2 Release (4.6.2)
• WEB容器：wildfly-8.1.0.Final
• zookeeper: zookeeper-3.4.9
• kafka: kafka-2.10-0.10.2.0
• maven: Eclipse Neon.2 Release(4.6.2)自带maven工具，版本为3.3.9

项目框架

　　项目框架如下图所示：

 

项目开发流程

　　首先搭建Maven Web Project框架，搭建过程可参考我的另一篇随笔maven web框架搭建，大家也可以在网上搜索更多的文章来学习，这里就不在重复描述啦。本项目命名为：kafkaSpringMVC。

　　框架搭建完成后，下面就需要引入Spring MVC 所需要的jar包和kafka客户端开发需要的jar包。本示例采用的Spring 版本为4.3.6.REALEASE，使用的kafka客户端版本为0.10.2.0。下面通过修改pom.xml文件来引入外部依赖包：

　　pom.xml

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 2     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> 3     <modelVersion>4.0.0</modelVersion> 4     <groupId>com.unionpay</groupId> 5     <artifactId>kafkaSpringMVC</artifactId> 6     <packaging>war</packaging> 7     <version>0.0.1-SNAPSHOT</version> 8     <name>kafkaSpringMVC Maven Webapp</name> 9     <url>http://maven.apache.org</url>10 11     <properties>12         <springframework>4.3.6.RELEASE</springframework>13     </properties>14 15     <dependencies>16         <dependency>17             <groupId>junit</groupId>18             <artifactId>junit</artifactId>19             <version>3.8.1</version>20             <scope>test</scope>21         </dependency>22 23         <dependency>24             <groupId>org.springframework</groupId>25             <artifactId>spring-core</artifactId>26             <version>${springframework}</version>27         </dependency>28 29         <dependency>30             <groupId>org.springframework</groupId>31             <artifactId>spring-context</artifactId>32             <version>${springframework}</version>33         </dependency>34 35         <dependency>36             <groupId>org.springframework</groupId>37             <artifactId>spring-tx</artifactId>38             <version>${springframework}</version>39         </dependency>40 41         <dependency>42             <groupId>org.springframework</groupId>43             <artifactId>spring-webmvc</artifactId>44             <version>${springframework}</version>45         </dependency>46 47         <dependency>48             <groupId>org.springframework</groupId>49             <artifactId>spring-web</artifactId>50             <version>${springframework}</version>51         </dependency>52 53         <dependency>54             <groupId>org.springframework</groupId>55             <artifactId>spring-jms</artifactId>56             <version>${springframework}</version>57         </dependency>58 59         <dependency>60             <groupId>org.apache.kafka</groupId>61             <artifactId>kafka-clients</artifactId>62             <version>0.10.2.0</version>63         </dependency>64 65     </dependencies>66     <build>67         <finalName>kafkaSpringMVC</finalName>68     </build>69 </project>

 

　　写完pom.xml文件后，保存。然后右键项目名称，选择Maven->Update Project... 更新项目，引入jar包。

 

　　Update Project完成后，可以在maven依赖包里看见刚才引入的本项目需要的jar包：

　　接下来编写web.xml文件：

　　web.xml

<?xml version="1.0" encoding="UTF-8"?> 2 <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 3     xmlns="http://java.sun.com/xml/ns/javaee" 4     xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" 5     id="WebApp_ID" version="3.0"> 6      7     <context-param> 8         <param-name>contextConfigLocation</param-name> 9         <param-value>classpath:kafka-beans.xml</param-value>10     </context-param>11     12     <listener>13         <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>14     </listener>15     16     <servlet>17         <servlet-name>springDispatcherServlet</servlet-name>18         <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>19         <init-param>20             <param-name>contextConfigLocation</param-name>21             <param-value>classpath:spring-mvc-dispatcher.xml</param-value>22         </init-param>23         <load-on-startup>1</load-on-startup>24     </servlet>25     26     <servlet-mapping>27         <servlet-name>springDispatcherServlet</servlet-name>28         <url-pattern>/</url-pattern>29     </servlet-mapping>30     31 </web-app>

　　简单明了，从web.xml描述来看，MVC映射是通过spring-mvc-dispatcher.xml文件说明的，而项目中用到的所有的model 则是通过kafka-beans.xml文件注入的。那么下面分别在resources文件夹下建立这两个xml配置文件：

　　spring-mvc-dispatcher.xml

<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans"    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"    xmlns:mvc="http://www.springframework.org/schema/mvc"    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd">    <context:component-scan base-package="com.unionpay.controller"></context:component-scan>        <bean        class="org.springframework.web.servlet.view.InternalResourceViewResolver">        <property name="prefix" value="/WEB-INF/views/"></property>        <property name="suffix" value=".jsp"></property>    </bean>    </beans>

　　就这么简单。由于kafka-beans.xml主要作用是配置注入的beans，所以按照程序，还是先建立好Model才符合逻辑。

　　在src/main/java路径下分别建立三个package包：com.unionpay.producer、com.unionpay.consumer、com.unionpay.controller。从字面意思很容易理解，producer包主要用于存放生产者，consumer包主要用于存放消费者，controller包主要用于存放逻辑控制类。

　　完成后在com.unionpay.producer包下建立KafkaProducerDemo.java文件（最好不要命名为Producer或者KafkaProducer，这样会与引入的jar包中原有的类重名）：

　　KafkaProducerDemo.java

package com.unionpay.producer; 2  3 import java.util.Properties; 4  5 import org.apache.kafka.clients.producer.KafkaProducer; 6 import org.apache.kafka.clients.producer.ProducerRecord; 7  8 public class KafkaProducerDemo { 9 10     Properties properties;11 12     public KafkaProducerDemo(Properties properties) {13         super();14         this.properties = properties;15     }16 17     public Properties getProperties() {18         return properties;19     }20 21     public void setProperties(Properties properties) {22         this.properties = properties;23     }24 25     public void sendMessage(String msg) {26 27         KafkaProducer<String, String> producer = new KafkaProducer<String, String>(properties);28 29         ProducerRecord<String, String> record = new ProducerRecord<String, String>(properties.getProperty("topic"),30                 msg);31         producer.send(record);32 33         producer.close();34 35     }36 37 }

　　Properties属性主要是为了配置KafkaProducer类，具体信息通过Spring注入，这样可以显得更加高大上和灵活。后面的配置文件中我们可以看得到具体的配置信息。

　　同样，在com.unionpay.consumer包下面新建类KafkaConsumerDemo.java:

package com.unionpay.consumer; 2  3 import java.util.Arrays; 4 import java.util.Properties; 5  6 import org.apache.kafka.clients.consumer.ConsumerRecord; 7 import org.apache.kafka.clients.consumer.ConsumerRecords; 8 import org.apache.kafka.clients.consumer.KafkaConsumer; 9 10 public class KafkaConsumerDemo {11 12     private Properties props;13     14     public KafkaConsumerDemo(Properties props) {15         super();16         this.props = props;17     }18 19     public Properties getProps() {20         return props;21     }22 23     public void setProps(Properties props) {24         this.props = props;25     }26 27     public String receive(){28         29         KafkaConsumer<String,String> consumer = new KafkaConsumer<String,String>(props);30         consumer.subscribe(Arrays.asList(props.getProperty("topic")));31         32         String msg = "";33         while(true){34             ConsumerRecords<String,String> consumerRecords = consumer.poll(100);35             for(ConsumerRecord<String, String> consumerRecord:consumerRecords){36                 msg += consumerRecord.value();37             }38             consumer.close();39             return msg;40         }41     }42     43 }

　　也是基于同样的原因，KafkaConsumer的配置信息properties也是通过Spring配置文件注入。

　　当Producer和Consumer编写完成后，就可以编写kafka-beans.xml文件啦：

　　kafka-beans.xml

<?xml version="1.0" encoding="UTF-8"?> 2 <beans xmlns="http://www.springframework.org/schema/beans" 3     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" 4     xmlns:mvc="http://www.springframework.org/schema/mvc" 5     xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd 6         http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd 7         http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd"> 8  9 10     <context:component-scan base-package="com.unionpay.producer"></context:component-scan>11     <context:component-scan base-package="com.unionpay.consumer"></context:component-scan>12 13 14     <bean id="kafkaProducerDemo" class="com.unionpay.producer.KafkaProducerDemo">15         <property name="properties">16             <props>17                 <prop key="topic">my-replicated-topic</prop>18                 <prop key="bootstrap.servers">127.0.0.1:9092</prop>19                 <prop key="acks">all</prop>20                 <prop key="key.serializer">org.apache.kafka.common.serialization.StringSerializer21                 </prop>22                 <prop key="value.serializer">org.apache.kafka.common.serialization.StringSerializer23                 </prop>24                 <prop key="buffer.memory">33554432</prop>25             </props>26 27         </property>28     </bean>29 30     <bean id="kafkaConsumerDemo" class="com.unionpay.consumer.KafkaConsumerDemo">31         <property name="props">32             <props>33                 <prop key="topic">my-replicated-topic</prop>34                 <prop key="bootstrap.servers">127.0.0.1:9092</prop>35                 <prop key="group.id">group1</prop>36                 <prop key="enable.auto.commit">true</prop>37                 <prop key="auto.commit.interval.ms">1000</prop>38                 <prop key="session.timeout.ms">30000</prop>39                 <prop key="key.deserializer">org.apache.kafka.common.serialization.StringDeserializer40                 </prop>41                 <prop key="value.deserializer">org.apache.kafka.common.serialization.StringDeserializer42                 </prop>43             </props>44 45         </property>46     </bean>47 </beans>

　　kafka的主要配置一共有三种：broker、producer和consumer，对于客户端来说就是后两种啦。而后两种的配置项从官方文件可以知道，每个都至少有30多种配置内容。通过上面这种注入配置方式的话，在<props><props>中随便添加配置内容，是不是很灵活呢^_^

　　下面在com.unionpay.controller包下编写Controller类，控制业务逻辑：

　　KafkaController.java

package com.unionpay.controller;import java.text.SimpleDateFormat;import java.util.Date;import javax.annotation.Resource;import org.springframework.stereotype.Controller;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import org.springframework.web.bind.annotation.RequestParam;import org.springframework.web.servlet.ModelAndView;import com.unionpay.consumer.KafkaConsumerDemo;import com.unionpay.producer.KafkaProducerDemo;@Controllerpublic class KafkaController {    @Resource(name = "kafkaProducerDemo")    KafkaProducerDemo producer;    @Resource(name = "kafkaConsumerDemo")    KafkaConsumerDemo consumer;    @RequestMapping(value = "/welcome")    public ModelAndView welcome() {        System.out.println("--------welcome--------");        ModelAndView mv = new ModelAndView();        mv.setViewName("welcome");        return mv;    }    @RequestMapping(value = "/sendmessage", method = RequestMethod.GET)    public ModelAndView sendMessage() {        System.out.println("--------sendmessage--------");        Date date = new Date();        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");        String now = sdf.format(date);        ModelAndView mv = new ModelAndView();        mv.addObject("time", now);        mv.setViewName("kafka_send");        return mv;    }    @RequestMapping(value = "/onsend", method = RequestMethod.POST)    public ModelAndView onsend(@RequestParam("message") String msg) {        System.out.println("--------onsend--------");        producer.sendMessage(msg);        ModelAndView mv = new ModelAndView();        mv.setViewName("welcome");        return mv;    }    @RequestMapping(value = "/receive")    public ModelAndView receive() {        System.out.println("--------receive--------");                String msg = consumer.receive();                ModelAndView mv = new ModelAndView();        mv.addObject("msg", msg);        mv.setViewName("kafka_receive");        return mv;    }}

　　到目前为止，我们的MVC中已经完成了两个啦（M and C），下面编写最后的三个JSP文件。从解析配置文件（spring-mvc-dispatcher.xml）来看，我们的JSP页面应该建立在/WEB-INF/views/目录下，所以我们首先在/WEB-INF/目录下建立views文件夹。然后在该文件夹下面建立三个jsp文件：

　　welcome.jsp

<%@ page language="java" contentType="text/html; charset=UTF-8" 2     pageEncoding="UTF-8"%> 3 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 4 <html> 5 <head> 6 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 7 <title>welcome</title> 8 </head> 9 <body>10     <h1>Welcome</h1>11     <h2><a href="sendmessage">Send a Message</a></h2>12     <h2><a href="receive">Get a Message</a></h2>13 </body>14 </html>

　　kafka_send.jsp

<%@ page language="java" contentType="text/html; charset=UTF-8" 2     pageEncoding="UTF-8"%> 3 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 4 <html> 5 <head> 6 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 7 <title>kafka_send</title> 8 </head> 9 <body>10     <h1>Send a Message</h1>11     <form action="onsend" method="post">12     MessageText:<textarea name="message">${time}</textarea>13     <br>14     <input type="submit" value="Submit">15     </form>16     17     <h2><a href="welcome">RETURN HOME</a></h2>18 19 </body>20 </html>

　　kafka-receive.jsp

<%@ page language="java" contentType="text/html; charset=UTF-8" 2     pageEncoding="UTF-8"%> 3 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 4 <html> 5 <head> 6 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 7 <title>kafka_receive</title> 8 </head> 9 <body>10 11     <h1>Kafka_Reveive!!!</h1>12     <h2>Receive Message : ${msg}</h2>13     <h2><a href="welcome">RETURN HOME</a></h2>14 </body>15 </html>

　　啊，终于大功告成啦。下面就要品尝我们的劳动果实啦。将项目部署在容器里，然后首先启动zookeeper集群服务器，然后启动kafka集群服务器：

//启动zookeeper集群服务器cd ~/DevelopEnvironment/zookeeper-3.4.9-kafka/bin./zkServer.sh start//启动kafka集群服务器cd ~/DevelopEnvironment/kafka_2.10-0.10.2.0/bin./kafka-server-start.sh ../config/server.properties ./kafka-server-start.sh ../config/server-1.properties ./kafka-server-start.sh ../config/server-2.properties

　　然后通过Eclipse启动容器：

　　

　　从上面终端打印出来的信息可以知道，部署成功啦。下面就要轮回到本文开头啦，在网页地址栏中输入：http://127.0.0.1:8080/kafkaSpringMVC/welcome 进入欢迎界面，然后按照开始描述的操作进行操作，看看能否成功发送和接受消息呢？

　　怎么样，你成功了吗？反正我是成功了，也希望你也成功啦。如果出现什么错误的话也千万别着急，去根据报错信息找找原因，因为你也马上就要成功啦。

　　源码下载：kafkaSpringMVC.zip

参考文献

• kafkaProducer官方参考文档
• kafkaConsumer官方参考文档