log4j实战
来源:互联网 发布:人工智能湖南论坛 编辑:程序博客网 时间:2024/06/05 10:54
原文:http://www.iteye.com/topic/1112588
场景如下:
由于项目要做一个统计分析与记录的功能,为了后期分析用户的一些行为,同时记录的东西要同时输出多处,并且可配置输出想到了使用log4j,知道他是可以同时指定多个输出目的地,并且如有变更,直接修改配置文件。
关于log4j的基本概论及使用,见http://www.iteye.com/topic/378077,这个帖子讲的很细,归纳了很多东西
问题一:把登录用户每次访问的ip地址 时间等记录下来,存放到日志文件里,再存一份到数据库
第一步:记录用户访问,写一个filter如下,里面MDC是一个类似map的东西,只不过做了线程方式的封装使用,每个线程会有自己的一份map,可以看这个http://www.iteye.com/topic/1112590
- public class UserLogFilter implements Filter {
- private Logger logger;
- @Override
- public void destroy() {
- // TODO Auto-generated method stub
- }
- @Override
- public void doFilter(ServletRequest request, ServletResponse response,
- FilterChain chain) throws IOException, ServletException {
- HttpServletRequest httpRequest = (HttpServletRequest) request;
- Object user = httpRequest.getSession().getAttribute(
- Constants.User.LOGIN_USER);
- if (user != null) {
- Integer usr_id = (Integer) ReflectionUtils.invokeGetterMethod(user, "usrId");
- String log_ip = request.getLocalAddr();
- SimpleDateFormat format=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
- Date now=new Date(System.currentTimeMillis());
- MDC.put("usr_id", usr_id);
- MDC.put("log_title", "网站访问记录");
- MDC.put("log_type", "记录");
- MDC.put("log_title", "网站访问记录");
- MDC.put("log_datetime", format.format(now));
- MDC.put("log_ip", log_ip);
- logger.info(MDC.getContext());
- }
- chain.doFilter(request, response);
- }
- @Override
- public void init(FilterConfig filterConfig) throws ServletException {
- logger=Logger.getLogger(UserLogFilter.class);
- }
- }
第二部:配置log4j.properties,输出到文件与数据库
引用
#不懂的话,参考给出的第一个连接地址一样,j2EE项目详细控制
log4j.logger.com.bhaman.yiyaosou.util.web=INFO,,project-util,project-util-db
#project-util-web file appender
log4j.appender.project-util=org.apache.log4j.DailyRollingFileAppender
log4j.appender.project-util.file=${user.home}/logs/project-util.log
log4j.appender.project-util.layout=org.apache.log4j.PatternLayout
log4j.appender.project-util.threshold=INFO
log4j.appender.project-util.layout.conversionPattern=%d [%X{usr_id}/%X{log_ip}/%X{req.id} - %X{entranceMethod} - %X{req.requestURIWithQueryString}] %-5p %c - %m%n
log4j.appender.project-util-db=com.log4j.service.DBAppender
log4j.appender.project-util-db.bufferSize=16
log4j.appender.project-util-db.threshold=INFO
#此处对应filter里面的MDC里面的键值对,你懂的
log4j.appender.project-util-db.sql=insert into user_log (usr_id,log_title,log_category,log_type,log_datetime,log_ip) VALUES ('%X{usr_id}','%X{log_title}','%X{log_type}','%X{log_title}','%X{log_datetime}','%X{log_ip}')
第三部:由于log4j里面给出的输出JDBCappender,是有问题的,首先面对业务需求,他是用JDBC,性能上问题很大,特别是现在这个应用。log4j支持自己写appender
直接看JDBCAppender的源码
如下里面提到有做了缓冲,两个ArrayList,一个来存东西,一个来控制清零后默认的缓冲大小不变,具体看源码,配置文件可以直接配置初始化大小
- <p>Each append call adds to an <code>ArrayList</code> buffer. When
- the buffer is filled each log event is placed in a sql statement
- (configurable) and executed.
- <b>BufferSize</b>, <b>db URL</b>, <b>User</b>, & <b>Password</b> are
- configurable options in the standard log4j ways.
- <p>The <code>setSql(String sql)</code> sets the SQL statement to be
- used for logging -- this statement is sent to a
- <code>PatternLayout</code> (either created automaticly by the
- appender or added by the user). Therefore by default all the
- conversion patterns in <code>PatternLayout</code> can be used
- inside of the statement. (see the test cases for examples)
- protected int bufferSize = 1;
- /**
- * ArrayList holding the buffer of Logging Events.
- */
- protected ArrayList buffer;
- /**
- * Helper object for clearing out the buffer
- */
- protected ArrayList removes;
里面执行的sql,连接url 什么的就是配置文件里面配置,,会在调用的时候set进来
- protected String databaseURL = "jdbc:odbc:myDB";
- /**
- * User to connect as for default connection handling
- */
- protected String databaseUser = "me";
- /**
- * User to use for default connection handling
- */
- protected String databasePassword = "mypassword";
那么我自己写的appender继承它就好了,直接使用它的缓冲及sql执行,那么唯一要变的就是连接了。
我要从连接池里面取出,怎么做呢?看下面源码中注释-->>
引用
<li>Override <code>getConnection()</code> to pass any connection
you want. Typically this is used to enable application wide
connection pooling.
<li>Override <code>closeConnection(Connection con)</code> -- if
you override getConnection make sure to implement
<code>closeConnection</code> to handle the connection you
generated. Typically this would return the connection to the
pool it came from.
<li>Override <code>getLogStatement(LoggingEvent event)</code> to
produce specialized or dynamic statements. The default uses the
sql option value.
我不需要覆写getLogStatement,我想改变的只是连接的获取方式罢了,如上分析后,直接写我的appender,里面用到了高效的BoneCP连接池,在spring里面本来是有配置这个的,但log4j是独立于spring的,是无法获取到,只能自己再来一份
- public class DBAppender extends org.apache.log4j.jdbc.JDBCAppender {
- private BoneCP connectionPool = null;
- private Connection connection = null;
- private static Logger logger=Logger.getLogger(DBAppender.class);
- public DBAppender() {
- // 设置连接池配置信息
- BoneCPConfig config = new BoneCPConfig();
- PropetiesUtil p;
- try {
- Properties P = new Properties();
- P.load(DBAppender.class.getClassLoader().getResourceAsStream("application.properties"));
- // 数据库的JDBC URL
- config.setJdbcUrl(P.getProperty("jdbc.url"));
- // 数据库用户名
- config.setUsername(P.getProperty("jdbc.username"));
- // 数据库用户密码
- config.setPassword(P.getProperty("jdbc.password"));
- // 数据库连接池的最小连接数
- config.setMinConnectionsPerPartition(5);
- // 数据库连接池的最大连接数
- config.setMaxConnectionsPerPartition(10);
- config.setPartitionCount(1);
- // 设置数据库连接池
- connectionPool = new BoneCP(config);
- } catch (SQLException e) {
- // TODO Auto-generated catch block
- logger.error("连接池配置加载异常",e);
- } catch (IOException e) {
- // TODO Auto-generated catch block
- logger.error("加载配置文件IO异常",e);
- }
- // fetch a connection
- }
- @Override
- protected Connection getConnection() throws SQLException {
- if(connection==null||connection.isClosed()){
- connection = connectionPool.getConnection();
- }
- return connection;
- }
- @Override
- protected void closeConnection(Connection con) {
- // TODO Auto-generated method stub
- try {
- connection.close();
- connection=null;
- } catch (SQLException e) {
- // TODO Auto-generated catch block
- logger.error("连接没正常关闭",e);
- }
- }
- }
第四步:一切OK,然后就测试
日志文件:
引用
2011-07-21 11:07:01,125 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:01, log_title=网站访问记录, usr_id=282}
2011-07-21 11:07:03,640 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:03, log_title=网站访问记录, usr_id=282}
2011-07-21 11:07:04,796 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:04, log_title=网站访问记录, usr_id=282}
2011-07-21 11:07:08,906 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:08, log_title=网站访问记录, usr_id=282}
2011-07-21 11:07:09,281 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:09, log_title=网站访问记录, usr_id=282}
2011-07-21 11:07:14,531 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:07:14, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:10,984 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:10, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:11,796 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:11, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:22,078 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:22, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:22,875 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:22, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:28,562 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:28, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:30,250 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:30, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:31,390 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:31, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:32,750 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:32, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:33,781 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:33, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:36,156 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:36, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:41,578 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:41, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:43,156 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:43, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:44,968 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:44, log_title=网站访问记录, usr_id=282}
2011-07-21 11:11:48,765 [282/127.0.0.1/ - - ] INFO com.bhaman.yiyaosou.util.web.UserLogFilter - {log_type=记录, log_ip=127.0.0.1, log_datetime=2011-07-21 11:11:48, log_title=网站访问记录, usr_id=282}
数据库:
第五步:
比如现在系统要记录登录用户与非登录用户各个时间段得访问与访问连接数,最后跟购买行为统计挂上钩,进行数据挖掘,但现在又改变需求,还要分析登录用户里面的各种级别的用户等
系统里面需要统计分析的东西很多的话,怎么解决?难道配置各种filter然后输出两地?本来太多的fiter就会带来性能问题,多了岂不是?
大范围记录一次到日志文件,然后写job从任务文件里面分析出数据然后写入数据库于日志文件。
阅读全文
0 0
- Log4j实战
- Log4J实战
- log4j实战
- log4j详解与实战
- log4j详解与实战
- log4j - 详解与实战
- log4j 应用 实战篇
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j 详解与实战
- log4j详解与实战
- log4j详解与实战
- log4j详解与实战
- python3 怎么把竖行输出转为横行输出 (加入逗号的改变)
- Nginx配置文件参数
- pat 乙级 1037. 在霍格沃茨找零钱(20)
- Curl
- 线性回归之代价函数除2m
- log4j实战
- C语言--链表与CMap
- 线性代数的本质(Essense Of Linear Algebra)[1]
- HDU 1260 Tickets
- shell 脚本echo和printf学习
- react native学习笔记7——组件生命周期
- 【Unity与23种设计模式】代理模式(Proxy)
- A
- protect继承