pgloader

来源：互联网发布：java输出三角形编辑：程序博客网时间：2024/06/02 02:10

使用pgloader加载数据到postgresql中

替换conf文件的日期
sed -i “s/[0-9]{4}-[0-9]{2}-[0-9]{2}/${YESTERDAY}/” /export/loganalyse/tuan/xuelei/vocation/loganaly/pgloader-stat-trace-pgdata3.conf

执行加载数据
python /usr/bin/pgloader -c /export/loganalyse/pgdata3.conf

相关配置文件
[jiao@l-jiao.s.cn6 ～]$ cat pgloader-stat-trace-pgdata3.conf

[pgsql]host = l-data3.vc.cn5port = 5432base = data_sasuser = pgdbapass = XXXXXXXXXXXXXXXXXXXXXXlog_file            = /export/loganalyse/tuan/xuelei/vocation/loganaly/zhangjun-shell/pgloader-conf/log/pgdata3-trace-2015-12-01.loglog_min_messages    = ERRORclient_min_messages = WARNINGlc_messages         = Cclient_encoding = 'utf-8'　#input_encoding = 'utf-8'copy_every      = 20000commit_every    = 20000#copy_delimiter  = %null         = "NULL"empty_string = ""max_parallel_sections = 4[pgloader]table     = wl_data.stat_traceformat    = textfilename  = /export/loganalyse/tuan/data/trace.2015-12-01.logfield_sep = \tcolumns   = report_date,requesttime,source,mode,uid,pids,env_type,query,pt,act,ts,platform,vid,ntitle,ntitlelen,nimg,nimglen,userid,gidreject_log =   /export/loganalyse/tuan/pgloader-conf/log/trace-reject-2015-12-01.logreject_data =  /export/loganalyse/tuanpgloader-conf/log/trace-statistics-reject-2015-12-01.dataquotechar = ^Aescapechar = ^A

0 0