SQOOP之——安装部署笔记
来源:互联网 发布:大学生网络攻防大赛 编辑:程序博客网 时间:2024/05/16 04:07
转载请注明出处:http://blog.csdn.net/l1028386804/article/details/51594008
sqoop是一个把关系型数据库数据抽向hadoop的工具。同时,也支持将hive、pig等查询的结果导入关系型数据库中存储。由于,本人部署的hadoop版本是2.2.0,所以sqoop的版本是:sqoop-1.99.3-bin-hadoop200
1、下载sqoop
wget https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gz2、解压sqoop安装文件
tar -zxvf sqoop-1.99.3-bin-hadoop200.tar.gz3、配置sqoop环境变量
export SQOOP_HOME=/home/liuyazhuang/sqoop/sqoopexport PATH = $SQOOP_HOME/bin:$PATH修改sqoop——server关于hadoop共享jar的引用
vi server/conf/catalina.properties找到common.loader行,把/usr/lib/hadoop/lib/*.jar改成你的hadoop jar 包目录
例如:/home/hadoop/hadoop/share/hadoop/yarn/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/yarn/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/*.jar
vi server/conf/sqoop.properties找到:mapreduce.configuration.directory行,修改值为你的hadoop配置文件目录
如:/home/liuyazhuang/hadoop/hadoop/etc/hadoop
并且替换@LOGDIR@ 和@BASEDIR@ :
0,$ s/@LOGDIR@/logs/g
0,$ s/@BASEDIR@/base/g
然后找到你的数据库jdbc驱动复制到sqoop/lib目录下,如果不存在则创建.
vi /etc/profile增加以下内容:
export SQOOP_HOME=/home/hadoop/sqoopexport PATH=$PATH:$SQOOP_HOME/binexport CATALINA_BASE=$SQOOP_HOME/serverexport LOGDIR=$SQOOP_HOME/logs/
启动 [root@liuyazhuang sqoop]# ./bin/sqoop.sh server startSqoop home directory: /home/liuyazhuang/sqoop/sqoopSetting SQOOP_HTTP_PORT: 12000Setting SQOOP_ADMIN_PORT: 12001Using CATALINA_OPTS:Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001Using CATALINA_BASE: /home/liuyazhuang/sqoop/sqoop/serverUsing CATALINA_HOME: /home/liuyazhuang/sqoop/sqoop/serverUsing CATALINA_TMPDIR: /home/liuyazhuang/sqoop/sqoop/server/tempUsing JRE_HOME: /usr/local/jdk1.7.0Using CLASSPATH: /home/liuyazhuang/sqoop/sqoop/server/bin/bootstrap.jar(sqoop服务端是一个跑在tomcat上的服务程序)关闭 sqoop server :./bin/sqoop.sh server stop使用sqoop客户端:[root@liuyazhuang sqoop]# bin/sqoop.sh clientSqoop home directory: /home/liuyazhuang/sqoop/sqoopSqoop Shell: Type 'help' or '\h' for help.sqoop:000> show version --all#显示版本:show version --all显示连接器:show connector --all创建连接:create connection --cid 1client version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013server version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013Protocol version: [1]sqoop:000>
1 0
- SQOOP之——安装部署笔记
- sqoop学习笔记-sqoop安装部署
- [Sqoop]Sqoop安装与部署
- Hadoop之——SQOOP笔记
- Sqoop之——安装与使用
- Sqoop安装部署
- Sqoop笔记一之简介安装
- Sqoop介绍、搭建、安装、部署
- Sqoop-1.99.7安装部署
- Sqoop-1.4.6安装部署
- sqoop概念和安装部署
- Sqoop 简介及安装部署
- sqoop安装笔记
- sqoop之安装配置
- hadoop安装之-sqoop
- 详解Sqoop的架构和安装部署
- sqoop的安装部署与使用
- Sqoop数据交互工具——笔记
- 大陆台湾翻译之辩
- AAC音频格式分析与解码
- 打造属于你的LayoutManager
- JAVA抓取网站数据-----JSOUP
- Linux查看连接数,并发数
- SQOOP之——安装部署笔记
- C# 多线程的自动管理(线程池)
- output——标签,拖动进度条,显示进度
- C#调用C++结构体内存数据不对问题
- Android中的Service:Binder,Messenger,AIDL(2)
- SoundPool的使用(转)
- mysql对表是操作以及函数的简介
- 如何用Android Studio打多包名APK
- hdu 1492The number of divisors(约数) about Humble Numbers(因子个数)