Sqoop HDFS导出到MySQL
来源:互联网 发布:114源码 编辑:程序博客网 时间:2024/05/22 11:43
在MySQL中创建表
CREATE TABLE `tree1` ( `id` int(11) NOT NULL AUTO_INCREMENT, `treeNumber` varchar(100) NOT NULL, `productinformationId` int(11) NOT NULL, PRIMARY KEY (`id`)) DEFAULT CHARSET=utf8;
执行命令
sqoop export --connect jdbc:mysql://node1:3306/phx \--username root \--table tree1 -m 1 \--export-dir /sqoop/tree2
结果
[root@node1 conf]# sqoop export --connect jdbc:mysql://node1:3306/phx \--username root \--table tree1 -m 1 \--export-dir /sqoop/tree2Warning: /csh/link/sqoop/../hcatalog does not exist! HCatalog jobs will fail.Please set $HCAT_HOME to the root of your HCatalog installation.Warning: /csh/link/sqoop/../accumulo does not exist! Accumulo imports will fail.Please set $ACCUMULO_HOME to the root of your Accumulo installation.16/04/19 01:10:23 INFO sqoop.Sqoop: Running Sqoop version: 1.4.616/04/19 01:10:23 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.16/04/19 01:10:23 INFO tool.CodeGenTool: Beginning code generation16/04/19 01:10:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tree1` AS t LIMIT 116/04/19 01:10:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tree1` AS t LIMIT 116/04/19 01:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /csh/link/hadoopNote: /tmp/sqoop-root/compile/8efdea8893437a1359c77de6d1bd2395/tree1.java uses or overrides a deprecated API.Note: Recompile with -Xlint:deprecation for details.16/04/19 01:10:28 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/8efdea8893437a1359c77de6d1bd2395/tree1.jar16/04/19 01:10:29 INFO mapreduce.ExportJobBase: Beginning export of tree1SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/csh/software/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/csh/software/hbase-1.1.4/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]16/04/19 01:10:30 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar16/04/19 01:10:38 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative16/04/19 01:10:38 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative16/04/19 01:10:38 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps16/04/19 01:10:38 INFO client.RMProxy: Connecting to ResourceManager at node1/192.168.161.11:803216/04/19 01:10:48 INFO input.FileInputFormat: Total input paths to process : 116/04/19 01:10:48 INFO input.FileInputFormat: Total input paths to process : 116/04/19 01:10:49 INFO mapreduce.JobSubmitter: number of splits:116/04/19 01:10:49 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative16/04/19 01:10:49 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1461026006811_000316/04/19 01:10:51 INFO impl.YarnClientImpl: Submitted application application_1461026006811_000316/04/19 01:10:51 INFO mapreduce.Job: The url to track the job: http://node1:8088/proxy/application_1461026006811_0003/16/04/19 01:10:51 INFO mapreduce.Job: Running job: job_1461026006811_000316/04/19 01:11:33 INFO mapreduce.Job: Job job_1461026006811_0003 running in uber mode : false16/04/19 01:11:33 INFO mapreduce.Job: map 0% reduce 0%16/04/19 01:12:24 INFO mapreduce.Job: map 100% reduce 0%16/04/19 01:12:27 INFO mapreduce.Job: Job job_1461026006811_0003 completed successfully16/04/19 01:12:27 INFO mapreduce.Job: Counters: 30 File System Counters FILE: Number of bytes read=0 FILE: Number of bytes written=137080 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=251 HDFS: Number of bytes written=0 HDFS: Number of read operations=4 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Job Counters Launched map tasks=1 Rack-local map tasks=1 Total time spent by all maps in occupied slots (ms)=45106 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=45106 Total vcore-milliseconds taken by all map tasks=45106 Total megabyte-milliseconds taken by all map tasks=46188544 Map-Reduce Framework Map input records=7 Map output records=7 Input split bytes=122 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=258 CPU time spent (ms)=1680 Physical memory (bytes) snapshot=86806528 Virtual memory (bytes) snapshot=2086785024 Total committed heap usage (bytes)=17235968 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=016/04/19 01:12:27 INFO mapreduce.ExportJobBase: Transferred 251 bytes in 109.7035 seconds (2.288 bytes/sec)16/04/19 01:12:27 INFO mapreduce.ExportJobBase: Exported 7 records.
在MySQL中查看tree1表中数据
个人博客原文:
Sqoop HDFS导出到MySQL
0 0
- Sqoop HDFS导出到MySQL
- SQOOP从HDFS导出数据到MySQL
- 1.5 使用Sqoop从HDFS导出数据到MySQL
- Sqoop MySQL 导入到HDFS
- sqoop操作之HDFS导出到ORACLE
- 利用sqoop把Mysql中的表数据导出到HDFS下的文本文件里
- Sqoop对Mysql数据库导入导出到hdfs(基于Hadoop 2.6.0 )
- Sqoop 数据从HDFS导入到mysql
- SQOOP从MySQL导入数据到HDFS
- 【hadoop Sqoop】Sqoop从mysql导数据到hdfs
- 3.sqoop 导入到HDFS,导出到RDBMS
- 利用Sqoop从HDFS导出数据到DB
- Sqoop 从hdfs中把数据导出到oracle
- [Sqoop]将Hive数据表导出到Mysql
- sqoop 导出 hive分区表 数据到 mysql
- Sqoop导出数据到MYSQL问题
- sqoop从hbase导出数据到mysql
- sqoop导出hive表到mysql中
- 上传文件jQuery 的ajaxFileUpload 方法上传文件
- 多位数乘单位数
- iOS 微信支付
- 搭建webrtc服务器(ubuntu14.04)
- Laravel 动态读取mysql表数据(包含列、值
- Sqoop HDFS导出到MySQL
- shell编程——if语句 if -z -n -f -eq -ne -lt
- 红糖水白糖水程序
- 颓の第17周
- 线程安全问题及解决办法
- 安卓开发应注意的问题
- Your content must have a ListView whose id attribute is 'android.R.id.list'
- 练习多位数乘一位数
- VC++动态链接库(DLL)编程深入浅出