hadoop 集群 远程访问 mysql(通过sqoop从远程数据库服务器向hdfs迁移数据) 屡次失败的原因

来源:互联网 发布:js自定义属性 编辑:程序博客网 时间:2024/06/04 19:01

首先,我介绍一下我搭建的环境(win8系统上的1个VM的工作站,跑了6个centos虚拟机),6个节点,3个datanode,1个 sourcemanager 2个namenode。

设置了虚拟网络ip (非localhost),以下问题在老鸟看来是在平常不过了,但我还是只小鸟,这个问题的解决是我又上了一个台阶啊。同时,我也希望借此博文帮助到大家,让大家少走弯路。聪明的你也可以在我后面揭晓解决方案之前琢磨一下我可能遇到的问题。这样,我想,或许对你更加有帮助啦。

之前在网上收了很多资料,其中有介绍有是jar包导错的,或者原因是访问服务器没有指定特定数据库是带来的问题,还有些是由于权限不足带来的问题,但这些都没有解决我的问题,清一色的会抛出如下exception!!!发火

15/04/24 08:49:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
15/04/24 08:50:36 ERROR manager.CatalogQueryManager: Failed to list databases
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 0 ms ago.
........
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 1 ms ago.
       .........
Caused by: java.net.ConnectException: Connection timed out
  .........
15/04/24 08:50:36 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 0 ms ago.
java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 0 ms ago.
........
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 0 ms ago.
     ......
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


Last packet sent to the server was 1 ms ago.
    .......
Caused by: java.net.ConnectException: Connection timed out

      .........

后来,找度娘实在解决不了了,只有去google搜了以下问题,其中,搜到了一条原因,他是这样的:

  1. Something in between Java and DB is blocking connections, e.g. a firewall or proxy. 
然后,我恍然大悟,我的天哪!!!防火墙!然后,我就想直接把windows防火墙干掉,没成功,好像被360保护住了。然后,我打开了防火墙上供mysql远程访问的3306端口,成功!!!
谢谢能靠这儿的各位朋友。以后,我还会遇到的问题及解决方案与大家分享的!

0 1