hue搭建(hadoop分布式环境）

一、分布式集群成员及所有框架

flume-1.5.0-cdh5.3.6-bin 

hbase-0.98.6-cdh5.3.6  

hue-3.7.0-cdh5.3.6  

sqoop-1.4.5-cdh5.3.6（ mysql-connector-java-5.1.31.jar）
hadoop-2.5.0-cdh5.3.6   

hive-0.13.1-cdh5.3.6  （ mysql-connector-java-5.1.31.jar）

jdk1.7.0_67         

zookeeper-3.4.5-cdh5.3.6

二、集群规划

    rainbow.com.cn01                                   rainbow.com.cn02                             rainbow.com.cn03
zookeeper        zookeeper zookeeper
namenode                                                namenode
datanode        datanode datanode
                                                               resourcemanager                                resourcemanager
nodemanager nodemanager nodemanager
                                                                                                                        historysever
Hmaster (Hmaster)                                          (Master)
hregionserver         hregionserver hregionserver

三、hue搭建

   1、前提以上框架已搭建ok

   2、解压安装hue-3.7.0-cdh5.3.6  

   3、修改配置文件hue.ini

           (1)hue.ini----[desktop]

[desktop]  # Set this to a random string, the longer the better.  # This is used for secure hashing in the session store.  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o             # Webserver listens on this address and port  http_host=rainbow.com.cn01                                                      http_port=8888  # Time zone name  time_zone=Asia/Shanghai

          (2)与hdfs集成 

          hue.ini----[hdfs clusters]-----ns1

  [[hdfs_clusters]]    # HA support by using HttpFs    [[[default]]]      # Enter the filesystem uri      fs_defaultfs=hdfs://ns1:8020      # NameNode logical name.      ## logical_name=      # Use WebHdfs/HttpFs as the communication mechanism.      # Domain should be the NameNode or HttpFs host.      # Default port is 14000 for HttpFs.       webhdfs_url=http://ns1:50070/webhdfs/v1      # Change this if your HDFS cluster is Kerberos-secured      ## security_enabled=false      # Default umask for file and directory creation, specified in an octal value.      ## umask=022      # Directory of the Hadoop configuration  hadoop_conf_dir=/opt/modules/hadoop-2.5.0-cdh5.3.6/etc/hadoop  hadoop_hdfs_home=/opt/modules/hadoop-2.5.0-cdh5.3.6  hadoop_bin=/opt/modules/hadoop-2.5.0-cdh5.3.6/bin

               修改 hadoop---etc/hadoop/core-site.xml

<property><name>hadoop.proxyuser.hue.hosts</name><value>*</value></property><property><name>hadoop.proxyuser.hue.groups</name> <value>*</value></property>

               修改 hadoop---etc/hadoop/hdfs-site.xml

<property><name>dfs.webhdfs.enabled</name><value>true</value></property>

将hadoop/etc/hadoop下面的hdfs-site.xml、core-site.xml yarn-site.xml复制到hue/conf目录下

$ cp hdfs-site.xml core-site.xml yarn-site.xml /opt/modules/hue-3.7.0-cdh5.3.6/desktop/conf/

每次配置完一项都重新启动hue与相应的框架并观察http://rainbow.com.cn03:8888页面。

          (3)与yarn集成 --hue.ini

          [yarn_clusters]

 [[yarn_clusters]]    [[[default]]]      # Enter the host on which you are running the ResourceManager      resourcemanager_host=rainbow.com.cn03      # The port where the ResourceManager IPC listens on       resourcemanager_port=8032      # Whether to submit jobs to this cluster      submit_to=True      # Resource Manager logical name (required for HA)      ## logical_name=      # Change this if your YARN cluster is Kerberos-secured      ## security_enabled=false      # URL of the ResourceManager API      resourcemanager_api_url=http://rainbow.com.cn03:8088      # URL of the ProxyServer API      proxy_api_url=http://rainbow.com.cn03:8088      # URL of the HistoryServer API      history_server_api_url=http://rainbow.com.cn03:19888      # In secure mode (HTTPS), if SSL certificates from Resource Manager's      # Rest Server have to be verified against certificate authority      ## ssl_cert_ca_verify=False    # HA support by specifying multiple clusters    # e.g.     [[[ha]]]      # Resource Manager logical name (required for HA)   ！！！注意热备       logical_name=rmcluster  # Configuration for MapReduce (MR1)

             （4）与hive集成

                   1、hive-site.xml

                   远程登录配置

<property>  <name>hive.server2.long.polling.timeout</name>  <value>5000</value>  <description>Time in milliseconds that HiveServer2 will wait, before responding to asynchronous calls that use long polling</description></property><property>  <name>hive.server2.thrift.port</name>  <value>10000</value>  <description>Port number of HiveServer2 Thrift interface.  Can be overridden by setting $HIVE_SERVER2_THRIFT_PORT</description></property><property>  <name>hive.server2.thrift.bind.host</name>  <value>rainbow.com.cn01</value>  <description>Bind host on which to run the HiveServer2 Thrift interface.  Can be overridden by setting $HIVE_SERVER2_THRIFT_BIND_HOST</description></property>

                  2、hue.ini

[beeswax]  # Host where HiveServer2 is running.  # If Kerberos security is enabled, use fully-qualified domain name (FQDN).   hive_server_host=rainbow.com.cn01  # Port where HiveServer2 Thrift server runs on.   hive_server_port=10000  # Hive configuration directory, where hive-site.xml is located   hive_conf_dir=/opt/modules/hive-0.13.1-cdh5.3.6/conf/

         （5）与RDBMS集成

                修改hue.ini

<pre name="code" class="plain">[librdbms]  # The RDBMS app can have any number of databases configured in the databases  # section. A database is known by its section name  # (IE sqlite, mysql, psql, and oracle in the list below).  [[databases]]    # sqlite configuration.    ## [[[sqlite]]]      # Name to show in the UI.       nice_name=SQLite      # For SQLite, name defines the path to the database.       name=/opt/modules/hue-3.7.0-cdh5.3.6/desktop/desktop.db       # Database backend to use.      engine=sqlite      # Database options to send to the server when connecting.      # https://docs.djangoproject.com/en/1.4/ref/databases/      ## options={}    # mysql, oracle, or postgresql configuration.    [[[mysql]]]                                          [[[mysql]]]的注释要放开      # Name to show in the UI.       nice_name="My SQL DB"      # For MySQL and PostgreSQL, name is the name of the database.      # For Oracle, Name is instance of the Oracle server. For express edition      # this is 'xe' by default.      ##name=mysqldb      # Database backend to use. This can be:      # 1. mysql      # 2. postgresql      # 3. oracle       engine=mysql      # IP or hostname of the database to connect to.      ## host=localhost      # Port the database server is listening to. Defaults are:      # 1. MySQL: 3306      # 2. PostgreSQL: 5432      # 3. Oracle Express Edition: 1521       port=3306      # Username to authenticate with when connecting to the database.       user=root      # Password matching the username to authenticate with when      # connecting to the database.      password=root      # Database options to send to the server when connecting.      # https://docs.djangoproject.com/en/1.4/ref/databases/      ## options={}

    （6）与oozie集成

     （7）与hbase集成