Hue 可视化的Hadoop架构工具

转载地址：http://www.datawhy.net/index.php?m=blog&f=view&id=42

1）简介

大数据可视化工具

2）编译

下载地址
编译帮助文档

##解压
$ tar -zxvf hue-3.7.0-cdh5.3.6.tar.gz -C /opt/cdh/
$ cd /opt/cdh
$ mv hue-3.7.0-cdh5.3.6hue-3.7.0
##yum安装相关组件（联网）
$sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel
##编译
$ cd /opt/cdh/hue-3.7.0
$ make apps
##编译好的hue 
$ cd /opt/cdh/hue-3.7.0/apps

3）安装

配置文件目录：$ cd /opt/cdh/hue-3.7.0/desktop/conf  ---> hue.ini

Hue.ini位于 Hue的 desktop/conf/ 目录下

配置如下（desktop）： 

[desktop]  # Set this to a random string, the longer the better.  # This is used for secure hashing in the session store.  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn


启动：
$ cd /opt/cdh/hue-3.7.0/
$ build/env/bin/supervisor
访问：
http://bigdata.caiwei.me:8888/
4）集成hdfs
配置文档HDFS
##hdfs-site.xml
dfs.webhdfs.enabledname>
truevalue>
property>
##core-site.xml
hadoop.proxyuser.hue.hostsname>
*value>
property>
hadoop.proxyuser.hue.groupsname>
*value>
property>
##集群之外访问：httpfs-site.xml
httpfs.proxyuser.hue.hostsname>
*value>
property>
httpfs.proxyuser.hue.groupsname>
*value>
property>
配置文档hue.ini
fs_defaultfs=hdfs://bigdata.caiwei.me:8020
# Default port is 14000 for HttpFs.
webhdfs_url=http://bigdata.caiwei.me:50070/webhdfs/v1
# Directory of the Hadoop configuration
hadoop_conf_dir=/opt/cdh/hadoop-2.5.0/etc/hadoop
# Directory of the Hadoop home
hadoop_hdfs_home=/opt/cdh/hadoop-2.5.0/
# Directory of the Hadoop bin
hadoop_bin=/opt/cdh/hadoop-2.5.0/bin

6）集成yarn
需要启动 hiveserver2$ nohup hiveserver2 &
配置文档hue.ini
# Enter the host on which you are running the ResourceManager
resourcemanager_host=bigdata.caiwei.me.com
# The port where the ResourceManager IPC listens on
resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# URL of the ResourceManager API
resourcemanager_api_url=http://bigdata.caiwei.me:8088
# URL of the ProxyServer API
proxy_api_url=http://bigdata.caiwei.me:8088
# URL of the HistoryServer API
history_server_api_url=http://bigdata.caiwei.me:19888

7）集成hive
配置文档hue.ini
注意：本地metastore不用配置。不是本地需要配置：https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-Local/EmbeddedMetastoreDatabase(Derby)
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=bigdata.caiwei.me
# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/opt/cdh/hive-0.13.1/conf
# Timeout in seconds for thrift calls to Hive service
server_conn_timeout=120

5）集成RDBMS
  [[databases]]
    # mysql, oracle, or postgresql configuration.
    [[[mysql]]]
      # Name to show in the UI.
      nice_name="My SQL DB"

      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      # this is 'xe' by default.
      name=test

      # Database backend to use. This can be:
      # 1. mysql
      # 2. postgresql
      # 3. oracle
      engine=mysql

      # IP or hostname of the database to connect to.
      host=bigdata.caiwei.me

      # Port the database server is listening to. Defaults are:
      # 1. MySQL: 3306
      # 2. PostgreSQL: 5432
      # 3. Oracle Express Edition: 1521
      port=3306

      # Username to authenticate with when connecting to the database.
      user=root

      # Password matching the username to authenticate with when
      # connecting to the database.
      password=123456