Ambari 监控信息的提取

场景：

---

 * 通过Ambari + HDP搭建的大数据基础管理平台，目前用户不想登录Ambari而想直接将Ambari的监控相关信息整合到其他的Web项目中。  如将下面资源使用用量相关的提取出来

• 解决方式：
  
  • 本来是想着找到Ambari Dashboard功能的相关链接，然后通过模拟登录将这些视图提取出来，或者通过Ambari自带的权限管理设置用户只拥有查看Ambari Dashboard视力的权限。最终发现在不深入学习Ambari源码的情况下，上述两种方式实现起来太不容易了。
  • 因而最终采用直接读取记录了Ambari监控信息的数据库，然后通过可视化工具再实现将监控视图呈现如来。

获取Ambari监控信息的数据库

• 具体操作步骤如下所示：

1、先切换用户

[root@hdp39 ~]# su postgres

2、重新使用amabri用户进入postgresql(默认密码bigdata)

bash-4.1$ psql -U ambaricould not change directory to "/root"Password for user ambari: psql (8.4.20)Type "help" for help.ambari=> 

3、列举数据库
列举数据库，相当于mysql的show databases

ambari=> \l                                      List of databases     Name     |   Owner   | Encoding |  Collation  |    Ctype    |     Access privileges      --------------+-----------+----------+-------------+-------------+---------------------------- ambari       | postgres  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres                                                                 : postgres=CTc/postgres                                                                 : ambari=CTc/postgres postgres     | postgres  | UTF8     | en_US.UTF-8 | en_US.UTF-8 |  ranger       | rangerdba | UTF8     | en_US.UTF-8 | en_US.UTF-8 |  ranger_audit | postgres  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres                                                                 : postgres=CTc/postgres                                                                 : ranger_logger=CTc/postgres template0    | postgres  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres                                                                 : postgres=CTc/postgres template1    | postgres  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres                                                                 : postgres=CTc/postgres(6 rows)

4、列举表
列举表，相当于mysql的show tables(当前已经在ambari数据库)

ambari=> \dt                     List of relations Schema |             Name              | Type  |  Owner   --------+-------------------------------+-------+---------- ambari | adminpermission               | table | postgres ambari | adminprincipal                | table | postgres ambari | adminprincipaltype            | table | postgres ambari | adminprivilege                | table | postgres ambari | adminresource                 | table | postgres ambari | adminresourcetype             | table | postgres ambari | alert_current                 | table | postgres ambari | alert_definition              | table | postgres ambari | alert_group                   | table | postgres ambari | alert_group_target            | table | postgres ambari | alert_grouping                | table | postgres ambari | alert_history                 | table | postgres ambari | alert_notice                  | table | postgres ambari | alert_target                  | table | postgres ambari | alert_target_states           | table | postgres ambari | ambari_operation_history      | table | postgres ambari | ambari_sequences              | table | postgres ambari | artifact                      | table | postgres ambari | blueprint                     | table | postgres ambari | blueprint_configuration       | table | postgres ambari | blueprint_setting             | table | postgres ambari | cluster_version               | table | postgres ambari | clusterconfig                 | table | postgres ambari | clusterconfigmapping          | table | postgres ambari | clusterhostmapping            | table | postgres ambari | clusters                      | table | postgres ambari | clusterservices               | table | postgres ambari | clusterstate                  | table | postgres ambari | confgroupclusterconfigmapping | table | postgres ambari | configgroup                   | table | postgres ambari | configgrouphostmapping        | table | postgres ambari | ds_fileresourceitem_1         | table | ambari ambari | ds_jobimpl_4                  | table | ambari ambari | ds_savedquery_5               | table | ambari ambari | ds_testbean_2                 | table | ambari ambari | ds_udf_3                      | table | ambari ambari | execution_command             | table | postgres ambari | extension                     | table | postgres ambari | extensionlink                 | table | postgres ambari | groups                        | table | postgres ambari | host_role_command             | table | postgres ambari | host_version                  | table | postgres ambari | hostcomponentdesiredstate     | table | postgres ambari | hostcomponentstate            | table | postgres ambari | hostconfigmapping             | table | postgres ambari | hostgroup                     | table | postgres ambari | hostgroup_component           | table | postgres ambari | hostgroup_configuration       | table | postgres ambari | hosts                         | table | postgres ambari | hoststate                     | table | postgres ambari | kerberos_descriptor           | table | postgres ambari | kerberos_principal            | table | postgres ambari | kerberos_principal_host       | table | postgres ambari | key_value_store               | table | postgres ambari | members                       | table | postgres ambari | metainfo                      | table | postgres ambari | permission_roleauthorization  | table | postgres ambari | qrtz_blob_triggers            | table | postgres ambari | qrtz_calendars                | table | postgres ambari | qrtz_cron_triggers            | table | postgres ambari | qrtz_fired_triggers           | table | postgres ambari | qrtz_job_details              | table | postgres ambari | qrtz_locks                    | table | postgres ambari | qrtz_paused_trigger_grps      | table | postgres ambari | qrtz_scheduler_state          | table | postgres ambari | qrtz_simple_triggers          | table | postgres ambari | qrtz_simprop_triggers         | table | postgres ambari | qrtz_triggers                 | table | postgres ambari | remoteambaricluster           | table | postgres ambari | remoteambariclusterservice    | table | postgres ambari | repo_version                  | table | postgres ambari | request                       | table | postgres ambari | requestoperationlevel         | table | postgres ambari | requestresourcefilter         | table | postgres ambari | requestschedule               | table | postgres ambari | requestschedulebatchrequest   | table | postgres ambari | role_success_criteria         | table | postgres ambari | roleauthorization             | table | postgres ambari | servicecomponent_history      | table | postgres ambari | servicecomponentdesiredstate  | table | postgres ambari | serviceconfig                 | table | postgres ambari | serviceconfighosts            | table | postgres ambari | serviceconfigmapping          | table | postgres ambari | servicedesiredstate           | table | postgres ambari | setting                       | table | postgres ambari | stack                         | table | postgres ambari | stage                         | table | postgres ambari | topology_host_info            | table | postgres ambari | topology_host_request         | table | postgres ambari | topology_host_task            | table | postgres ambari | topology_hostgroup            | table | postgres ambari | topology_logical_request      | table | postgres ambari | topology_logical_task         | table | postgres ambari | topology_request              | table | postgres ambari | upgrade                       | table | postgres ambari | upgrade_group                 | table | postgres ambari | upgrade_item                  | table | postgres ambari | users                         | table | postgres ambari | viewentity                    | table | postgres ambari | viewinstance                  | table | postgres ambari | viewinstancedata              | table | postgres ambari | viewinstanceproperty          | table | postgres ambari | viewmain                      | table | postgres ambari | viewparameter                 | table | postgres ambari | viewresource                  | table | postgres ambari | viewurl                       | table | postgres ambari | widget                        | table | postgres ambari | widget_layout                 | table | postgres ambari | widget_layout_user_widget     | table | postgres(109 rows)

5、查看表结构
相当于desc tblname

ambari-> \d users                                       Table "ambari.users"        Column         |            Type             |                  Modifiers                  -----------------------+-----------------------------+--------------------------------------------- user_id               | integer                     | not null principal_id          | bigint                      | not null ldap_user             | integer                     | not null default 0 user_name             | character varying(255)      | not null user_type             | character varying(255)      | not null default 'LOCAL'::character varying create_time           | timestamp without time zone | default now() user_password         | character varying(255)      |  active                | integer                     | not null default 1 active_widget_layouts | character varying(1024)     | default NULL::character varyingIndexes:    "pk_users" PRIMARY KEY, btree (user_id)    "unq_users_0" UNIQUE, btree (user_name, user_type)Foreign-key constraints:    "fk_users_principal_id" FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id)Referenced by:    TABLE "members" CONSTRAINT "fk_members_user_id" FOREIGN KEY (user_id) REFERENCES users(user_id)

• 在列出所有表时，通过表名可以大概确定 alert_current，alert_definition表，笔者第一次就直接查询alert_current 表，经过对比其最新时间戳（去掉后三位）发现记录的信息确实是当前整个平台最新记录的信息。如下：

在linux中对时间戳进行转换（去掉后三位）[root@hdp39 ~]# date -d @1505959825Thu Sep 21 10:10:25 CST 2017

select alert_id,definition_id,latest_timestamp,latest_text,occurrences,firmness from alert_current where definition_id=24;     66 |            24 |    1505959825614 | Remaining Capacity:[151043118841], Total Capacity:[18% Used, 184952968704]   

在Ambari界面中发现alert_definition表的信息则是如下信息

具体监控信息

ambari=> select definition_id from alert_definition where label='DataNode Heap Usage'; definition_id ---------------            15ambari=> select alert_id,definition_id,latest_timestamp,latest_text,occurrences,firmness from alert_current where definition_id=15; alert_id | definition_id | latest_timestamp |                    latest_text                     | occurrences | firmness ----------+---------------+------------------+----------------------------------------------------+-------------+----------       59 |            15 |    1505977580226 | Used Heap:[5%, 47.36313 MB], Max Heap: 1004.0 MB   |       12217 | HARD       65 |            15 |    1505977585580 | Used Heap:[6%, 64.779465 MB], Max Heap: 1004.0 MB  |       12218 | HARD       62 |            15 |    1505977581769 | Used Heap:[13%, 130.01476 MB], Max Heap: 1004.0 MB |       12218 | HARD(3 rows)

通过Ambari接口获取相关信息

笔者比较懒并没有认真研究Ambari 接口，因而直接通过Debug模式查看其调用的接口，然后模拟登录并发送对应的接口获取相关信息。如下，获取集群CPU使用情况：

通过上图可以获知，发送对应的请求可获得Json相关信息。

• 最终可通过BI工具或Echarts等工具对相关信息进行可视化呈现，如下只是简单说明可以这么做而已。
  
  由上述可知，通过alert_definition 与alert_current 表可得出具体最新的监控信息，当然还可以通过组合其他表而得出更多的监控信息。