mapreduce系列(10)--自定义Inputformat

来源:互联网 发布:按键精灵mac版怎么用 编辑:程序博客网 时间:2024/05/22 06:50
0: jdbc:hive2://mini01:10000> select * from session limit 10;+----------------+-----------------------------------+--------------------------+--------------------+-----------------------------------------------------------------------+----------------------------+----------------------------+-------------------+-------------------------------+-----------------------+-----------------------------+----------------------------+------------------------------------+-----------------+----------------------+------------------+------------------+-------------------+-------------+---------------------+-------------------+--+| session.appid  |          session.appkey           | session.application_ver  | session.channelid  |                          session.session_id                           | session.local_time_string  | session.local_date_string  | session.duration  |      session.terminal_id      | session.device_model  | session.device_manufacture  | session.device_resolution  |         session.device_cpu         | session.access  | session.access_type  | session.carrier  | session.country  | session.language  | session.os  | session.os_version  | session.timezone  |+----------------+-----------------------------------+--------------------------+--------------------+-----------------------------------------------------------------------+----------------------------+----------------------------+-------------------+-------------------------------+-----------------------+-----------------------------+----------------------------+------------------------------------+-----------------+----------------------+------------------+------------------+-------------------+-------------+---------------------+-------------------+--+| 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225164024813_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165722191          | 20170225                   | 8108              | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165722192_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165726952          | 20170225                   | 4559              | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165726953_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165734705          | 20170225                   | 7396              | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165734708_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165803164          | 20170225                   | 27999             | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165803167_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165849707          | 20170225                   | 44360             | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165849708_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165853061          | 20170225                   | 3212              | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165853065_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165854179          | 20170225                   | 982               | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165854180_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165917006          | 20170225                   | 22777             | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165917009_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165930321          | 20170225                   | 13064             | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 || 10391          | 14677915459507539aa11e10150e4b25  | 1.0.2.26961              | 数字家庭Android        | 20170225165930323_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | 20170225165935414          | 20170225                   | 4938              | 4952511019810210210257525257  | NX513J                | nubia                       | 1920*1080                  | AArch64 Processor rev 1 (aarch64)  | Wi-Fi           | unknown              | Orange           | unknown          | unknown           | Android     | 5.1.1               | 8                 |+----------------+-----------------------------------+--------------------------+--------------------+-----------------------------------------------------------------------+----------------------------+----------------------------+-------------------+-------------------------------+-----------------------+-----------------------------+----------------------------+------------------------------------+-----------------+----------------------+------------------+------------------+-------------------+-------------+---------------------+-------------------+--+
0: jdbc:hive2://mini01:10000> select * from activity limit 10;+-----------------+-----------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------------------------+-----------------------------+-----------------------------+----------------------------+-------------------------+--+| activity.appid  |          activity.appkey          |                          activity.session_id                          |                                  activity.activityname                                  | activity.local_time_string  | activity.local_date_string  | activity.activityduration  | activity.activityindex  |+-----------------+-----------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------------------------+-----------------------------+-----------------------------+----------------------------+-------------------------+--+| 10391           | 14677915459507539aa11e10150e4b25  | 20170225164024813_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.SmartMainActivity                                                 | 0                           | 0                           | 8108                       | 1                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165722192_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.SmartMainActivity                                                 | 0                           | 0                           | 4559                       | 1                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165726953_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.setting.SettingActivity                                           | 0                           | 0                           | 998                        | 1                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165726953_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.device.hardware.smarthardware.chuseraccount.view.CHLoginActivity  | 0                           | 0                           | 6398                       | 2                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165734708_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.setting.SettingActivity                                           | 0                           | 0                           | 1388                       | 1                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165734708_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.device.hardware.smarthardware.chuseraccount.view.CHLoginActivity  | 0                           | 0                           | 26611                      | 2                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165803167_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.setting.SettingActivity                                           | 0                           | 0                           | 377                        | 1                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165803167_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.SmartMainActivity                                                 | 0                           | 0                           | 590                        | 2                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165803167_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.SmartMainActivity                                                 | 0                           | 0                           | 43393                      | 3                       || 10391           | 14677915459507539aa11e10150e4b25  | 20170225165849708_14:3E:BF:FF:94:49_14677915459507539AA11E10150E4B25  | com.cmri.universalapp.setting.SettingActivity                                           | 0                           | 0                           | 942                        | 1                       |+-----------------+-----------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------------------------+-----------------------------+-----------------------------+----------------------------+-------------------------+--+
0: jdbc:hive2://mini01:10000> select * from event where appid!=10391 limit 10;+--------------+-----------------------------------+----------------------------------------------------+--------------------------+--------------------------+------------+--------------+------------+------------------+---------------------------------------+----------------+--+| event.appid  |           event.appkey            |                  event.session_id                  | event.local_time_string  | event.local_date_string  | event.tag  | event.label  | event.acc  | event.attribute  |           event.terminal_id           | event.version  |+--------------+-----------------------------------+----------------------------------------------------+--------------------------+--------------------------+------------+--------------+------------+------------------+---------------------------------------+----------------+--+| 10392        | 1467853866494ceb53c83655097037de  | B9DB0E62-946D-4F5C-B049-ADA6D713D3171490251609284  | 20170323144658286        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | B9DB0E62-946D-4F5C-B049-ADA6D713D317  | 1.1.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195615058        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195632549        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195615058        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195632549        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195615058        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195632549        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195615058        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195632549        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          || 10392        | 1467853866494ceb53c83655097037de  | 017326BC-63D2-4920-9F64-ED88C2CC21461490266907953  | 20170323195615058        | 20170323                 | AndNews    | 为你推荐         | 1          | news_hot         | 017326BC-63D2-4920-9F64-ED88C2CC2146  | 1.2.0          |+--------------+-----------------------------------+----------------------------------------------------+--------------------------+--------------------------+------------+--------------+------------+------------------+---------------------------------------+----------------+--+
+--------+--------+---------------------+-------+--+| appid  | label  |      attribute      |  uv   |+--------+--------+---------------------+-------+--+| 10391  | 数码     | digital             | 18    || 10391  | 瘦身     | fitness             | 7     || 10391  | 奇葩     | marvel              | 6     || 10391  | 电影     | movie               | 20    || 10391  | 大家     | news_authority      | 113   || 10391  | 育儿     | news_baby           | 4     || 10391  | 汽车     | news_car            | 19    || 10391  | 探索     | news_discovery      | 1     || 10391  | 娱乐     | news_entertainment  | 126   || 10391  | 美文     | news_essay          | 3     || 10391  | 书摘     | news_excerpt        | 1     || 10391  | 时尚     | news_fashion        | 19    || 10391  | 财经     | news_finance        | 80    || 10391  | 美食     | news_food           | 5     || 10391  | 游戏     | news_game           | 11    || 10391  | 历史     | news_history        | 27    || 10391  | 为你推荐   | news_hot            | 603   || 10391  | 咪咕     | news_migu           | 14    || 10391  | 军事     | news_military       | 76    || 10391  | 养生     | news_regimen        | 6     || 10391  | 体育     | news_sports         | 87    || 10391  | 科技     | news_tech           | 22    || 10391  | 为你推荐   | news_top            | 1     || 10391  | 头条     | news_top            | 605   || 10391  | 旅游     | news_travel         | 24    || 10391  | 国际     | news_world          | 22    || 10391  | 正能量    | positive            | 4     || 10391  | 辟谣     | rumor               | 1     || 10392  | 数码     | digital             | 30    || 10392  | 瘦身     | fitness             | 1     || 10392  | 奇葩     | marvel              | 1     || 10392  | 电影     | movie               | 56    || 10392  | 大家     | news_authority      | 158   || 10392  | 育儿     | news_baby           | 1     || 10392  | 汽车     | news_car            | 67    || 10392  | 探索     | news_discovery      | 1     || 10392  | 娱乐     | news_entertainment  | 565   || 10392  | 美文     | news_essay          | 1     || 10392  | 书摘     | news_excerpt        | 1     || 10392  | 时尚     | news_fashion        | 30    || 10392  | 财经     | news_finance        | 168   || 10392  | 美食     | news_food           | 1     || 10392  | 游戏     | news_game           | 52    || 10392  | 历史     | news_history        | 66    || 10392  | 为你推荐   | news_hot            | 1800  || 10392  | 咪咕     | news_migu           | 27    || 10392  | 军事     | news_military       | 35    || 10392  | 养生     | news_regimen        | 1     || 10392  | 体育     | news_sports         | 375   || 10392  | 科技     | news_tech           | 20    || 10392  | 头条     | news_top            | 489   || 10392  | 旅游     | news_travel         | 29    || 10392  | 国际     | news_world          | 28    || 10392  | 正能量    | positive            | 1     || 10392  | 辟谣     | rumor               | 1     |+--------+--------+---------------------+-------+--+
+--------+--------+---------------------+-------+--+| appid  | label  |      attribute      |  uv   |+--------+--------+---------------------+-------+--+| 10392  | 为你推荐   | news_hot            | 1800  || 10392  | 书摘     | news_excerpt        | 1     || 10392  | 体育     | news_sports         | 375   || 10392  | 养生     | news_regimen        | 1     || 10392  | 军事     | news_military       | 35    || 10392  | 历史     | news_history        | 66    || 10392  | 咪咕     | news_migu           | 27    || 10392  | 国际     | news_world          | 28    || 10392  | 大家     | news_authority      | 158   || 10392  | 头条     | news_top            | 489   || 10392  | 奇葩     | marvel              | 1     || 10392  | 娱乐     | news_entertainment  | 565   || 10392  | 探索     | news_discovery      | 1     || 10392  | 数码     | digital             | 30    || 10392  | 旅游     | news_travel         | 29    || 10392  | 时尚     | news_fashion        | 30    || 10392  | 正能量    | positive            | 1     || 10392  | 汽车     | news_car            | 67    || 10392  | 游戏     | news_game           | 52    || 10392  | 电影     | movie               | 56    || 10392  | 瘦身     | fitness             | 1     || 10392  | 科技     | news_tech           | 20    || 10392  | 美文     | news_essay          | 1     || 10392  | 美食     | news_food           | 1     || 10392  | 育儿     | news_baby           | 1     || 10392  | 财经     | news_finance        | 168   || 10392  | 辟谣     | rumor               | 1     |+--------+--------+---------------------+-------+--+
+--------+--------+---------------------+------+--+| appid  | label  |      attribute      |  uv  |+--------+--------+---------------------+------+--+| 10391  | 为你推荐   | news_hot            | 603  || 10391  | 为你推荐   | news_top            | 1    || 10391  | 书摘     | news_excerpt        | 1    || 10391  | 体育     | news_sports         | 87   || 10391  | 养生     | news_regimen        | 6    || 10391  | 军事     | news_military       | 76   || 10391  | 历史     | news_history        | 27   || 10391  | 咪咕     | news_migu           | 14   || 10391  | 国际     | news_world          | 22   || 10391  | 大家     | news_authority      | 113  || 10391  | 头条     | news_top            | 605  || 10391  | 奇葩     | marvel              | 6    || 10391  | 娱乐     | news_entertainment  | 126  || 10391  | 探索     | news_discovery      | 1    || 10391  | 数码     | digital             | 18   || 10391  | 旅游     | news_travel         | 24   || 10391  | 时尚     | news_fashion        | 19   || 10391  | 正能量    | positive            | 4    || 10391  | 汽车     | news_car            | 19   || 10391  | 游戏     | news_game           | 11   || 10391  | 电影     | movie               | 20   || 10391  | 瘦身     | fitness             | 7    || 10391  | 科技     | news_tech           | 22   || 10391  | 美文     | news_essay          | 3    || 10391  | 美食     | news_food           | 5    || 10391  | 育儿     | news_baby           | 4    || 10391  | 财经     | news_finance        | 80   || 10391  | 辟谣     | rumor               | 1    |+--------+--------+---------------------+------+--+
0 0
原创粉丝点击