Hive的UDAF编程：计算几何平均值

（1）eclipse上创建Map/Reduce工程，命名为GeoMeanPro，在创建前，先把hive/lib目录下的jar包复制到hadoop/lib目录下面；

（2）在创建的工程上添加class，新建包com.hive.geomean.udaf,并在包下建立GeoMean.java;

（3）GeoMean.java代码为：

package com.hive.geomean.udaf;

import org.apache.hadoop.hive.ql.exec.UDAF;

import org.apache.hadoop.hive.ql.exec.UDAFEvaluator;

import org.apache.hadoop.io.IntWritable;

public class GeoMean extends UDAF {  

  

    public static class GeoMeanUDAFEval implements UDAFEvaluator {  

        public static class PartialResult {  

            double sum;  

            long count;  

        }  

        private PartialResult pResult;   

        @Override  

        public void init() { 

            pResult = null;  

        }  

        //参数的入口函数

        public boolean iterate(IntWritable  value) {  

            if (value == null) {  

                return true;  

            }  

            if (pResult == null) {  

                pResult = new PartialResult(); 

                pResult.sum = 1;

                pResult.count = 0;

            }   

            pResult.sum *= value.get();  

            pResult.count++;  

            return true;  

        }  

        public PartialResult terminatePartial() {  

            return pResult;  

        }  

        public boolean merge(PartialResult other) {  

            if (other == null) {  

                return true;  

            }  

            if (pResult == null) {  

                pResult = new PartialResult(); 

                pResult.sum = 1;

                pResult.count = 0;

            }  

            pResult.sum *= other.sum;  

            pResult.count +=other.count;  

            return true;  

        }  

        public Double  terminate() {  

            if (pResult == null) {  

                return null;  

            }  

            return new Double (Math.pow(pResult.sum, 1.0/pResult.count)); 

        }  

    }  

}

（4）将工程export出jar包，并命名为geomean.jar，然后上传到/home/hadoop/class目录下：

（5）Hive的UDAF使用方法如下：

hive> add jar /home/hadoop/class/geomean.jar;                              

Added /home/hadoop/class/geomean.jar to class path

Added resource: /home/hadoop/class/geomean.jar

hive> create temporary function geomean as 'com.hive.geomean.udaf.GeoMean';      

OK

Time taken: 0.038 seconds

 

hive> select * from grade;

OK

1       90

2       80

3       70

Time taken: 0.112 seconds

 

hive> select geomean (grade) from grade;

Total MapReduce jobs = 1

Launching Job 1 out of 1

Number of reduce tasks determined at compile time: 1

In order to change the average load for a reducer (in bytes):

  set hive.exec.reducers.bytes.per.reducer=<number>

In order to limit the maximum number of reducers:

  set hive.exec.reducers.max=<number>

In order to set a constant number of reducers:

  set mapred.reduce.tasks=<number>

Starting Job = job_201503221120_0057, Tracking URL = http://Masterpc.hadoop:50030/jobdetails.jsp?jobid=job_201503221120_0057

Kill Command = /usr/hadoop/libexec/../bin/hadoop job  -kill job_201503221120_0057

Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1

2015-03-23 22:36:57,988 Stage-1 map = 0%,  reduce = 0%

2015-03-23 22:37:04,042 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.39 sec

2015-03-23 22:37:05,063 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.39 sec

。。。。。。 

2015-03-23 22:37:22,264 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.87 sec

MapReduce Total cumulative CPU time: 3 seconds 870 msec

Ended Job = job_201503221120_0057

MapReduce Jobs Launched: 

Job 0: Map: 1  Reduce: 1   Cumulative CPU: 3.87 sec   HDFS Read: 228 HDFS Write: 18 SUCCESS

Total MapReduce CPU Time Spent: 3 seconds 870 msec

OK

79.58114415792782

Time taken: 44.677 seconds

hive>