eclipse下编写hive udf函数

来源：互联网发布：刺客信条 mac字幕编辑：程序博客网时间：2024/06/04 19:39

Eclipse下新建java project，编写转换小写测试函数：

package com.tom;import org.apache.hadoop.hive.ql.exec.UDF;import org.apache.hadoop.io.Text;public class HiveUDF extends UDF{public Text evaluate(final Text str) {if(str == null) return null;return new Text(str.toString().toLowerCase());}}

关联jar包：

hive-exec-0.13.1.jarhadoop-common-2.5.1.jar

右键export打包为udf_toLower.jar文件，传到hive主机/root/udfs下。

在hive中添加jar包：

hive> add jar /root/udfs/udf_toLower.jar;                  Added /root/udfs/udf_toLower.jar to class pathAdded resource: /root/udfs/udf_toLower.jar

创建udf：

hive> create temporary function lower as 'com.tom.HiveUDF';OKTime taken: 0.021 seconds

测试：

hive> select * from user;OK1       DUSIZHONG2       TOM3       JIMTime taken: 0.811 seconds, Fetched: 3 row(s)hive> select lower(name) from user;Total jobs = 1Launching Job 1 out of 1Number of reduce tasks is set to 0 since there's no reduce operatorStarting Job = job_1415866521696_0023, Tracking URL = http://master:8088/proxy/application_1415866521696_0023/Kill Command = /usr/hadoop/bin/hadoop job  -kill job_1415866521696_0023Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 02014-11-20 16:43:47,391 Stage-1 map = 0%,  reduce = 0%2014-11-20 16:43:56,965 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.62 secMapReduce Total cumulative CPU time: 2 seconds 620 msecEnded Job = job_1415866521696_0023MapReduce Jobs Launched: Job 0: Map: 1   Cumulative CPU: 2.62 sec   HDFS Read: 227 HDFS Write: 18 SUCCESSTotal MapReduce CPU Time Spent: 2 seconds 620 msecOKdusizhongtomjimTime taken: 34.496 seconds, Fetched: 3 row(s)

删除udf：

hive> drop temporary function lower;OKTime taken: 0.016 seconds

0 0