[pig] encountered errors at the first time

来源:互联网 发布:网络奇兵1游侠 编辑:程序博客网 时间:2024/06/11 18:39

1. Error Handling : -stop_on_failure

Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.


[This option is recommended byperformance and Efficiency in http://pig.apache.org/docs/r0.9.1/perf.html ]

With multi-query execution Pig processes an entire script or a batch of statements at once. By default Pig tries to run all the jobs that result from that, regardless of whether some jobs fail during execution. To check which jobs have succeeded or failed use one of these options.

First, Pig logs all successful and failed store commands. Store commands are identified by output path. At the end of execution a summary line indicates success, partial failure or failure of all store commands.

Second, Pig returns different code upon completion for these scenarios:

  • Return code 0: All jobs succeeded

  • Return code 1: Used for retrievable errors

  • Return code 2: All jobs have failed

  • Return code 3: Some jobs have failed

In some cases it might be desirable to fail the entire script upon detecting the first failed job. This can be achieved with the "-F" or "-stop_on_failure" command line flag. If used, Pig will stop execution when the first failed job is detected and discontinue further processing. This also means that file commands that come after a failed store in the script will not be executed (this can be used to create "done" files).

This is how the flag is used:

$ pig -F myscript.pigor$ pig -stop_on_failure myscript.pig

2. UnknownHostException

Failed Jobs:JobId   Alias   Feature Message OutputsN/A     A,B     MAP_ONLY        Message: java.net.UnknownHostException: sonora.peking.corp.com: sonora.peking.corp.comCaused by: java.net.UnknownHostException: sonora.peking.corp.com

Solution:

add the hostname: sonora.peking.corp.com to your host's configuration file: /etc/hosts

127.0.0.1 localhost  sonora.peking.corp.com


原创粉丝点击