beam问题
来源:互联网 发布:值乎app 编辑:程序博客网 时间:2024/06/05 19:21
一、
警告: Could not read Source statistics: {}
java.lang.IllegalArgumentException: Error matching the pattern or glob /home/admin/flink-1.3.0/README.txt: status NOT_FOUNDat org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:383)
at org.apache.beam.sdk.io.FileBasedSource.getEstimatedSizeBytes(FileBasedSource.java:209)
at org.apache.beam.runners.flink.translation.wrappers.SourceInputFormat.getStatistics(SourceInputFormat.java:89)
at org.apache.flink.optimizer.dag.DataSourceNode.computeOperatorSpecificDefaultEstimates(DataSourceNode.java:166)
at org.apache.flink.optimizer.dag.OptimizerNode.computeOutputEstimates(OptimizerNode.java:589)
at org.apache.flink.optimizer.traversals.IdAndEstimatesVisitor.postVisit(IdAndEstimatesVisitor.java:61)
at org.apache.flink.optimizer.traversals.IdAndEstimatesVisitor.postVisit(IdAndEstimatesVisitor.java:32)
at org.apache.flink.optimizer.dag.DataSourceNode.accept(DataSourceNode.java:250)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:520)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.DataSinkNode.accept(DataSinkNode.java:248)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:477)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:398)
at org.apache.flink.client.program.ClusterClient.getOptimizedPlan(ClusterClient.java:333)
at org.apache.flink.client.program.ClusterClient.getOptimizedPlan(ClusterClient.java:739)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:428)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:211)
at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:188)
at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:172)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:118)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:283)
at org.apache.beam.examples.WordCount.main(WordCount.java:185)
在idea上运行beam的时候,出现这种警告,但是不影响最终结果的输出。
不过,运行的会卡3分钟多,去下载jar文件。
十月 24, 2017 6:29:26 下午 org.apache.flink.runtime.client.JobSubmissionClientActor$1 call
信息: Upload jar files to job manager akka.tcp://flink@flinkmaster-1:6123/user/jobmanager.
十月 24, 2017 6:29:26 下午 org.apache.flink.runtime.blob.BlobClient uploadJarFiles
信息: Blob client connecting to akka.tcp://flink@flinkmaster-1:6123/user/jobmanager
十月 24, 2017 6:32:49 下午 org.apache.flink.runtime.client.JobSubmissionClientActor$1 call
信息: Submit job to the job manager akka.tcp://flink@flinkmaster-1:6123/user/jobmanager.
二、
mvn exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Pflink-runner -Dexec.args="--runner=FlinkRunner \
--inputFile=/home/admin/flink/flink-1.3.2/README.txt \
--output=/tmp/counts \
--flinkMaster=hbase-1:6123 \
--filesToStage=target/word-count-beam-bundled-0.1.jar
执行这个命令式,参数=之间不要留有空格,否则会报错
WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Argument 'hbase-1:36398' does not begin with '--'
at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)
at org.apache.beam.sdk.options.PipelineOptionsFactory.parseCommandLine(PipelineOptionsFactory.java:1513)
at org.apache.beam.sdk.options.PipelineOptionsFactory.access$200(PipelineOptionsFactory.java:104)
at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:288)
at org.apache.beam.examples.WordCount.main(WordCount.java:174)
... 6 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.817s
[INFO] Finished at: Tue Oct 24 21:50:21 CST 2017
[INFO] Final Memory: 19M/342M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. null: InvocationTargetException: Argument 'hbase-1:36398' does not begin with '--' -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
- beam问题
- 【NoisyHeart的炼成】Part3:Beam的平滑问题
- Beam search
- Beam search
- beam search
- android Beam
- beam搜索
- Apache Beam
- beam原理
- Apache Beam发布--- apache beam概述
- Beam搜素算法
- HDU 5091 Beam Cannon
- Beam me out!
- Multi-beam 解析过程
- Beam Search Algorithm
- Android Nfc Beam数据传输
- Beam Search 简介
- erlang beam文件反编译
- WPF中,定时系统重启功能的实现
- 梯度下降法
- 《深入理解java虚拟机》学习笔记3——垃圾回收算法
- day32
- 使用BottomNavigationView底部导航栏、添加数量角标提醒
- beam问题
- tensorflow(0)-入门
- day33
- 4033: [HAOI2015]树上染色
- android的四种启动方式
- 素数判断&&求m内能被3整除的所有数之和
- VS2013 CAFFE CUDNNV6 CUDA V8.0配置
- 区间DP心得及例题
- [组合] Codeforces #571A. Lengthening Sticks