Spark 2.0 streaming中 InputDStream 和 ReceiverInputDStream 的区别

InputStream是所有输入流的抽象基类。 这个类提供start()和stop()方法，这两个方法被Spark 流系统用来启动开始接收数据和停止接收数据。

如果输入流仅在运行在driver结点的服务和线程产生的新数据生成RDD，那么可能直接继承这个InputStream。以FileINputDStream为例，它是InputStream的一个子类，在driver端监控一个HDFS目录，如果有新文件生成，则用新文件生成RDDs。 

如果实现的输入流需要在工作结点运行一个接收器，使用[[org.apache.spark.streaming.dstream.ReceiverInputDStream]]作为父类。

ReceiverInputDStream是一个抽像类，用于需要在工作结点启动一个接收器来接收外部数据。 ReceiverInputDStream的具体实现必须定义getReceiver方法来得到一个[[org.apache.spark.streaming.receiver.Receiver]] 对象。Receiver对象会发送到工作结点来接收数据。

/** * This is the abstract base class for all input streams. This class provides methods * start() and stop() which are called by Spark Streaming system to start and stop * receiving data, respectively. * Input streams that can generate RDDs from new data by running a service/thread only on * the driver node (that is, without running a receiver on worker nodes), can be * implemented by directly inheriting this InputDStream. For example, * FileInputDStream, a subclass of InputDStream, monitors a HDFS directory from the driver for * new files and generates RDDs with the new files. For implementing input streams * that requires running a receiver on the worker nodes, use * [[org.apache.spark.streaming.dstream.ReceiverInputDStream]] as the parent class. * * @param _ssc Streaming context that will execute this input stream */abstract class InputDStream[T: ClassTag](_ssc: StreamingContext)  extends DStream[T](_ssc) {

/** * Abstract class for defining any [[org.apache.spark.streaming.dstream.InputDStream]] * that has to start a receiver on worker nodes to receive external data. * Specific implementations of ReceiverInputDStream must * define [[getReceiver]] function that gets the receiver object of type * [[org.apache.spark.streaming.receiver.Receiver]] that will be sent * to the workers to receive data. * @param _ssc Streaming context that will execute this input stream * @tparam T Class type of the object of this stream */abstract class ReceiverInputDStream[T: ClassTag](_ssc: StreamingContext)  extends InputDStream[T](_ssc) {