Notes—Dense Vector and Sparse Vector

……未完待补充……
ref：
1. Spark官方文档
2. http://bbs.csdn.net/topics/391002544

在spark.ml.linalg里有两种vector——DenseVector 和 Sparse Vector，两者都继承于Vectors

1.两者区别
DenseVector： a value array

def:
Vectors.dense(values: Array[Double])
(直接把所有的元素都列出来了)

SparseVector : an index and a value array

def:
Vectors.sparse(size: Int, indices: Array[Int], values: Array[Double])
(存储元素的个数、以及非零元素的编号index和值value)

import org.apache.spark.mllib.linalg.{Vector, Vectors}// Create a dense vector (1.0, 0.0, 3.0).val dv: Vector = Vectors.dense(1.0, 0.0, 3.0)// Create a sparse vector (1.0, 0.0, 3.0) by specifying its indices and valuescorresponding to nonzero entries.val sv1: Vector = Vectors.sparse(3, Array(0, 2), Array(1.0, 3.0))// Create a sparse vector (1.0, 0.0, 3.0) by specifying its nonzero entries.val sv2: Vector = Vectors.sparse(3, Seq((0, 1.0), (2, 3.0)))

2.含类标签的点LabeledPoint

import org.apache.spark.mllib.linalg.Vectorsimport org.apache.spark.mllib.regression.LabeledPoint// Create a labeled point with a positive label and a dense feature vector.// 相当于这组dense特征的标签是1val pos = LabeledPoint(1.0, Vectors.dense(1.0, 0.0, 3.0))// Create a labeled point with a negative label and a sparse feature vector.// 相当于这组dense特征的标签是0val neg = LabeledPoint(0.0, Vectors.sparse(3, Array(0, 2), Array(1.0, 3.0)))

3.python的Scipy库也有类似的
https://docs.scipy.org/doc/scipy/reference/sparse.html