the complexity of solving a l0-norm optimization problem

the solution for a sparse recovery problem is given by:

min||x||0min||x||0

s.ty=Axs.ty=Ax

The definition of ||x||0||x||0 is no. of non-zero entries in xx. This is also called the sparsity of the vector.

i.e., we are asking for the sparsest solution xx, that satisfies y=Axy=Ax.

Consider the simplest case where xx is 1-sparse but you don't know the location of that non-zero entry. In such a case, we have (N1)(N1) possibilities for a 1-sparse vector and to find the solution we have to examine the values of all the (N1)(N1) possibilities for finding the unique minimizer. Similarly, if you are told that xx is kk-sparse, you need to search (Nk)(Nk) possibilities. i.e., the algorithm grows as (Nk)(Nk) with increase in kk. Since kk is not known a priori, you have to check for all the NN possible values of kk.

Hence, the complexity of the algorithm is ∑i=1N(Ni)∑i=1N(Ni). This is called as NP hard problem which is an acronym for Non-polynomial time complexity.