Some CVXOPT Installation Issue

Following last post, python version 2.6.
During CVXOPT installation,
fatal error: Python.h: No such file or directory
Solution: sudo apt-get install python-dev


http://stackoverflow.com/questions/4097339/missing-python-h-while-trying-to-compile-a-c-extension-module

CVXOPT Python Installation Instructions

http://courses.csail.mit.edu/6.867/wiki/index.php?title=Python_Installation_Instructions

Including how to install Lapack

Convex Optimization

http://web.njit.edu/~simeone/ece788.htm

Fundamental theorem of linear algebra

In mathematics, the fundamental theorem of linear algebra makes several statements regarding vector spaces. These may be stated concretely in terms of the rank r of an m×nmatrix A and its singular value decomposition:

First, each matrix  (A has m rows and n columns) induces four fundamental subspaces. These fundamental subspaces are:

name of subspace	definition	containing space	dimension	basis
column space, range or image	im(A) or range(A)		r (rank)	The first r columns of
nullspace or kernel	ker(A) or null(A)		n − r (nullity)	The last (n − r) columns of
row space or coimage	im(AT) or range(AT)		r	The first r rows of
left nullspace or cokernel	ker(AT) or null(AT)		m − r	The last (m − r) rows of

Range of a matrix (Column Space)

In linear algebra, the column space of a matrix (sometimes called the range of a matrix) is the set of all possible linear combinations of its column vectors. The column space of an m × n matrix is a subspace of m-dimensional Euclidean space. The dimension of the column space is called the rank of the matrix.


The column space of a matrix is the image or range of the corresponding matrix transformation.
The dimension of the column space is called the rank of the matrix.


Definition:

Let A be an m × n matrix, with column vectors v₁, v₂, ..., v_n. A linear combination of these vectors is any vector of the form

where c₁, c₂, ..., c_n are scalars. The set of all possible linear combinations of v₁,...,v_n is called the column space of A. That is, the column space of A is the span of the vectorsv₁,...,v_n.

Basis

The columns of A span the column space, but they may not form a basis if the column vectors are not linearly independent. Fortunately, elementary row operations do not affect the dependence relations between the column vectors. This makes it possible to use row reduction to find a basis for the column space.

The left null space of A is the set of all vectors x such that x^TA = 0^T. It is the same as the null space of the transpose of A. The left null space is the orthogonal complement to the column space of A.

This can be seen by writing the product of the matrix A^T and the vector x in terms of the dot product of vectors:

Subgradient Method

From wikipedia
Subgradient methods are iterative methods for solving convex minimization problems. Originally developed by Naum Z. Shor and others in the 1960s and 1970s, subgradient methods are convergent when applied even to a non-differentiable objective function. When the objective function is differentiable, subgradient methods for unconstrained problems use the same search direction as the method of steepest descent.
Subgradient projection methods are often applied to large-scale problems with decomposition techniques. Such decomposition methods often allow a simple distributed method for a problem.


http://en.wikipedia.org/wiki/Subgradient_method

Semi-definite and Gramian matrix

It is called positive-semidefinite (or sometimes nonnegative-definite) if

A matrix M is positive-semidefinite if and only if it arises as the Gram matrix of some set of vectors. In contrast to the positive-definite case, these vectors need not be linearly independent.

Gram matrix:

In linear algebra, the Gramian matrix (or Gram matrix or Gramian) of a set of vectors  in an inner product space is the Hermitian matrix of inner products, whose entries are given by .

http://en.wikipedia.org/wiki/Positive-semidefinite_matrix#Negative-definite.2C_semidefinite_and_indefinite_matrices