The goal of this project is to implement the support vector machine algorithm to train and use an SVM classifier, relying on a quadratic programming solver.
Here is an updated version of the handout that clarifies a few points that were ambiguous or misleading in class.
Write two functions in Python 3 with the following signatures:
train(data, targets, k, C=None) classify(svm, inputs)
Where
data is an array-like with shape (n,d),
containing n data points each of dimension d.
targets is a list of size n containing
the target values for the data points in data.
k is a kernel function
C is an error parameter for soft margin classification.
When set to None (default), hard margin classification is
used.
svm is something representing
an SVM classifier, whatever the train function returns.
inputs is an array-like with shape (x, d),
containing x new data data points
each of dimension d.
The train function returns something that
represents a support vector machine binary classifier,
which then can be used by classify.
I recommend using the
cvxopt
package, specifically the
cvxopt.solvers.qp function.
See also
this tutorial.
Of course, if you wish to find another library that supports quadratic
programming (or, um, implement your own...) you are welcome to
do so.
The function cvxopt.solvers.qp returns a dictionary
from which you can retrieve the solution using key
'x'.
(Thanks to Malaika for spotting this in an example.)
This is documented under the function
cvxopt.solvers.coneqp for which
qp is a wrapper.
See that documentation for details,
specifically find the paragraph that begins with
"coneqp returns a dictionary that...".
Apparently, the inputs to cvxopt.solvers.qp must be
cvxopt matrices, not (for example) numpy arrays, although it is very easy to convert
from one to another.
See this for documentation
about cvxopt matrices
and
this for
converting between numpy arrays and cvxopt matrices.
See this document
for a short (4 pages including code examples), readable introduction to
quadratic programming using cvxopt.
See the file ~tvandrun/Public/cs394/kernel.py
for examples on how to make kernel functions.
You may use this code or improve it.
Test your implementation on at least two data sets and report on the performance.
Submit your code in two files, svm.py
and test_svm.py
so that the following code will work:
import svm clsfyr = svm.train(X_train, y_train, poly) results = svm.classify(clsyr, X_test)
... where poly is (for example) a kernel function for
polynomial features.
Moreover, the following should work from the command line:
python3 test_svm.py
Which should display information about the performance of your classifier.
Finally, include a file README
that (briefly) describes how you tested your
classifier and to what results, and anything else you
think it would be good for me to know in order for me
to give your submission the fairest grading.
To turn in:
Copy svm.py, test_svm.py,
README, and
any other files your code needs (such as data sets for
testing) to
/cslab/class/cs394/[your userid]/svm
Due Mon, April 26