GPU_SVM Source Code Webpage

GPU_SVM Source Code Webpage

GPU_SVM is a GPU accelerated SVM based on Libsvm, as attached the README from Libsvm for the distribution.

System requiremnts:

We have updated the code to be compatible with the latest version of LIBSVM, which is LIBSVM 3.21.
Please have the NVIDIA Graphics card with CUDA support installed in the computer system, as well as the newest NVIDIA drivers and CUDA toolkit 10.0 installed in the operating system.
The current operating system is Linux-based, although we have our experiments tested on Ubuntu.

Please use the link for source code downloading under the GNU licence.

The idea and code are motivated by LIBSVM, ThunderSVM and the GPU-accelerated LIBSVM. Please check the paper for references.

Additional Information

Please feel free to cite following paper if GPU_SVM fits in your work:

H. Chen, L. Wang, Y. Jin, C. Chi, F. Li, H. Chu, J. Shen, "Hyperparameter Estimation in SVM with GPU Acceleration for Prediction of Protein-Protein Interactions", Proc. of IEEE International Conference on Big Data, 2019, pp. 1-8

Attached is the README from Libsvm

Libsvm is a simple, easy-to-use, and efficient software for SVM classification and regression. It solves C-SVM classification, nu-SVM classification, one-class-SVM, epsilon-SVM regression, and nu-SVM regression. It also provides an automatic model selection tool for C-SVM classification. This document explains the use of libsvm.

Libsvm is available at http://www.csie.ntu.edu.tw/~cjlin/libsvm Please read the COPYRIGHT file before using libsvm.

Table of Contents

Quick Start

If you are new to SVM and if the data is not large, please go to `tools' directory and use easy.py after installation. It does everything automatic -- from data scaling to parameter selection.

Usage: easy.py training_file [testing_file]

More information about parameter selection can be found in `tools/README.'

Installation and Data Format

On Unix systems, type make' to build thesvm-train' and `svm-predict' programs. Run them without arguments to show the usages of them.

On other systems, consult Makefile' to build them (e.g., see 'Building Windows binaries' in this file) or use the pre-built binaries (Windows binaries are in the directorywindows').

The format of training and testing data files is:

Each line contains an instance and is ended by a '' character. For in the training set, we have the following cases.

In the test set,

The pair : gives a feature (attribute) value: is an integer starting from 1 and is a real number. The only exception is the precomputed kernel, where starts from 0; see the section of precomputed kernels. Indices must be in ASCENDING order.

A sample classification data included in this package is heart_scale'. To check if your data is in a correct form, usetools/checkdata.py' (details in `tools/README').

Type svm-train heart_scale', and the program will read the training data and output the model fileheart_scale.model'. If you have a test set called heart_scale.t, then type svm-predict heart_scale.t heart_scale.model output' to see the prediction accuracy. Theoutput' file contains the predicted class labels.

For classification, if training data are in only one class (i.e., all labels are the same), then svm-train' issues a warning message:Warning: training data in only one class. See README for details,' which means the training data is very unbalanced. The label in the training data is directly returned when testing.

There are some other useful programs in this package.

svm-scale:

This is a tool for scaling input data file.

svm-toy:

This is a simple graphical interface which shows how SVM
separate data in a plane. You can click in the window to
draw data points. Use "change" button to choose class
1, 2 or 3 (i.e., up to three classes are supported), "load"
button to load data from a file, "save" button to save data to
a file, "run" button to obtain an SVM model, and "clear"
button to clear the window.

You can enter options in the bottom of the window, the syntax of
options is the same as `svm-train'.

Note that "load" and "save" consider dense data format both in
classification and the regression cases. For classification,
each data point has one label (the color) that must be 1, 2,
or 3 and two attributes (x-axis and y-axis values) in
[0,1). For regression, each data point has one target value
(y-axis) and one attribute (x-axis values) in [0, 1).

Type `make' in respective directories to build them.

You need Qt library to build the Qt version.
(available from http://www.trolltech.com)

You need GTK+ library to build the GTK version.
(available from http://www.gtk.org)

The pre-built Windows binaries are in the `windows'
directory. We use Visual C++ on a 64-bit machine.

`svm-train' Usage

Usage: svm-train [options] training_set_file [model_file] options: -s svm_type : set type of SVM (default 0) 0 -- C-SVC (multi-class classification) 1 -- nu-SVC (multi-class classification) 2 -- one-class SVM 3 -- epsilon-SVR (regression) 4 -- nu-SVR (regression) -t kernel_type : set type of kernel function (default 2) 0 -- linear: u'v 1 -- polynomial: (gammau'v + coef0)^degree 2 -- radial basis function: exp(-gamma|u-v|^2) 3 -- sigmoid: tanh(gammau'v + coef0) 4 -- precomputed kernel (kernel values in training_set_file) -d degree : set degree in kernel function (default 3) -g gamma : set gamma in kernel function (default 1/num_features) -r coef0 : set coef0 in kernel function (default 0) -c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1) -n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5) -p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1) -m cachesize : set cache memory size in MB (default 100) -e epsilon : set tolerance of termination criterion (default 0.001) -h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1) -b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0) -wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1) -v n: n-fold cross validation mode -q : quiet mode (no outputs)

The k in the -g option means the number of attributes in the input data.

option -v randomly splits the data into n parts and calculates cross validation accuracy/mean squared error on them.

See libsvm FAQ for the meaning of outputs.

`svm-predict' Usage

Usage: svm-predict [options] test_file model_file output_file options: -b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); for one-class SVM only 0 is supported

model_file is the model file generated by svm-train. test_file is the test data you want to predict. svm-predict will produce output in the output_file.

`svm-scale' Usage

Usage: svm-scale [options] data_filename options: -l lower : x scaling lower limit (default -1) -u upper : x scaling upper limit (default +1) -y y_lower y_upper : y scaling limits (default: no y scaling) -s save_filename : save scaling parameters to save_filename -r restore_filename : restore scaling parameters from restore_filename

See 'Examples' in this file for examples.

Tips on Practical Use

Examples

svm-scale -l -1 -u 1 -s range train > train.scale svm-scale -r range test > test.scale

Scale each feature of the training data to be in [-1,1]. Scaling factors are stored in the file range and then used for scaling the test data.

svm-train -s 0 -c 5 -t 2 -g 0.5 -e 0.1 data_file

Train a classifier with RBF kernel exp(-0.5|u-v|^2), C=10, and stopping tolerance 0.1.

svm-train -s 3 -p 0.1 -t 0 data_file

Solve SVM regression with linear kernel u'v and epsilon=0.1 in the loss function.

svm-train -c 10 -w1 1 -w-2 5 -w4 2 data_file

Train a classifier with penalty 10 = 1 * 10 for class 1, penalty 50 = 5 * 10 for class -2, and penalty 20 = 2 * 10 for class 4.

svm-train -s 0 -c 100 -g 0.1 -v 5 data_file

Do five-fold cross validation for the classifier using the parameters C = 100 and gamma = 0.1

svm-train -s 0 -b 1 data_file svm-predict -b 1 test_file data_file.model output_file

Obtain a model with probability information and predict test data with probability estimates

Precomputed Kernels

Users may precompute kernel values and input them as training and testing files. Then libsvm does not need the original training/testing sets.

Assume there are L training instances x1, ..., xL and. Let K(x, y) be the kernel value of two instances x and y. The input formats are:

New training instance for xi:

New testing instance for any x:

That is, in the training file the first column must be the "ID" of xi. In testing, ? can be any value.

All kernel values including ZEROs must be explicitly provided. Any permutation or random subsets of the training/testing files are also valid (see examples below).

Note: the format is slightly different from the precomputed kernel package released in libsvmtools earlier.

Examples:

Assume the original training data has three four-feature
instances and testing data has one instance:

15  1:1 2:1 3:1 4:1
45      2:3     4:3
25          3:1

15  1:1     3:1

If the linear kernel is used, we have the following new
training/testing sets:

15  0:1 1:4 2:6  3:1
45  0:2 1:6 2:18 3:0
25  0:3 1:1 2:0  3:1

15  0:? 1:2 2:0  3:1

? can be any value.

Any subset of the above training file is also valid. For example,

25  0:3 1:1 2:0  3:1
45  0:2 1:6 2:18 3:0

implies that the kernel matrix is

    [K(2,2) K(2,3)] = [18 0]
    [K(3,2) K(3,3)] = [0  1]

Library Usage

These functions and structures are declared in the header file svm.h'. You need to #include "svm.h" in your C/C++ source files and link your program withsvm.cpp'. You can see svm-train.c' andsvm-predict.c' for examples showing how to use them. We define LIBSVM_VERSION and declare `extern int libsvm_version;' in svm.h, so you can check the version number.

Before you classify test data, you need to construct an SVM model (`svm_model') using training data. A model can also be saved in a file for later use. Once an SVM model is available, you can use it to classify new data.

Java Version

The pre-compiled java class archive `libsvm.jar' and its source files are in the java directory. To run the programs, use

java -classpath libsvm.jar svm_train java -classpath libsvm.jar svm_predict java -classpath libsvm.jar svm_toy java -classpath libsvm.jar svm_scale

Note that you need Java 1.5 (5.0) or above to run it.

You may need to add Java runtime library (like classes.zip) to the classpath. You may need to increase maximum Java heap size.

Library usages are similar to the C version. These functions are available:

public class svm { public static final int LIBSVM_VERSION=323; public static svm_model svm_train(svm_problem prob, svm_parameter param); public static void svm_cross_validation(svm_problem prob, svm_parameter param, int nr_fold, double[] target); public static int svm_get_svm_type(svm_model model); public static int svm_get_nr_class(svm_model model); public static void svm_get_labels(svm_model model, int[] label); public static void svm_get_sv_indices(svm_model model, int[] indices); public static int svm_get_nr_sv(svm_model model); public static double svm_get_svr_probability(svm_model model); public static double svm_predict_values(svm_model model, svm_node[] x, double[] dec_values); public static double svm_predict(svm_model model, svm_node[] x); public static double svm_predict_probability(svm_model model, svm_node[] x, double[] prob_estimates); public static void svm_save_model(String model_file_name, svm_model model) throws IOException public static svm_model svm_load_model(String model_file_name) throws IOException public static String svm_check_parameter(svm_problem prob, svm_parameter param); public static int svm_check_probability_model(svm_model model); public static void svm_set_print_string_function(svm_print_interface print_func); }

The library is in the "libsvm" package. Note that in Java version, svm_node[] is not ended with a node whose index = -1.

Users can specify their output format by

your_print_func = new svm_print_interface()
{
    public void print(String s)
    {
        // your own format
    }
};
svm.svm_set_print_string_function(your_print_func);

Building Windows Binaries

Windows binaries are available in the directory `windows'. To re-build them via Visual C++, use the following steps:

  1. Open a DOS command box (or Visual Studio Command Prompt) and change to libsvm directory. If environment variables of VC++ have not been set, type

""C:Files (x86)Visual Studio 12.06464.bat""

You may have to modify the above command according which version of VC++ or where it is installed.

  1. Type

nmake -f Makefile.win clean all

  1. (optional) To build shared library libsvm.dll, type

nmake -f Makefile.win lib

  1. (optional) To build 32-bit windows binaries, you must
    1. Setup "C:Files (x86)Visual Studio 12.032.bat" instead of vcvars64.bat
    2. Change CFLAGS in Makefile.win: /D _WIN64 to /D _WIN32

Another way is to build them from Visual C++ environment. See details in libsvm FAQ.

Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.

See the README file in the tools directory.

MATLAB/OCTAVE Interface

Please check the file README in the directory `matlab'.

Python Interface

See the README file in python directory.

Additional Information

If you find LIBSVM helpful, please cite it as

Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

LIBSVM implementation document is available at http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf

For any questions and comments, please email cjlin@csie.ntu.edu.tw

Acknowledgments: This work was supported in part by the National Science Council of Taiwan via the grant NSC 89-2213-E-002-013. The authors thank their group members and users for many helpful discussions and comments. They are listed in http://www.csie.ntu.edu.tw/~cjlin/libsvm/acknowledgements