Project "Fuzzy Support Vector Machine"


Introduction

As mentioned in the other sections, this project is my first in the field of Machine Learning. Basically, this program is an implementation of the papers [1] and [2]. The considered papers contributed to the task of rule extraction from Support Vector Machines (SVM). Therefore, [2] creates SVFI (Support Vector Fuzzy Inference) rules based on support vectors of a given SVM by creating one rule per support vector. A rule consists of n clauses, where n is the number of dimensions. Each clause in a rule establishes a relationship between the dimension specific value of the vector and an input x. The class of the support vector is added after listing the clauses by the word “THEN” followed by the class and a coefficient which expresses the importance of that rule.

The following rule base has been learned from a simple two-dimensional data set with ten support vectors.

Rule 0: IF x0 isCloseTo 0.056 AND x1 isCloseTo 0.652 THEN 1.0 with 69.82
Rule 1: IF x0 isCloseTo 0.186 AND x1 isCloseTo 0.642 THEN 1.0 with 120.0
Rule 2: IF x0 isCloseTo 0.58 AND x1 isCloseTo 0.53 THEN 1.0 with 58.72
Rule 3: IF x0 isCloseTo 0.88 AND x1 isCloseTo 0.314 THEN 1.0 with 66.23
Rule 4: IF x0 isCloseTo 0.944 AND x1 isCloseTo 0.22 THEN 1.0 with 120.0
Rule 5: IF x0 isCloseTo 0.136 AND x1 isCloseTo 0.578 THEN -1.0 with 120.0
Rule 6: IF x0 isCloseTo 0.352 AND x1 isCloseTo 0.51 THEN -1.0 with 48.57
Rule 7: IF x0 isCloseTo 0.968 AND x1 isCloseTo 0.06 THEN -1.0 with 26.21
Rule 8: IF x0 isCloseTo 0.874 AND x1 isCloseTo 0.196 THEN -1.0 with 120.0
Rule 9: IF x0 isCloseTo 0.49 AND x1 isCloseTo 0.478 THEN -1.0 with 120.0

Obviously the interpretation of these rules is difficult because there is one rule for every support vector and each rule has n fuzzy clauses where n is the number of dimensions. Even for relatively small problems this approach leads to huge number of rules.

[1] enhances this SVFI approach by pruning the constructed rules both in number and in length. Additionally, the interpretation of the rules is incremented by using fuzzy partitions to describe the input space. The example above can be converted to the following four rules:

IF x0 is middle THEN CLASS=1.0 WITH 0.79
IF x0 is big AND x1 is small THEN CLASS=1.0 WITH 0.63
IF x1 is middle THEN CLASS=-1.0 WITH 0.79
IF x0 is big THEN CLASS=-1.0 WITH 0.79

The used example is named example_01 and is part of the downloadable archive.


Scope of operation

The scope of operation of the program is

  • Creating a SVM from raw data by using [3],
  • Create Support Vector Fuzzy Inference Rules [2],
  • Create interpretable Fuzzy Rules by using predefined fuzzy partitions [1]


GUI

The software provides an intuitive GUI to fulfill all tasks.
fuzzy_support_vector_machine_imagefuzzy_support_vector_machine_image


Download

  • FSVM.zip(0.6 MB) including the software, the examples and the developer-documentation (made with doxygen)
  • FSVM_Report (0.4 MB) Report of my Laboratory Internship, containing interpretation of examples and an evaluation of the work of [1].

This software, all it`s documentation and example-files are published under Creative Commons "deed", see here for further information.

You are free to share (to copy, distribute and transmit the work) and remix (to adapt) the work under the following conditions: Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Noncommercial. You may not use this work for commercial purposes. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.

 

References

[1] Stergios Papadimitriou and Konstantinos Terzidis. Efficient and interpretable fuzzy classifiers from data with support vector learning. Intelligent Data Analysis 9 (2005), page 527 to 550.
[2] Yixin Chen and James Z. Wang, Support Vector Learning for Fuzzy Rule-Based Classification Systems, IEEE Transactions on Fuzzy Systems, Vol II, No. 6, December 2003 , page 716 to 728.
[3] Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm