Convex optimization for binary classifier aggregation in multiclass problems

Sunho Park, Tae Hyun Hwang, Seungjin Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multiclass problems are often decomposed into multiple binary problems that are solved by individual binary classifiers whose results are integrated into a final answer. Various methods, including all-pairs (APs), one-versus-all (OVA), and error correcting output code (ECOC), have been studied, to decompose multiclass problems into binary problems. However, little study has been made to optimally aggregate binary problems to determine a final answer to the multiclass problem. In this paper we present a convex optimization method for an optimal aggregation of binary classifiers to estimate class membership probabilities in multiclass problems. We model the class membership probability as a softmax function which takes a conic combination of discrepancies induced by individual binary classifiers, as an input. With this model, we formulate the regularized maximum likelihood estimation as a convex optimization problem, which is solved by the primal-dual interior point method. Connections of our method to large margin classifiers are presented, showing that the large margin formulation can be considered as a limiting case of our convex formulation. In the experiments on human disease classification, we demonstrate that our method outperforms existing aggregation methods as well as direct methods, in terms of the classification accuracy and F-score.

Original languageEnglish (US)
Title of host publicationSIAM International Conference on Data Mining 2014, SDM 2014
PublisherSociety for Industrial and Applied Mathematics Publications
Pages280-288
Number of pages9
Volume1
ISBN (Print)9781510811515
DOIs
Publication statusPublished - 2014
Event14th SIAM International Conference on Data Mining, SDM 2014 - Philadelphia, United States
Duration: Apr 24 2014Apr 26 2014

Other

Other14th SIAM International Conference on Data Mining, SDM 2014
CountryUnited States
CityPhiladelphia
Period4/24/144/26/14

    Fingerprint

Keywords

  • Binary classifier aggregation
  • Convex optimization
  • Human disease classification
  • Large margin learning
  • Multiclass learning

ASJC Scopus subject areas

  • Computer Science Applications
  • Software

Cite this

Park, S., Hwang, T. H., & Choi, S. (2014). Convex optimization for binary classifier aggregation in multiclass problems. In SIAM International Conference on Data Mining 2014, SDM 2014 (Vol. 1, pp. 280-288). Society for Industrial and Applied Mathematics Publications. https://doi.org/10.1137/1.9781611973440.32