Accuracy bounds for ensembles under 0-1 loss

R. Bouckaert

2002

Working Paper No. 04/02

Abstract

This paper is an attempt to increase the understanding in the behavior of ensembles for discrete variables in a quantitative way.  A set of tight upper and lower bounds for the accuracy of an ensemble is presented for wide classes of ensemble algorithms, including bagging and boosting.  The ensemble accuracy is expressed in terms of the accuracies of the members of the ensemble. Since those bounds represent best and worst case behavior only, we study typical behavior as well, and discuss its properties.  A parameterised bound is presented which describes ensemble bahavior as a mixture of dependent base classifier and independent base classifier areas.  Some empirical results are presented to support our conclusions.


Working Papers Series, ISSN: 1170-487X

Contact: working-papers@cs.waikato.ac.nz

Department of Computer Science, University of Waikato, Hamilton, New Zealand.

a Greenstone Digital Library