A Pruning of Random Forests: a diversity-based heuristic measure to simplify a random forest ensemble

Abstract

Random forests are among the most successful ensemble methods. They are fast, noise-resistant and do not suffer from over-learning. Moreover, they offer possibilities of explanation and visualization. In this paper, we propose to simplify a set of random forests using an entropy function that measures the diversity of trees in the forest. The function is used in two types of paths: an SFS path and a path based on genetic algorithms. The proposed methods are applied to datasets of the UCI Repository. The results are encouraging and provide ensembles of smaller sizes with performances that are similar to or even,in some cases,exceed the performances of the initial forest. Moreover, the comparison between the two methods shows that in most cases SFS provides reduced ensemble compared to GA, but the latter gives better success rates in the majority of cases.