Why Voting?

The idea behind boosting, and voting as well can be phrased as follows: Can a group of people somehow arrive at highly reasoned decisions, despite the weak judgement of the individual members?

The aim is to create a good classifier by combining several weak classifiers. A weak classifier is a classifier which is able to produce results that are only slightly better than guessing at random.

The basic approach is to apply repeatedly (in boosting this is done in an iterative way) a weak classifier to modifications of the data. In voting we simply apply the law of large numbers while in boosting we give more weight to misclassified data in each iteration.

Decision trees play an important role as our weak classifier. They serve as the basic method.