Random Forests for Complete Beginners

Given a new point (x,y),

We need to add another decision node to our decision tree:

Pretty simple, right? For example, the root node in our tree from earlier used the x feature with a test threshold of 2:

Intuitively, we want a decision node that makes a “good” split, where “good” can be loosely defined as separating different classes as much as possible. First, we calculate the Gini Impurity of the whole dataset:

Then, we calculate the Gini Impurities of the two branches:

Finally, we calculate Gini Gain by subtracting the weighted branch impurities from the original impurity:

We can calculate Gini Gain for every possible split in the same way:

After trying all thresholds for both x and y, we’ve found that the x=2 split has the highest Gini Gain, so we’ll make our root decision node use the x feature with a threshold of 2.

Source: victorzhou.com