GROWING THE TREE

We typically grow the tree by successively splitting each node until nodes can no longer be split. Though this is locally optimal, it is not globally optimal. Nevertheless this produces useful trees with minimum computational complexity.

We can continue splitting nodes until:

• No more splits are possible (all samples at a node belong in the same class).
  
  
• The greatest likelihood increase (entropy reduction) falls below our pre-set threshold.
  
  
• The number of data samples falling in a leaf node falls below some threshold.

Nodes which can no longer be split are declared terminal nodes. When all active nodes are terminal nodes, tree growing terminates.