Decision Trees and Random Forests

#Learning Objectives

• Implement Gini impurity, entropy, and information gain calculations to find optimal decision tree splits
• Build a decision tree classifier from scratch with pre-pruning controls (max depth, min samples)
• Construct a random forest using bootstrap sampling and feature randomization, and explain why it reduces variance
• Compare MDI feature importance with permutation importance and identify when MDI is biased