Feature Selection

#Learning Objectives

• Implement filter methods (variance threshold, mutual information, chi-squared) and wrapper methods (RFE, forward selection) from scratch
• Explain why mutual information captures nonlinear feature-target relationships that correlation misses
• Compare L1 regularization (embedded selection) with RFE (wrapper selection) and evaluate their computational tradeoffs
• Build a feature selection pipeline that combines multiple methods and demonstrate improved generalization on held-out data