Table of contents

🎓 Intended learning outcomes

At the end of this lesson, the student is expected to:

• Understand that there exist use-cases in machine learning where model interpretability is more important than getting the best possible performance
• Recognise the similarities between flowcharts and decision trees
• Be able to give a formal definition of decision trees as recursive, greedy algorithms
• Understand how decision trees can be visualised both as graphs and as partitions of the feature space
• Understand how decision trees can be used for both classification and regression problems
• Define questions and question sets for numerical and categorical features
• Explain the learning algorithm used for fitting decision tree models
• Understand and be able to use the loss functions based on mean squared error, information gain and Gini impurity for building decision trees
• Be able to apply simple stopping criteria as regularisation techniques for decision-tree learning
• Be aware of at least one problem that is particularly ill-suited for decision-tree learning

This lesson introduces decision trees, which are a simple and intuitive model for classification and regression. Decision trees are very popular in applied machine learning, and are used in state-of-the-art approaches for certain types of problems. They also provide another good illustration of how concepts from information theory are useful in machine learning.

Motivating example: Ice cream preferences