Decision Trees: An Intuitive Approach with Scikit-Learn in Python

A decision tree works like a flowchart. It asks one question, then branches on the answer, and keeps going until it reaches a leaf. A leaf holds the answer.

In this guide, we’ll walk through the process of building a decision tree using the well known Scikit-Learn library in Python, a popular choice for many data science practitioners. While there are various libraries like TensorFlow available for machine learning, Scikit-Learn remains a popular choice for its simplicity and efficiency.

Understanding Classifiers

Imagine a program that has to say whether a photo shows a man or a woman. Writing those rules by hand is a lot of work, and if the categories change you write them all again. A classifier learns the rules from examples instead.

A classifier looks at labelled data and works out where the classes separate. Give it a photo it has never seen and it gives you a label back.

The typical flow with classifiers involves:

  • Gathering data
  • Training the classifier with this data
  • Making predictions using the trained classifier
    Illustration of supervised learning
    For our decision tree classifier, we’ll be feeding it labeled data, a process known as supervised learning. Though our example uses simple data arrays, real-world applications generally require substantial datasets for accurate predictions.

Visualizing Decision Trees

Every node in a decision tree is a question, and each answer sends you left or right. These decisions are determined by input conditions, and as you traverse the tree based on these conditions, you ultimately arrive at an outcome or leaf.
Graphical representation of a decision tree
The computer walks the tree using the values you give it. With inputs like [False, True] it can predict ‘You’.

Setting up Scikit-Learn

To begin, ensure you have Scikit-Learn (sklearn) installed. If not, you can install it using pip:

sudo pip install sklearn

Additionally, it’s essential to have scipy installed:

sudo pip install scipy

Crafting a Decision Tree

First, let’s import the necessary module and initialize our decision tree model:

from sklearn import tree
clf = tree.DecisionTreeClassifier()

Next, let’s set up our training data:

# Features: [height, hair-length, voice-pitch]
X = [ 
    [180, 15,0],
    [167, 42,1],
    [136, 35,1],
    [174, 15,0],
    [141, 28,1]
] 
Y = ['man', 'woman', 'woman', 'man', 'woman']

Now, let’s train the classifier and make a prediction:

from sklearn import tree

clf = tree.DecisionTreeClassifier()

# Features: [height, hair-length, voice-pitch]
X = [ 
    [180, 15,0],
    [167, 42,1],
    [136, 35,1],
    [174, 15,0],
    [141, 28,1]
] 
Y = ['man', 'woman', 'woman', 'man', 'woman']

clf = clf.fit(X, Y)
prediction = clf.predict([[133, 37,1]])
print(prediction)

Reading helps, writing fixes it. PyChallenge has exercises on this and you can try them right now.