Predict Loan Default with Decision Trees

Predict Loan Default with Decision Trees: A Step-by-Step Guide

Introduction

Predicting loan default is a critical task in finance analytics. Banks and financial institutions need to assess the risk associated with lending money to individuals or businesses. By accurately predicting whether a borrower will default, lenders can minimize losses and make informed decisions.

In this tutorial, we will use a Decision Tree classifier to predict loan default risk. Decision Trees are intuitive, interpretable, and effective for classification tasks, making them ideal for beginners in finance analytics.

We will use the Kaggle Loan Prediction dataset, which contains historical loan data with features like income, credit history, and loan amount. This dataset is well-suited for this tutorial because it includes both numerical and categorical features, allowing us to practice data cleaning, feature encoding, and model evaluation.

By the end of this tutorial, you will learn:

  • How to load and explore a loan dataset.
  • How to preprocess data for machine learning.
  • How to train and evaluate a Decision Tree model.
  • How to interpret the model’s predictions.

Let’s get started!

Step 1: Setting Up the Environment

Before we begin, ensure you have the following Python libraries installed:

  • pandas for data manipulation.
  • numpy for numerical operations.
  • scikit-learn for machine learning.
  • matplotlib and seaborn for data visualization.

You can install these libraries using pip:

pip install pandas numpy scikit-learn matplotlib seaborn

Step 2: Loading and Exploring the Dataset

Loading the Dataset

First, let’s load the dataset. You can download the Kaggle Loan Prediction dataset from Kaggle. For this tutorial, we assume the dataset is saved as loan_data.csv.

import pandas as pd

# Load the dataset
data = pd.read_csv('loan_data.csv')
print(data.head())

Exploring the Dataset

Let’s explore the dataset to understand its structure and features.

# Display basic information about the dataset
print(data.info())

# Display summary statistics
print(data.describe())

Understanding the Features

The dataset typically includes the following features:

  • Loan_ID: Unique identifier for each loan.
  • Gender: Male or Female.
  • Married: Whether the applicant is married (Yes/No).
  • Dependents: Number of dependents.
  • Education: Graduate or Not Graduate.
  • Self_Employed: Whether the applicant is self-employed (Yes/No).
  • ApplicantIncome: Income of the applicant.
  • CoapplicantIncome: Income of the co-applicant.
  • LoanAmount: Loan amount requested.
  • Loan_Amount_Term: Term of the loan in months.
  • Credit_History: Credit history (1 for good, 0 for bad).
  • Property_Area: Urban, Semi-Urban, or Rural.
  • Loan_Status: Target variable (Y for approved, N for rejected).

Step 3: Data Cleaning and Preprocessing

Handling Missing Values

Missing values can affect the performance of our model. Let’s check for missing values and handle them appropriately.

# Check for missing values
print(data.isnull().sum())

# Fill missing values for categorical features with the mode
data['Gender'].fillna(data['Gender'].mode()[0], inplace=True)
data['Married'].fillna(data['Married'].mode()[0], inplace=True)
data['Dependents'].fillna(data['Dependents'].mode()[0], inplace=True)
data['Self_Employed'].fillna(data['Self_Employed'].mode()[0], inplace=True)

# Fill missing values for numerical features with the median
data['LoanAmount'].fillna(data['LoanAmount'].median(), inplace=True)
data['Loan_Amount_Term'].fillna(data['Loan_Amount_Term'].median(), inplace=True)
data['Credit_History'].fillna(data['Credit_History'].median(), inplace=True)

Encoding Categorical Features

Machine learning models require numerical input. Let’s encode categorical features using Label Encoding or One-Hot Encoding.

from sklearn.preprocessing import LabelEncoder

# Initialize LabelEncoder
label_encoder = LabelEncoder()

# Encode categorical features
data['Gender'] = label_encoder.fit_transform(data['Gender'])
data['Married'] = label_encoder.fit_transform(data['Married'])
data['Education'] = label_encoder.fit_transform(data['Education'])
data['Self_Employed'] = label_encoder.fit_transform(data['Self_Employed'])
data['Property_Area'] = label_encoder.fit_transform(data['Property_Area'])
data['Loan_Status'] = label_encoder.fit_transform(data['Loan_Status'])

Feature Scaling

Feature scaling is not strictly necessary for Decision Trees, but it can help with visualization and other models. For this tutorial, we will skip scaling.

Step 4: Exploratory Data Analysis (EDA)

Visualizing the Target Variable

Let’s visualize the distribution of the target variable (Loan_Status).

import matplotlib.pyplot as plt
import seaborn as sns

# Plot the distribution of Loan_Status
sns.countplot(x='Loan_Status', data=data)
plt.title('Distribution of Loan Status')
plt.show()

Loan Status Distribution

From the plot, we can see the distribution of approved and rejected loans. This helps us understand the class balance in our dataset.

Visualizing Numerical Features

Let’s visualize the distribution of numerical features like ApplicantIncome and LoanAmount.

# Plot the distribution of ApplicantIncome
sns.histplot(data['ApplicantIncome'], kde=True)
plt.title('Distribution of Applicant Income')
plt.show()

Applicant Income Distribution

Visualizing Categorical Features

Let’s visualize the relationship between categorical features and the target variable.

# Plot the relationship between Education and Loan_Status
sns.countplot(x='Education', hue='Loan_Status', data=data)
plt.title('Loan Status by Education')
plt.show()

Loan Status by Education

Step 5: Training the Decision Tree Model

Splitting the Dataset

We will split the dataset into training and testing sets to evaluate the model’s performance.

from sklearn.model_selection import train_test_split

# Define features and target
X = data.drop(['Loan_ID', 'Loan_Status'], axis=1)
y = data['Loan_Status']

# Split the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Training the Model

Now, let’s train a Decision Tree classifier.

from sklearn.tree import DecisionTreeClassifier

# Initialize the Decision Tree classifier
dt_classifier = DecisionTreeClassifier(random_state=42)

# Train the model
dt_classifier.fit(X_train, y_train)

Evaluating the Model

Let’s evaluate the model’s performance using accuracy, precision, recall, and the confusion matrix.

from sklearn.metrics import accuracy_score, precision_score, recall_score, confusion_matrix

# Make predictions
y_pred = dt_classifier.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy:.2f}')

# Calculate precision
precision = precision_score(y_test, y_pred)
print(f'Precision: {precision:.2f}')

# Calculate recall
recall = recall_score(y_test, y_pred)
print(f'Recall: {recall:.2f}')

# Confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)
print('Confusion Matrix:')
print(conf_matrix)

Visualizing the Decision Tree

Let’s visualize the Decision Tree to understand how it makes decisions.

from sklearn.tree import plot_tree

# Plot the Decision Tree
plt.figure(figsize=(20, 10))
plot_tree(dt_classifier, feature_names=X.columns, class_names=['Rejected', 'Approved'], filled=True)
plt.show()

Decision Tree Visualization

Step 6: Improving the Model

Hyperparameter Tuning

We can improve the model’s performance by tuning hyperparameters like max_depth, min_samples_split, and min_samples_leaf.

from sklearn.model_selection import GridSearchCV

# Define the parameter grid
param_grid = {
    'max_depth': [3, 5, 7, 10],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4]
}

# Initialize GridSearchCV
grid_search = GridSearchCV(DecisionTreeClassifier(random_state=42), param_grid, cv=5)

# Perform grid search
grid_search.fit(X_train, y_train)

# Get the best parameters
best_params = grid_search.best_params_
print(f'Best Parameters: {best_params}')

Training the Improved Model

Let’s train the model with the best parameters.

# Initialize the improved Decision Tree classifier
improved_dt_classifier = DecisionTreeClassifier(
    max_depth=best_params['max_depth'],
    min_samples_split=best_params['min_samples_split'],
    min_samples_leaf=best_params['min_samples_leaf'],
    random_state=42
)

# Train the improved model
improved_dt_classifier.fit(X_train, y_train)

# Evaluate the improved model
y_pred_improved = improved_dt_classifier.predict(X_test)
accuracy_improved = accuracy_score(y_test, y_pred_improved)
print(f'Improved Accuracy: {accuracy_improved:.2f}')

Conclusion and Next Steps

In this tutorial, we learned how to predict loan default risk using a Decision Tree classifier. We covered the following steps:

  1. Loading and exploring the dataset.
  2. Cleaning and preprocessing the data.
  3. Performing exploratory data analysis (EDA).
  4. Training and evaluating a Decision Tree model.
  5. Improving the model through hyperparameter tuning.

Key Insights

  • Decision Trees are intuitive and effective for classification tasks.
  • Data cleaning and preprocessing are crucial for model performance.
  • Hyperparameter tuning can significantly improve model accuracy.

Next Steps

  • Experiment with other machine learning models like Random Forest or Logistic Regression.
  • Explore feature engineering techniques to create new features.
  • Deploy the model as a web application using Flask or Django.

This tutorial provides a solid foundation for predicting loan default risk. By following these steps, you can build and evaluate your own models for finance analytics. Happy coding!

Comments

Post a Comment

Popular posts from this blog

Build a Loan Approval Predictor Using Python and Machine Learning

Image
Introduction In the modern financial landscape, banks and lending institutions process thousands of loan applications daily. Making accurate loan approval decisions is crucial for minimizing financial risk while ensuring deserving applicants receive funding. This is where machine learning comes to the rescue! In this beginner-friendly tutorial, we'll build a loan approval predictor using Python and popular machine learning libraries. You'll learn how to analyze financial data, preprocess it effectively, and create a classification model that can predict whether a loan application should be approved or rejected. What You'll Learn How to handle real-world finance datasets Essential data preprocessing techniques Building and evaluating classification models Using Python , Pandas , and Scikit-learn for machine learning The Dataset We'll be working with the Kaggle Loan Approval dataset , which contains information...
Read more

Predict Stock Prices with LSTM Networks

Image
Predict Stock Prices with LSTM Networks Stock price prediction is a challenging yet highly rewarding application of machine learning. Accurate forecasts can help investors make informed decisions, hedge risks, and optimize portfolios. In this tutorial, you'll learn how to build a Long Short-Term Memory (LSTM) model— a type of recurrent neural network (RNN) —to predict stock prices using historical data from Yahoo Finance . By the end of this guide, you'll be able to: Load and preprocess financial time-series data. Create sequences for LSTM training. Build, train, and evaluate an LSTM model using TensorFlow/Keras . Visualize predictions and interpret results. Why Use LSTMs for Stock Price Prediction? Traditional time-series models like ARIMA assume linearity and struggle with complex patterns. LSTMs , however, excel at capturing long-term dependencies and non-linear trends in sequential data—making them ideal for financial forecasting. We'll use Yahoo Finance s...
Read more

Predict Customer Churn with Random Forest

Image
Predict Customer Churn with Random Forest Introduction Customer churn—when customers stop using a service—is a critical challenge for telecommunication companies. Predicting churn allows businesses to proactively retain customers, reduce revenue loss, and improve customer satisfaction. In this tutorial, we’ll use Python and Random Forest , a powerful machine learning algorithm, to predict customer churn using the Kaggle Telco Churn dataset . By the end of this guide, you’ll learn: How to load and preprocess real-world telecom data. How to train and evaluate a Random Forest classifier . How to interpret feature importance to understand what drives churn. Let’s get started! Step 1: Load and Explore the Dataset First, we’ll load the dataset and perform Exploratory Data Analysis (EDA) to understand its structure. 1.1 Import Libraries import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns from sklear...
Read more