We are going to now implement Bagging utilizing Scikit-Study on a pattern dataset. We’ll evaluate a single determination tree with a Bagging ensemble of determination bushes to look at efficiency enhancements.
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.metrics import accuracy_score
from sklearn.datasets import make_classification
# Create an artificial dataset
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
# Break up into prepare and take a look at units
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Practice a single determination tree
single_tree = DecisionTreeClassifier(random_state=42)
single_tree.match(X_train, y_train)
# Predictions
y_pred_tree = single_tree.predict(X_test)# Consider Efficiency
acc_tree = accuracy_score(y_test, y_pred_tree)
print(f"Single Determination Tree Accuracy: {acc_tree:.4f}")
# Practice a Bagging classifier with a number of determination bushes
bagging_model = BaggingClassifier(
base_estimator=DecisionTreeClassifier(), # Weak learner
n_estimators=50, # Variety of base fashions
random_state=42,
bootstrap=True # Permits bootstrapping
)
bagging_model.match(X_train, y_train)# Predictions
y_pred_bagging = bagging_model.predict(X_test)# Consider Efficiency
acc_bagging = accuracy_score(y_test, y_pred_bagging)
print(f"Bagging Classifier Accuracy: {acc_bagging:.4f}")
print(f"Enchancment in accuracy: {(acc_bagging - acc_tree) * 100:.2f}%")
Single Determination Tree Accuracy: 0.85
Bagging Classifier Accuracy: 0.89
Enchancment in accuracy: 4.00%
This exhibits that Bagging improves the accuracy of a high-variance mannequin like Determination Timber.
