🀞 𝐃𝐞𝐦𝐲𝐬𝐭𝐢𝐟𝐲𝐢𝐧𝐠 𝐂𝐨𝐧𝐟𝐮𝐬𝐢𝐨𝐧 𝐌𝐚𝐭𝐫𝐢𝐱

𝐇𝐨𝐰 𝐭𝐨 𝐒𝐩𝐨𝐭 𝐄𝐫𝐫𝐨𝐫𝐬 𝐢𝐧 𝐂𝐥𝐚𝐬𝐬𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 🔍

Understanding 𝒘𝒉𝒆𝒓𝒆 𝒚𝒐𝒖𝒓 𝒎𝒐𝒅𝒆𝒍 𝒈𝒆𝒕𝒔 𝒊𝒕 𝒘𝒓𝒐𝒏𝒈 𝒊𝒔 𝒄𝒓𝒖𝒄𝒊𝒂𝒍 to improving performance. Lets learn, how a Confusion Matrix can help with this understanding.

𝐖𝐡𝐚𝐭 𝐢𝐬 𝐌𝐢𝐬𝐜𝐥𝐚𝐬𝐬𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧?

𝑀𝑖𝑠𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 ℎ𝑎𝑝𝑝𝑒𝑛𝑠 𝑤ℎ𝑒𝑛 𝑎 𝑚𝑜𝑑𝑒𝑙 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑠 𝑎𝑛 𝑖𝑛𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑐𝑙𝑎𝑠𝑠 𝑙𝑎𝑏𝑒𝑙 𝑓𝑜𝑟 𝑎 𝑔𝑖𝑣𝑒𝑛 𝑖𝑛𝑝𝑢𝑡. For example, classifying a cat 🐱 as a dog 🐶.

𝐖𝐡𝐚𝐭’𝐬 𝐚 𝐂𝐨𝐧𝐟𝐮𝐬𝐢𝐨𝐧 𝐌𝐚𝐭𝐫𝐢𝐱?

It is a table used to describe the 𝑝𝑒𝑟𝑓𝑜𝑟𝑚𝑎𝑛𝑐𝑒 𝑜𝑓 𝑎 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑚𝑜𝑑𝑒𝑙 𝑏𝑦 𝑐𝑜𝑚𝑝𝑎𝑟𝑖𝑛𝑔 𝑎𝑐𝑡𝑢𝑎𝑙 𝑙𝑎𝑏𝑒𝑙𝑠 𝑤𝑖𝑡ℎ 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑙𝑎𝑏𝑒𝑙𝑠. It breaks down predictions into:

• 𝐓𝐫𝐮𝐞 𝐏𝐨𝐬𝐢𝐭𝐢𝐯𝐞𝐬 (𝐓𝐏): Correct positive predictions.
• 𝐓𝐫𝐮𝐞 𝐍𝐞𝐠𝐚𝐭𝐢𝐯𝐞𝐬 (𝐓𝐍): Correct negative predictions.
• 𝐅𝐚𝐥𝐬𝐞 𝐏𝐨𝐬𝐢𝐭𝐢𝐯𝐞𝐬 (𝐅𝐏): Incorrectly predicted positives.
• 𝐅𝐚𝐥𝐬𝐞 𝐍𝐞𝐠𝐚𝐭𝐢𝐯𝐞𝐬 (𝐅𝐍): Incorrectly predicted negatives.

from torchmetrics import ConfusionMatrix
from mlxtend.plotting import plot_confusion_matrix

# 2. Setup confusion matrix
confmat = ConfusionMatrix(num_classes=len(class_names),
                          task='multiclass')

confmat_tensor = confmat(preds=y_pred_tensor,
                         target=test_data.targets)

# 3. Plot the confusion matrix
fig, ax = plot_confusion_matrix(
    conf_mat=confmat_tensor.numpy(),
    class_names=class_names,
    figsize=(10, 7)
);

𝐇𝐨𝐰 𝐂𝐚𝐧 𝐚 𝐂𝐨𝐧𝐟𝐮𝐬𝐢𝐨𝐧 𝐌𝐚𝐭𝐫𝐢𝐱 𝐇𝐞𝐥𝐩 𝐈𝐝𝐞𝐧𝐭𝐢𝐟𝐲 𝐄𝐫𝐫𝐨𝐫 𝐑𝐞𝐠𝐢𝐨𝐧𝐬?

The confusion matrix helps you identify error classes:
•  𝐅𝐚𝐥𝐬𝐞 𝐏𝐨𝐬𝐢𝐭𝐢𝐯𝐞𝐬 (𝐅𝐏): Indicates over-predicting certain classes.
•  𝐅𝐚𝐥𝐬𝐞 𝐍𝐞𝐠𝐚𝐭𝐢𝐯𝐞𝐬 (𝐅𝐍): Shows under-prediction or missed classifications.

By studying the matrix, you can spot patterns like:
•  Which classes are most often confused with others.
•  Whether certain classes are consistently misclassified.

𝐇𝐨𝐰 𝐭𝐨 𝐑𝐞𝐚𝐝 𝐚 𝐂𝐨𝐧𝐟𝐮𝐬𝐢𝐨𝐧 𝐌𝐚𝐭𝐫𝐢𝐱?

• 𝐃𝐢𝐚𝐠𝐨𝐧𝐚𝐥 𝐯𝐚𝐥𝐮𝐞s: Represent correct predictions (the more, the better!).
• 𝐎𝐟𝐟-𝐝𝐢𝐚𝐠𝐨𝐧𝐚𝐥 𝐯𝐚𝐥𝐮𝐞𝐬: Represent incorrect predictions or misclassifications.
• Top-right quadrant → False Positives (FP).
• Bottom-left quadrant → False Negatives (FN).

𝐹𝑜𝑟 𝑒𝑥𝑎𝑚𝑝𝑙𝑒, 𝑎 ℎ𝑖𝑔ℎ 𝐹𝑁 𝑐𝑜𝑢𝑛𝑡 𝑚𝑖𝑔ℎ𝑡 𝑠𝑖𝑔𝑛𝑎𝑙 𝑡ℎ𝑒 𝑚𝑜𝑑𝑒𝑙 𝑖𝑠 𝑚𝑖𝑠𝑠𝑖𝑛𝑔 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑎𝑠𝑒𝑠, 𝑤ℎ𝑖𝑐ℎ is 𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 for 𝑓𝑟𝑎𝑢𝑑 𝑑𝑒𝑡𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑟 𝑚𝑒𝑑𝑖𝑐𝑎𝑙 𝑑𝑖𝑎𝑔𝑛𝑜𝑠𝑒𝑠.

𝐓𝐢𝐩𝐬 💡

• 𝐔𝐬𝐞 𝐰𝐢𝐭𝐡 𝐢𝐦𝐛𝐚𝐥𝐚𝐧𝐜𝐞𝐝 𝐝𝐚𝐭𝐚𝐬𝐞𝐭𝐬: For datasets where some classes have more samples than others, better to use confusion matrix.
• 𝐏𝐫𝐞𝐜𝐢𝐬𝐢𝐨𝐧 𝐚𝐧𝐝 𝐑𝐞𝐜𝐚𝐥𝐥: Combine the confusion matrix with Precision and Recall to understand model’s performance.
• 𝐓𝐮𝐧𝐞 𝐭𝐡𝐫𝐞𝐬𝐡𝐨𝐥𝐝𝐬: If FPs or FNs are too high, consider adjusting your classification thresholds.
• 𝐋𝐨𝐨𝐤 𝐟𝐨𝐫 𝐩𝐚𝐭𝐭𝐞𝐫𝐧𝐬: Are certain classes consistently misclassified? Reviewing your data or refining features for those classes.

#MachineLearning #AI #DataScience #DeepLearning #PyTorch #ConfusionMatrix #Classification #ModelTraining