June 2024 – Ghazi Hudeihed

Monthly Archives: June 2024

Blog, Machine Learning

Dropout Regularization

Dropout How does the mask impact memory during training? While the masks used in dropout regularization introduce some additional memory overhead during training, this impact is generally modest compared to the overall memory usage of the neural network model. The benefits of improved generalization and reduced overfitting often outweigh the minor increase in memory usage….

June 19, 2024 Ghazi Hudeihed

backpropagation backward pass batch size Chain Rule channels Convolutional Neural Network Drop Channel Drop Layer DropBlock feature maps feature size Forward Pass generalization gradient inference neural network Overfitting pattern extraction Regularization residual connections ResNet

Blog, Machine Learning

Mitigating Overfitting with Ridge Regression: A Step-by-Step Guide Using Polynomial Regression

Introduction One of the simplest ways to simulate overfitting is to use polynomial regression on a small dataset. We can fit a high-degree polynomial to a small dataset, which will lead to overfitting. Then we can see how regularization techniques like Ridge Regression (L2 regularization) help to mitigate the overfitting. Step 1: Generate a Small…

June 1, 2024 Ghazi Hudeihed

Data Analysis Data Science L2 Regularization Machine Learning Model Generalization Overfitting Polynomial Regression Python Ridge Regression scikit-learn