Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
|
@@ -10,4 +10,60 @@ pinned: false
|
|
| 10 |
short_description: 'DCLR leads by a significant margin against LION-ADAM '
|
| 11 |
---
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
| 10 |
short_description: 'DCLR leads by a significant margin against LION-ADAM '
|
| 11 |
---
|
| 12 |
|
| 13 |
+
title: DCLR Optimizer for CIFAR-10 Classification
|
| 14 |
+
emojis: ["๐", "๐ง ", "๐"]
|
| 15 |
+
colorFrom: green
|
| 16 |
+
colorTo: indigo
|
| 17 |
+
SDK: gradio
|
| 18 |
+
SDK_version: 4.19.1
|
| 19 |
+
app_file: app.py
|
| 20 |
+
human_use: This Space demonstrates the DCLR optimizer for image classification on CIFAR-10.
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
# ๐ DCLR Optimizer for CIFAR-10 Image Classification
|
| 24 |
+
|
| 25 |
+
This Hugging Face Space showcases the **DCLR (Dynamic Consciousness-based Learning Rate)** optimizer applied to a SimpleCNN model for image classification on the CIFAR-10 dataset.
|
| 26 |
+
|
| 27 |
+
## ๐ง Introduction to DCLR Optimizer
|
| 28 |
+
|
| 29 |
+
The DCLR optimizer is a novel approach that dynamically adjusts the learning rate based on the model's 'consciousness' (represented by the entropy of its output activations) and the gradient norm. Unlike traditional optimizers with fixed or schedule-based learning rates, DCLR aims to adapt more intelligently to the training landscape. It's designed to promote faster convergence and potentially better generalization by moderating step sizes based on the certainty of predictions and the steepness of the loss surface.
|
| 30 |
+
|
| 31 |
+
In our analysis, the best-tuned DCLR configuration demonstrated superior test accuracy compared to Adam, Lion, DCLRAdam, and even its original untuned configuration, highlighting its potential when properly configured.
|
| 32 |
+
|
| 33 |
+
## ๐ก How to use the Gradio Demo
|
| 34 |
+
|
| 35 |
+
**Webcam/Sketch (Optional)**: If enabled, you might be able to use your webcam or draw an image directly.
|
| 36 |
+
**Get Predictions**: The model will automatically process your image and display the top 3 predicted classes for the CIFAR-10 dataset along with their confidence scores.
|
| 37 |
+
|
| 38 |
+
Try uploading images of planes, cars, birds, cats, deer, dogs, frogs, horses, ships, or trucks to see how well the model classifies them!
|
| 39 |
+
|
| 40 |
+
## ๐๏ธ Model Architecture and Dataset
|
| 41 |
+
|
| 42 |
+
### Model: SimpleCNN
|
| 43 |
+
|
| 44 |
+
The model used is a `SimpleCNN`, a lightweight Convolutional Neural Network designed for basic image classification tasks. It consists of:
|
| 45 |
+
- Two convolutional layers (`nn.Conv2d`), each followed by a ReLU activation and Max Pooling.
|
| 46 |
+
- Fully connected layers (`nn.Linear`) to process the flattened feature maps and output class scores.
|
| 47 |
+
|
| 48 |
+
### Dataset: CIFAR-10
|
| 49 |
+
|
| 50 |
+
The model was trained on the **CIFAR-10 dataset**, which comprises 60,000 32x32 color images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images. The 10 classes are:
|
| 51 |
+
- `plane`, `car`, `bird`, `cat`, `deer`, `dog`, `frog`, `horse`, `ship`, `truck`
|
| 52 |
+
|
| 53 |
+
## โ๏ธ Hyperparameter Tuning and Optimal Parameters
|
| 54 |
+
|
| 55 |
+
Extensive hyperparameter tuning was performed for the DCLR optimizer using a grid search approach over different learning rates (`lr`) and `lambda_` values. The tuning process involved training the model for a fixed number of epochs (5 epochs for initial screening) with various combinations and evaluating their test accuracy.
|
| 56 |
+
|
| 57 |
+
Our analysis identified the following optimal hyperparameters for DCLR on this task:
|
| 58 |
+
- **Learning Rate (`lr`)**: `0.1`
|
| 59 |
+
- **Lambda (`lambda_`)**: `0.1`
|
| 60 |
+
|
| 61 |
+
This best-tuned DCLR configuration achieved a final test accuracy of **70.70%** over 20 epochs, significantly outperforming the original DCLR configuration and other optimizers like Adam and DCLRConscious, and performing competitively with Lion and DCLRAdam.
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
## ๐ Acknowledgments
|
| 65 |
+
|
| 66 |
+
- The DCLR optimizer is inspired by research into dynamic learning rate adaptation based on Rendered Frame theory.
|
| 67 |
+
- CIFAR-10 dataset is provided by the Canadian Institute for Advanced Research.
|
| 68 |
+
- Gradio and Hugging Face for providing an excellent platform for sharing ML demos.
|
| 69 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|