RFTSystems commited on
Commit
6184900
ยท
verified ยท
1 Parent(s): a8da7b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -0
README.md CHANGED
@@ -10,4 +10,60 @@ pinned: false
10
  short_description: 'DCLR leads by a significant margin against LION-ADAM '
11
  ---
12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
10
  short_description: 'DCLR leads by a significant margin against LION-ADAM '
11
  ---
12
 
13
+ title: DCLR Optimizer for CIFAR-10 Classification
14
+ emojis: ["๐Ÿš€", "๐Ÿง ", "๐Ÿ“ˆ"]
15
+ colorFrom: green
16
+ colorTo: indigo
17
+ SDK: gradio
18
+ SDK_version: 4.19.1
19
+ app_file: app.py
20
+ human_use: This Space demonstrates the DCLR optimizer for image classification on CIFAR-10.
21
+ ---
22
+
23
+ # ๐Ÿš€ DCLR Optimizer for CIFAR-10 Image Classification
24
+
25
+ This Hugging Face Space showcases the **DCLR (Dynamic Consciousness-based Learning Rate)** optimizer applied to a SimpleCNN model for image classification on the CIFAR-10 dataset.
26
+
27
+ ## ๐Ÿง  Introduction to DCLR Optimizer
28
+
29
+ The DCLR optimizer is a novel approach that dynamically adjusts the learning rate based on the model's 'consciousness' (represented by the entropy of its output activations) and the gradient norm. Unlike traditional optimizers with fixed or schedule-based learning rates, DCLR aims to adapt more intelligently to the training landscape. It's designed to promote faster convergence and potentially better generalization by moderating step sizes based on the certainty of predictions and the steepness of the loss surface.
30
+
31
+ In our analysis, the best-tuned DCLR configuration demonstrated superior test accuracy compared to Adam, Lion, DCLRAdam, and even its original untuned configuration, highlighting its potential when properly configured.
32
+
33
+ ## ๐Ÿ’ก How to use the Gradio Demo
34
+
35
+ **Webcam/Sketch (Optional)**: If enabled, you might be able to use your webcam or draw an image directly.
36
+ **Get Predictions**: The model will automatically process your image and display the top 3 predicted classes for the CIFAR-10 dataset along with their confidence scores.
37
+
38
+ Try uploading images of planes, cars, birds, cats, deer, dogs, frogs, horses, ships, or trucks to see how well the model classifies them!
39
+
40
+ ## ๐Ÿ—๏ธ Model Architecture and Dataset
41
+
42
+ ### Model: SimpleCNN
43
+
44
+ The model used is a `SimpleCNN`, a lightweight Convolutional Neural Network designed for basic image classification tasks. It consists of:
45
+ - Two convolutional layers (`nn.Conv2d`), each followed by a ReLU activation and Max Pooling.
46
+ - Fully connected layers (`nn.Linear`) to process the flattened feature maps and output class scores.
47
+
48
+ ### Dataset: CIFAR-10
49
+
50
+ The model was trained on the **CIFAR-10 dataset**, which comprises 60,000 32x32 color images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images. The 10 classes are:
51
+ - `plane`, `car`, `bird`, `cat`, `deer`, `dog`, `frog`, `horse`, `ship`, `truck`
52
+
53
+ ## โš™๏ธ Hyperparameter Tuning and Optimal Parameters
54
+
55
+ Extensive hyperparameter tuning was performed for the DCLR optimizer using a grid search approach over different learning rates (`lr`) and `lambda_` values. The tuning process involved training the model for a fixed number of epochs (5 epochs for initial screening) with various combinations and evaluating their test accuracy.
56
+
57
+ Our analysis identified the following optimal hyperparameters for DCLR on this task:
58
+ - **Learning Rate (`lr`)**: `0.1`
59
+ - **Lambda (`lambda_`)**: `0.1`
60
+
61
+ This best-tuned DCLR configuration achieved a final test accuracy of **70.70%** over 20 epochs, significantly outperforming the original DCLR configuration and other optimizers like Adam and DCLRConscious, and performing competitively with Lion and DCLRAdam.
62
+
63
+
64
+ ## ๐Ÿ™ Acknowledgments
65
+
66
+ - The DCLR optimizer is inspired by research into dynamic learning rate adaptation based on Rendered Frame theory.
67
+ - CIFAR-10 dataset is provided by the Canadian Institute for Advanced Research.
68
+ - Gradio and Hugging Face for providing an excellent platform for sharing ML demos.
69
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference