A Detailed Review and Implementation of Dense Neural Networks for Handwritten Digit Recognition
Keywords:
handwritten digit recognition, deep learning, machine leraningAbstract
Handwritten digit recognition is an important task in computer vision and deep learning research. It serves as a standard for measuring how well classification algorithms perform. Convolutional neural networks (CNNs) lead this area because they excel at capturing spatial relationships, but fully connected dense neural networks can still perform well if they are properly preprocessed and adjusted. This paper offers a detailed review and practical implementation of a dense feedforward neural network for classifying grayscale handwritten digit images from an MNIST-like dataset. We explain the data preprocessing steps, architecture design, training methods, and evaluation metrics. The model, built using TensorFlow Keras, reaches over 97% validation accuracy in 10 epochs. This shows that dense architectures are still a good choice for simple classification tasks. We also compare our results with previous studies and discuss the model's strengths, weaknesses, and possible improvements.
Downloads
References
[1] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
[2] L. Deng, "The MNIST database of handwritten digit images for machine learning research," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, Nov. 2012.
[3] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
[4] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105, 2012.
[5] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[6] C. Szegedy et al., "Going deeper with convolutions," Proc. IEEE CVPR, pp. 1–9, 2015.
[7] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proc. IEEE CVPR, pp. 770–778, 2016.
[8] F. Chollet, "Xception: Deep learning with depthwise separable convolutions," Proc. IEEE CVPR, pp. 1251–1258, 2017.
[9] M. Sandler et al., "MobileNetV2: Inverted residuals and linear bottlenecks," Proc. IEEE CVPR, pp. 4510–4520, 2018.
[10] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[11] N. Srivastava et al., "Dropout: A simple way to prevent neural networks from overfitting," Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
[12] M. Abadi et al., "TensorFlow: Large-scale machine learning on heterogeneous systems," arXiv preprint arXiv:1603.04467, 2016.
[13] F. Rosenblatt, "The perceptron: A probabilistic model for information storage and organization in the brain," Psychological Review, vol. 65, no. 6, pp. 386–408, 1958.
[14] G. Hinton, S. Osindero, and Y. Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006.
[15] P. Y. Simard, D. Steinkraus, and J. C. Platt, "Best practices for convolutional neural networks applied to visual document analysis," Proc. ICDAR, vol. 3, pp. 958–962, 2003.
[16] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. Springer, 2009.
[17] S. S. Haykin, Neural Networks and Learning Machines, 3rd ed. Pearson, 2009.
[18] V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," Proc. ICML, pp. 807–814, 2010.
[19] M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.
[20] TensorFlow Developers, "TensorFlow documentation: MNIST classification," 2023. [Online]. Available: https://www.tensorflow.org/tutorials
Published
Issue
Section
License
Copyright (c) 2025 shruti patel (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
