Overfitting Remedy by Sparsifying Regularization on Fully-Connected Layers of CNNs

Publication date: Available online 18 August 2018Source: NeurocomputingAuthor(s): Qi Xu, Ming Zhang, Zonghua Gu, Gang PanAbstractDeep learning, especially Convolutional Neural Networks (CNNs), has been widely applied in many domains. The large number of parameters in a CNN allow it to learn complex features, however, they may tend to hinder generalization by over-fitting training data. Despite many previously proposed regularization methods, over-fitting is still a problem in training a robust CNN. Among many factors that lead to over-fitting, the numerous parameters of fully-connected layers (FCLs) of a typical CNN should be taken into account. This paper proposes the SparseConnect, a simple idea which alleviates over-fitting by sparsifying connections to FCLs. Experimental results on three benchmark datasets MNIST, CIFAR10 and ImageNet show that the SparseConnect outperforms several state-of-the-art regularization methods.

Source: Neurocomputing - August 18, 2018 Category: Neuroscience Source Type: research