Joint Principal Component Analysis and Supervised k Means Algorithm via Non-Iterative Analytic Optimization Approach

It is worth noting that the traditional methods for performing both the dimensional reduction and the classification are via the two steps iterative approaches. In this case, performing the dimensional reduction does not consider the classification. On the other hand, the classification is performed in the original feature domain and it does not consider the dimensional reduction. Here, the transform matrix only takes an effect on the dimensional reduction, but not on the classification. The synergy between the dimensional reduction and the classification has been ignored. As a result, the overall performance is not optimal. To address this issue, this paper proposes a joint principal component analysis (PCA) and supervised k means approach for performing the dimensional reduction and the classification simultaneously. In particular, both the reconstruction error due to the dimensional reduction as well as the total distance between the cluster centers and the feature vectors in the transformed domain are minimized subject to the unitary condition of the transform matrix. Here, we have two decision variables. They are the transform matrix and the cluster centers, instead of a single decision variable in each iteration in the traditional iterative method. To find the analytical solution of the optimization problem, the first order derivative condition of the optimization problem is first expressed as the matrix equations. However, there is a structure deficiency on the matrix ...
Source: IEEE Transactions on Signal Processing - Category: Biomedical Engineering Source Type: research