Rough face alignments result in suboptimal performance of face identification. This study presents an approach for identifying the gender based on facial images without proper face alignments. Instead of just using only the detected face patch for identification, a set of patches is randomly cropped around the face detection region. Each patch set is represented by a linear subspace and compared with other linear subspaces by measuring their canonical correlations. A similarity matrix comprised of the canonical correlations is then incorporated into an indefinite-kernel SVM formulation. The number of support vectors, which we call support subspaces, can be decided automatically, hence, we can avoid the dimension selection problem observed in our previous work. Our experimental results demonstrate that the proposed approach outperforms state-of-the-art methods.