Rough face alignments lead to suboptimal performance of face identification systems. In this study, we present a novel approach for identifying genders from facial images without proper face alignments. Instead of using only one input for test, we generate an image set by randomly cropping out a set of image patches from a neighborhood of the face detection region. Each image set is represented as a subspace and compared with other image sets by measuring the canonical correlation between two associated subspaces. By finding an optimal discriminative transformation for all training subspaces, the proposed approach with unaligned facial images is shown to outperform the state-of-the-art methods with face alignment.