Discovering new intents is a crucial task in dialogue system. Most existing methods are limited in transferring the prior knowledge from known intents to new intents. These meth- ods also have difficulties in providing high-quality supervised signals to learn clustering-friendly features for grouping un- labeled intents. In this work, we propose an effective method (Deep Aligned Clustering) to discover new intents with the aid of limited known intent data. Firstly, we leverage a few labeled known intent samples as prior knowledge to pre-train the model. Then, we perform k-means to produce cluster as- signments as pseudo-labels. Moreover, we propose an align- ment strategy to tackle the label inconsistency during cluster- ing assignments. Finally, we learn the intent representations under the supervision of the aligned pseudo-labels. With an unknown number of new intents, we predict the number of intent categories by eliminating low-confidence intent-wise clusters. Extensive experiments on two benchmark datasets show that our method is more robust and achieves substantial improvements over the state-of-the-art methods.