Driving pattern recognition based on driving status features (GPS, gear, and speed etc.) is of central importance in the development of intelligent transportation. While it is expensive and labor intensive to obtain a large amount of labeled driving data in real applications. It makes the driving pattern recognition particularly difficult for those domains without labeled data. In this paper, to tackle this challenging recognition task, we propose a novel and robust Transfer Learning method for Driving Pattern Recognition (TLDPR) that can transfer knowledge from other related source domains with labeled data to the target domain. Compared to the traditional supervised learning, one of the major difficulties of transfer learning is that the data from different domains may have distinct distributions. The proposed TLDPR is able to reduce the distribution difference in RKHS between the samples in target and source domain with the same driving pattern, and it can preserve the local manifold structure simultaneously. In addition, an iterative ensemble strategy is implemented to make the model more robust using the pseudo-labels. To evaluate the performance of TLDPR, comprehensive experiments have been conducted on parking lots datasets. The results show TLDPR can substantially outperform the state-of-the-art methods.