The inhibition of dipeptidyl peptidase IV (DPP-IV) has emerged as an attractive target in the treatment of type 2 diabetes. In view of this development, a critical analysis of structural requirements of the DPP-IV inhibitors is envisioned to identify the significant features toward design of selective inhibitors. The comparative molecular field analysis (CoMFA) and comparative molecular similarity indices analysis (CoMSIA) contour plots of pyrrolidine based analogues are used to analyze the structural requirements of a DPP-IV active site. The CoMFA model has shown a cross-validated q2 of 0.651 with a non-cross-validated r2 of 0.882 and explained 70.6% variance in the activity of external test compounds. In this, the steric and electrostatic fields have respectively contributed 59.8 and 40.2%, respectively, to the explained activity of the compounds. The CoMSIA model has shown optimum predictivity (cross-validated q2 = 0.661; non-cross-validated r2 = 0.803; external test set’s predictive r2 = 0.706) with four molecular fields namely, steric, electrostatic, hydrogen bond (HB)-donor, and HB-acceptor. The contour plots of molecular fields resulting from these studies have suggested: (i) steric restriction with small electron rich substituent at 2- and 3-position of pyrrolidine ring, (ii) presence of electropositive ring linker between the pyrrolidine head and aryl tail, (iii) presence of electron-rich groups around the aryl tail moiety, and (iv) presence of sulfonamide between the ring linker and aryl tail which would increase DPP-IV binding affinity of the compounds. These findings will help in the design of structurally related/new compounds as potential DPP-IV inhibitors.