Natural Orifice Translumenal Endoscopic Surgery is an emerging procedure that requires training and adoption to be successful. Currently no objective performance metrics exist for evaluating skills for NOTES. In this work, we have improved upon our previous study on objective performance metrics using kinematic measures by introducing two new measures, the flex and the roll and recruiting more subjects to increase the statistical power. The measures were evaluated in a transgastric NOTES appendectomy procedure performed with ex-vivo organs using the EASIE-RTM trainer box. Four motion tracking sensors attached to an endoscope were used to measure the scope position and orientation to compute the kinematic measures. Results from our study showed that completion time, economy of motion, jerk and roll of the scope are valid kinematic measures to differentiate between expert and novice NOTES surgeons.