Methods: . In the mentored student project (MSP) programme at Melaka Manipal Medical College, students undertake a short-term group research project under the guidance of their mentor. After data collection and analysis, students are required to write an abstract, present a poster and also write individual reflective summaries of their research experience. We evaluated the MSP programme using reflective summaries of a batch of undergraduate medical students. Data from 41 reflective summaries were analysed using the thematic analysis approach. The learning outcomes at the third and fourth levels of the Kirkpatrick evaluation model were determined from the summaries.
Results: . Students' reflective summaries indicated that they were satisfied with the MSP experience. In all the summaries, there was a mention of an improvement in teamwork skills through MSP. Improved relations with mentors were another relevant outcome. Improvement in communication skills and a positive change related to research attitude were also reported by students.
Conclusions: . Reflective summaries as a means to evaluate the MSP programme was found to be an easy, feasible and cost-effective method. The qualitative approach adopted for data analysis enabled the programme coordinators to assess the strengths and barriers of the programme.
MATERIALS AND METHODS: MCQ items in papers taken from Year II Parts A, B and C examinations for Sessions 2001/02, and Part B examinations for 2002/03 and 2003/04, were analysed to obtain their difficulty indices and discrimination indices. Each paper consisted of 250 true/false items (50 questions of 5 items each) on topics drawn from different disciplines. The questions were first constructed and vetted by the individual departments before being submitted to a central committee, where the final selection of the MCQs was made, based purely on the academic judgement of the committee.
RESULTS: There was a wide distribution of item difficulty indices in all the MCQ papers analysed. Furthermore, the relationship between the difficulty index (P) and discrimination index (D) of the MCQ items in a paper was not linear, but more dome-shaped. Maximal discrimination (D = 51% to 71%) occurred with moderately easy/difficult items (P = 40% to 74%). On average, about 38% of the MCQ items in each paper were "very easy" (P > or =75%), while about 9% were "very difficult" (P <25%). About two-thirds of these very easy/difficult items had "very poor" or even negative discrimination (D < or =20%).
CONCLUSIONS: MCQ items that demonstrate good discriminating potential tend to be moderately difficult items, and the moderately-to-very difficult items are more likely to show negative discrimination. There is a need to evaluate the effectiveness of our MCQ items.