The Two-Stage Examination: Assessment for Collaborative Learning

The work presented in this paper is related to the use of the two-stage examination assessment to promote collaborative learning that has an impact on students’ engagement, learning and performance. It is an initiative to support Student Success project at Kent University. The project aims to reduce the attainment gap of various cohorts of students and enhance their academic performance. This paper presents the analysis and results obtained by applying the 2-stage examination assessment in a second-year undergraduate computer science module entitled Software Engineering Process. The 2-stage examination used in this study has proven that effective learning can take place when students work collaboratively. The data used for the analysis is students’ overall performance in the module and also the data collected by distributing a questionnaire to students at the end of the academic term in addition to online-survey conducted during the final exam preparation period. Students’ performance of the targeted module has been recorded, analysed and contrasted with the previous year cohort. In addition, students’ feedback related to their learning experience is recorded and anlaysed. As per the students’ performance, questionnaire and survey analysis results, one can consider that the 2-stage examination is a unique assessment, beneficial and very useful for final examination preparation.

The work presented in this paper is related to the use of the two-stage examination assessment to promote collaborative learning that has an impact on students' engagement, learning and performance. It is an initiative to support Student Success project at Kent University. The project aims to reduce the attainment gap of various cohorts of students and enhance their academic performance. This paper presents the analysis and results obtained by applying the 2-stage examination assessment in a second-year undergraduate computer science module entitled Software Engineering Process. The 2-stage examination used in this study has proven that effective learning can take place when students work collaboratively. The data used for the analysis is students' overall performance in the module and also the data collected by distributing a questionnaire to students at the end of the academic term in addition to online-survey conducted during the final exam preparation period. Students' performance of the targeted module has been recorded, analysed and contrasted with the previous year cohort. In addition, students' feedback related to their learning experience is recorded and anlaysed. As per the students' performance, questionnaire and survey analysis results, one can consider that the 2-stage examination is a unique assessment, beneficial and very useful for final examination preparation.

Introduction
This paper concerns the implementation of the two-stage examination as the assessment of a second-year undergraduate computer science module named Software Engineering Process-CO548. The assessment is performed in week-9 of the term. In week-10, a face-to-face feedback session is delivered to highlight the strength and weaknesses of students' answers in addition to exploring methods of targeting exam-style questions. The data collected for this research is related to 119 students' performance. For the student cohort that attended the 2stage-examination, a questionnaire conducted in week-10 and an online survey conducted during final exams preparation period that consists of five questions probed students' views on the assessment and the learning experience. The initial findings show promising results that include enhanced students' engagement, maintained good performance against fixed assessment criteria and improved exam preparation and revision techniques. Student evaluations confirmed that using this type of assessment was beneficial and contributed to a significantly improved learning experience. The paper demonstrates and evaluates the effectiveness of the 2-stage examination as an assessment for collaborative learning. This work differs from previous work in some aspects such as the module subject area; to our knowledge, the type of assessment 2-stage examination has been used and analysed in medical science, physics, and other science subjects but little/not used or comprehensively analysed for specific technical modules such as those in computer science. Furthermore, the group formation process has been mainly managed by the teaching team through using Myers--Briggs Type Indicator (MBTI)-like personality test to help composing groups that have balance in personality types. The analysed data demonstrated how the assessment was able to provide students with a sound understanding of software engineering concepts that are covered in the 2-stage assessment. The analysis of students' feedback on specific examination questions that requires knowledge application is then linked to Bloom's taxonomy of thinking skills. The promising results concerning students learning experience, engagement and the maintained performance, are the main contributions of the work.

Related Research
Assessment and in particular the summative one produces a measure which sums up students' achievement and which only describes what has been achieved (Brown, S. & Knight, P., 2012). Exams are the key summative assessment components in which students are intensely engaged with study materials. However, exams in higher education provide no feedback and hence it isn't clear how much it contributes to students' learning (Black, P., & Wiliam, D., 1998). The Higher Education literature suggests that assessment should contribute to students' learning therefore, to overcome the exams' issue, the 2-stage exam came into practice. It is a relatively simple way to solve the problem of the traditional exams where a student starts with an individual attempt and then answer the same/similar questions in a group setting with immediate feedback from peers. Two-stage exams have been proved successful for sciences courses. For example (Gilley& Clarkston, 2014) implemented the 2-stage exam on a natural disaster course. The authors reported increases in student learning as a result of the collaborative part of the exam. Another study conducted by (Leight et al., 2012) provided the impact of the 2-stage exam on the retention of the contents of a biology course. Other studies demonstrated the impact of the 2-stage exam on academic performance, for example, (Stearns ,1996), reported an increase in students' performance after taking the mid-term exams in a 2-stage format.  incorporated the 2-stage exam in introductory physics module. The authors reported both advantages and disadvantages of the exam format, students' opinions, and the exam contribution to students' learning. Fournier et al. (2017) indicated that the group element reduces test anxiety. Students are aware of having the opportunity to obtain grades in weaker areas, which could build their confidence. Another study on physics teaching explored the outcome of using 2-stage exam on physics students' learning which is captured through students' feedback (Wieman et al., 2014). Furthermore, the 2-stage exams have been implemented in medical courses such as (Lindsley et al., 2016) examined the collaborative learning skills of medical students using the 2-stage examination. The authors divided the cohort into two groups and each group conducted 6 exams (one and two stages) alternatively. Research findings showed that final examination performance was not different from previous exams however, there was an improvement in the performance related to concepts that have been covered in the 2-stage exam. All these studies confirmed that the 2-stage exam develops students' collaboration skills, reduced exam anxiety, and increased students' motivation. However, the limitations were mainly related to the small number of exam questions to spare time for the group attempt, the administrative efforts required and differences in groups' harmony. In this, work, the groups' harmony is relatively resolved by using a questionnaire that is inspired by the MBTI psychological test (Carskadon, 1994) that helps composing groups that have balance in personalities which enable individuals to demonstrate their contribution in the group work.
Moreover, the work was inspired by seeing the success of the 2-stage exam on a software engineering process module and how popular it was from the students' perspective.

Method
This study, used two methods to explore the impact of using 2-stage examination on students' learning, engagement and performance. The analysis reported here is a combination of quantitative and qualitative analysis of data collected from a second-year computer science students to examine the learning experience obtained by conducting the new form of assessment. In order to explore the characteristics of the selected cohorts, data related to entry qualification and demographics have been gathered and compared with previous year cohort took the same module. The selected module for this study is level-5 module delivered over 12 weeks in 2-hour lecture and 1-hour class tutorial each week. The intended learning outcomes of the module are: LO1: Describe, explain and carry out the processes used in the production of quality software LO2: Describe the processes, techniques and deliverables associated with requirements engineering LO3: Describe a variety of approaches employed in software development and indicate the circumstances where such approaches may be appropriate LO4: Appreciate a range of software architectures and design processes. LO5: Understand the role of verification and validation, and the importance of testing. LO6: Identify the roles and responsibilities of members of a software development team and the methods of intercommunication. LO7: Understand project management including project scheduling, staffing, cost estimation and budgeting, configuration management, quality assurance and process improvement. LO8: Discuss the professional and legal duties software engineers owe to their employers, employees, customers and the wider public Table 2 shows the assessment components of the previous academic year and the proposed new assessment patterns of the current academic year. P-slips refer to the participation slips that were distributed to students in classes for their active participation. The total number of students taking the module with the new assessment components is 77 divided into 4 groups i.e 18-20 students per tutorial group. Following Biggs's concepts of constructive alignment of intended learning outcomes and assessment tasks (Biggs,2003) and other authors such as (Lie, 2002) and (Race, 2005) with opinion that students learn a massive amount from each other if they have been given time, place and opportunity to do so. In this work, each of the 4 groups have been subdivided into smaller groups (4-5) students each using MBTI-like questionnaire. In the proposed assessment pattern, the group work contributes mainly in the overall assessment that constitute 50% of the final mark of the module. Group assessment encourages discussions among students and knowledge testing. Moreover, it is constructively aligned with the intended learning outcomes of the module in particular LO6 and LO7. As mentioned earlier, students are divided into smaller groups to support and encourage collaborative learning. Forming groups can be a challenging task and issues relating to group harmony can have a negative impact on the performance of individual student. Therefore, the teaching team delivering the module designed an evaluation along the lines of a MBTI psychological test, targeted at group formation (Carskadon, 1994). The MBTI categories of learning styles relating to orientation (Extrovert or Introvert), perception (Sensing or Intuitive), decision making (Thinking or Feeling), and attitude to (Judgment or Perception) have been used. The MBTI-like questionnaire consists of 70 questions that covers the above categories. It was distributed to students during the first session of the term to help identify their type of personality. The teaching team worked actively in groups' formation with the aid of the MBTI-like questionnaire results to help identifying the type of personalities needed for each group. The results of the questionnaire are summarized in the table 3 below showing the type of personalities, number of students in each category and the percentage. As it is evident from Table 3, we can see that the highest percentage of Computer Science students in this group were E/ISTJ's followed by E/INTJ and E/INFJ respectively. There were 5 students who haven't participated in the questionnaire in which then we ensured that they joined different groups. At the end, the formed groups were balanced in terms of having a robust range of personalities stated above, with at least one extrovert per group (given that the assessment whole-class presentation is one task of each group) and ideally one judging personality (given that several stages of length-limited reports are required), one intuitive and one sensing personality (given that group discussion is needed, ranging from brainstorming activity to a requirements elicitation exercise) in addition to group work portion of the 2-stage examination. These diverse group members allowed the teaching team to demonstrate the different contributions that each team member could make at a certain point in the term-long group assessment. Furthermore, it is considered the group size that should be either 4 or 5 members to allow students to develop a sense of identity and ownership of work (Kerr & Bruun, 1983). The result of this was that group harmony was enhanced, and indeed played a key factor in the overall performance of most groups. It was planned to conduct the 2-stage examination in week-9 of the term. This assessment covered the four past study weeks' materials. The students assessed in an individual attempt first (closed-book) for 30 minutes. Students then will work with their group teams on the same set of the questions for another 25 minutes and hand in a group answer (open-book). This type of assessment helped students to engage in exam preparation and training at an early stage of the academic year i.e about 5 months before the final exam. Putting them under exam conditions is crucial in particular for the students of Higher National Diploma (HND) vocational entry qualification i.e less experience of exams. The students made aware of the details of the 2-stage exam in week-1 in addition to the other components of the assessment. Furthermore, two weeks before the exam, the lecturer reminded the students to get prepared for it. In week-8, written examination guidelines were published on the Moodle page of the module.

Analysis and Discussion of Findings
This section concerns with the analysis of the cohorts used in this research and the gathered data from students. Firstly, Students' academic performance in the 2-stage exam is captured in Figure-1. The average mark is 67.39% and the standard deviation is 16.70. Overall students' performance of 2019 new assessment is slightly higher to 2018 performance. In 2018, the average CO548 assessment mark was 67.9% and the standard deviation was 1.86 while in 2019 the average mark is 68.07% and the standard deviation is 1.22. This is an acceptable result taking in consideration changing the assessment from open book to closed book which is a bit of challenge to students and the two-stage examination was a new type of assessment implemented for the first time in the School of Computing. Figure-2 below provides CO548 overall assessment performance details of the two cohorts. In this study, a questionnaire is distributed to students to gather their opinions about the feedback session delivered in Week-10. Forty-six students participated in answering the questionnaire. As shown in Figure 3, 89% of students provided positive feedback while 5 students didn't find the feedback session very useful. We further discussed with few students and realised that they were not prepared for the exam and they scored low marks. One student reported that he prefers individual face-to-face feedback rather than a group feedback session. In week 23, i.e., during the final exam preparation period, twenty-four students volunteered to respond to the survey questions as shown below: Students' responses were detailed and justified. Most of them confirmed that this exam format motivated them to study and to engage with the module contents. A quite considerable number of responses refer to the fruitful discussions took place in the group work portion as they already answered the same questions in the individual portion of the exam. We observed that during the group discussion, students expressed reasons for their answers, which are based on their understanding of the questions. In the end, they either defended or abandoned their original answers. As a result, this discussion provided timely, individualized peer feedback that addressed students' misconception. Moreover, the face-to-face feedback session further condensed students' understanding of the concepts and also highlighted the shortages that the individual/group encountered during the exam. The participants comments are analysed using iterative coding to allow common themes to emerge from the collected data. Therefore, comments are classified according to the following categories: Assessment Format (AF), Active Learning (AL), Group Work Experience (GWE), Final Exam Preparation (FEP), and Using 2-stage exam in other Modules (UM). Feedback codes, description, count of occurrences, positive and negative comments are summarized in Table 4 below. A look at the students' feedback shows that a vast majority of students (90%) had a positive opinion of this exam format (expressed in 239 positive comments) whereas only 10% expressed the negative opinion in 12 negative comments. These results demonstrate students' high engagement with this format of assessment. Positive (119) comments related to collaborative learning (Active Learning and Group Work Experience) reflects the impact of this type of assessment on the overall students' learning experience.

Conclusions and Future Work
This paper describes the implementation of the 2-stage exam in the second-year undergraduate module entitled Software Engineering Process at Kent University. The aim of using the 2-stage exam was to promote collaborative learning, prepare for the final examination, and maintain good academic performance. The author drew on experiences gained from teaching and assessment design to show how the 2-stage exam can be applied in order to create a collaborative learning environment that reinforces knowledge and enhance exam preparation skills. MBTI-like test has been utilized for groups' formation which has proved successful as demonstrated by assessment performance and students' feedback. The learning experience was evaluated through students' performance of both cohorts and students' feedback on the 2-stgae examination. Both questionnaire (n=77) and survey written comments (n-24) have been used, as well as analysis of students' academic performance. The overall results appear to be promising. The vast majority of the participating students felt that the 2-stage exam is useful, interesting, and promote collaborative learning. For future work, we are in the process of incorporating the 2-stage examination in computer science module as a form of online assessment during these unprecedented times when we moved into blended learning, teaching and assessment. The 2-stage examination scenario could eliminate cases of collusion/plagiarism if no proctoring system is available.