To determine the effectiveness of video-technology in teaching general surgery operations to undergraduate medical students.
This questionnaire-based quasi-experimental study was conducted prospectively in the department of Surgery at the end of Surgery II course. The questionnaire comprised of items asking about demography, prior operating room experience, overall satisfaction and items based on principles of cognitive theory of multimedia learning.
Department of Surgery, Prince Sattam bin Abdulaziz University, Saudi Arabia.
Four consecutive batches of students, from 1st February 2014 to 30th April 2017. The inclusion criteria were all students attending Surgery II course. Incomplete responses were excluded. There were 125 students in the 4 batches, with 93 complete responses.
The response rate was 74.4% from sample size of 125. All students were male, and the mean age of the sample was 23.34 (± 0.155) years (ranges from 21-28 years). Forty-eight (51.6%) students have observed complete live surgery before in the operation theatre; 37 of these students were satisfied with video-based operative teachings, whereas 11 students were not satisfied. There were 45 (48.4%) students who have not observed operative room surgery; however, 29 of these students were satisfied, whereas 16 students were not satisfied. Overall, 66 (71%) students were satisfied (Group A), whereas 27 (29%) were not satisfied (Group B). The means of CTML based items (50.92) reflects effective delivery of video-sessions. Group A students showed significantly better performance in informal retention test (P = 0.026), whereas no significant difference was observed in MCQ exam (P = 0.374) and OSCE (P = 0.203) between group A and B students.
The video-based operative sessions are effective mode of teaching general surgery operation in resource limited setting.
Video-based, Technology-based, General surgery operations, CTML, Multimedia learning, Cognitive theory of multimedia learning
Dynamic images were used in clinical teaching to facilitate learning of operative procedures [1,2]. Multimedia learning implies learning by pictorial and verbal modes; verbal learning can be from written or spoken words, whereas pictorial learning can be from still or dynamic images e.g. video or animation [3,4]. This is explained by the following 3 principles of cognitive theory of multimedia learning (CTML) [4,5]:
1. The learners have separate processing channels for pictures and words.
2. They have limited capacity of working memory for information processing.
3. They need appropriate cognitive processing for meaningful learning to occur, e.g. paying attention, conceptual organization and integration with prior knowledge.
In CTML model for information processing and learning, the memory is divided into 3 parts: Sensory memory consisting of sensory replication of the presentation for a very brief period, working memory responsible for the processing of the presented material for a short period, and long-term memory representing the already stored knowledge for long periods [4]. CTML model also shows different cognitive processes that contributes in learning: Selection of words and pictures (images) represent transferring of presented material for further processing, organization of words and images represent coherent presentation, and integration represent cognitive association of presented material with relevant knowledge acquired before from long-term memory [4,5].
1. Extraneous cognitive processing must be reduced. It does not contribute to learning and is caused by poor instructional design. This can be achieved by eliminating extraneous material (coherence principle), highlighting essential material (signaling principle), not adding written material to spoken words (redundancy principle), and placing printed words near corresponding images (contiguity principle) [4,5].
2. Essential cognitive processing must not be reduced but managed in a manner that does not overload the cognitive capacity. It is required to represent the presented material in the working memory (i.e. selecting). This can be achieved by providing prior necessary knowledge (pre-training), allowing the learners to divide the lesson into segments (segmenting), and presenting words orally (modality) [4,5].
3. Generative cognitive processing needs to be promoted. It aimed at integration and organization of presented material, largely derived by learner's motivation [6,7]. This can be achieved by presenting images and words together (multimedia), presenting words as normal conversation (personalization), and using voice from human rather than machine (voice principle) [4,5].
Simple knowledge recall given in presented material is measured by Retention tests, whereas application of learned knowledge to new situations is measured by Transfer tests. Quality of learning outcome is determined by the results of these tests. Poor performance at both these tests indicates no learning, whereas good performance in retention test and poor performance in transfer test indicates rote learning. Good performance at both tests indicates useful learning [4,5].
The College of Medicine, Prince Sattam bin Abdulaziz University, is affiliated with King Khalid Hospital (KKH), Alkharj, for clinical subjects' teaching. General Surgery is divided into 4 courses; Surgery I is taught in Grade 4 (Semester II), Surgery II in Grade 5 (Semester II), Surgery III in Grade 6 (Semester I), and Surgery IV also in Grade 6 (Semester II). Surgery II comprises gastrointestinal tract, pancreato hepatobiliary tract, and breast and endocrine surgery. The teaching methods include lectures, bed-side/clinicals and tutorials. The operation theatre of KKH consists only 2 general surgery operating rooms (OR), which are small and can't accommodate more than 2 students at any time. There were about 30-40 students in each grade, and they were divided into 2-3 groups for bed-side/clinicals. In semester II, due to overcrowding of 3 batches, it is not possible to send students to OR. Also, there is no facility of audio-visual transmission from the OR. So, to provide operative exposure to the students, video-based operative sessions were started. Different operative videos were selected (from internet resources) based on lecture topics. These were played in the lecture hall, with running commentary provided by faculty and interspersed with questions and answers. Based on this background, this questionnaire study was designed to determine the effectiveness of this video-based teaching program.
A post-program-only quasi-experimental study design was used, as true comparison group and pre-program assessment are either not available or not possible. We divide the students into two groups based on prior operating room (OR) experience (group A = yes, group B = no). It was conducted prospectively, in the department of Surgery at Prince Sattam Bin Abdulaziz University (PSAU), Saudi Arabia from 1st February 2014 to 30th April 2017. The participants were 4 successive batches of grade 5 MBBS students, who were attending Surgery II course. The convenience sampling technique was used, wherein all students of grade 5 attending Surgery II course were asked to fill the questionnaires towards the end of second semester; this forms the inclusion criteria. Incomplete responses were excluded. The response rate was 74.4% from the students who were present on the day of data collection. The study sample comprises a total of 125 medical students.
Formal ethical approval was not required for questionnaire research that doesn't involve patients; however, the study was conducted according to the ethical standards of Helsinki Declaration (1964) and its later amendments. The questionnaires were distributed to each of the 4 batches at one time in the classroom, at the end of Surgery II course. Informed consent was obtained at that time, especially assuring them about the confidentiality of the data.
Video-based operative sessions in Surgery II course uses verbal and pictorial modes of learning in transferring knowledge, based on the principles of cognitive theory of multimedia learning. Care was taken in designing these sessions, so as to decrease the extraneous load, manage essential cognitive load and promote generative processes. The extraneous load was decreased by eliminating extraneous material, highlighting essential material, avoiding add on-screen text, and placing printed words near related images. The essential cognitive load was managed by providing pre-training, segmenting, and presenting words in spoken form. The generative processes were promoted by using multimedia, personalization and voice principles. The learning outcome was assessed by retention and transfer tests.
This includes a questionnaire and measurement of learning outcomes. The questionnaire consists of demography (age and sex), prior operating room experience, overall satisfaction with video-based sessions, & CTML based 13 items (Table 1). These 13 items were recorded on a 5-point Likert scale (where 1 = strongly disagree and 5 = strongly agree). The instrument aims to measure the extent of effective application of CTML. The overall score of the questionnaire (ranging from 13 to 65) gives a snapshot of the CTML successful application. Items 1-6 addresses extraneous processing score, items 7-10 essential cognitive processing score, and items 11-13 generative cognitive processing score. The items were designed such that a mean score of 13 would indicate complete failure, whereas mean score of 65 would indicate perfect implementation of CTML. The Cronbach's Alpha for CTML portion was 0.798. Content validity was assessed by 3 independent reviewers, who review each item in view of cognitive theory of multimedia learning.
Table 1: CTML based questionnaire. View Table 1
Informal retention tests (end of the session oral questions) measure how well the students remember the presented material; students were assessed by asking them questions like summarize the essential operative steps, name instruments used during the procedure, differences between different operative approaches, etc. Transfer test comprises MCQ and OSCE (taken as mid-term and final exams) to measure learners' application of knowledge to new situations. In OSCE 2-3 stations (out of 10-12) were based on operative sessions, while in MCQs about 7-10% questions were scenario based from operative sessions. Less than < 60% score indicates no learning, whereas score of 60% or more indicates useful learning.
The data was entered and analyzed on SPSS version 24. The variables noted & analyzed were student's demographic data, prior operating room experience, overall satisfaction and individual items designed according to the principles of CTML. The data was presented in numbers, percentages, and mode and mean scores. For statistical analysis students were divided into two groups, based on prior operating room (OR) experience, and the results of both groups compared. A p-value of < 0.05 indicates statistically significant result.
The response rate was 74.4% (93) students from sample 125 medical students. Four incomplete responses were excluded, whereas 28 students were absent at the time of data collection. All students were male, and their mean age was 23.34 years (SE ± 0.155 years, range 21-28 years).
Table 1 summarizes the results of CTML based items, with mean & mode values. The mean of total score is 50.92, which is about 78.38% of maximum (65). Table 2 summarizes one-sample statistics of CTML questionnaire. The overall score as well as group items scores are statistically significant (Table 1 and Table 2).
Table 2: One-Sample Test statistics of CTML questionnaire. View Table 2
Forty-eight (51.61%) students had previous experience of observing surgery in the operation theatre (Group A), whereas 45 (48.39%) students had no such exposure (Group B). However, 37 (77%) Group A students & 29 (64%) Group B students were satisfied with these video operating sessions (Table 3). However, this expression of satisfaction is only subjective (Table 3).
Table 3: Overall satisfaction vs. Prior OR experience. View Table 3
Table 4 and Table 5 shows the learning outcome results in relation to prior OR experience and overall satisfaction, respectively. The students who showed overall satisfaction with these video operative sessions had statistically better performance at informal retention test (p = 0.026) (Table 4 and Table 5).
Table 4: Learning outcome in 2 groups (prior OR experience vs. no experience). View Table 4
Table 5: Learning outcome vs. satisfaction with video sessions. View Table 5
Words and images acting together provides better mental understanding, with words contributing to theoretical explanations and images providing visual impressions, but to be effective it has to adhere to the learning principles [8,9]. We achieve 78.34% success in effective use of CTML. Yue, et al. reported a major review of multimedia animations in medical education applying the principles of CTML [2]. They analyzed 860 animations from 20 developers and found overall effectiveness of 45.55% [2]. Some reported examples of multimedia use in medical education includes demonstration of epithelia and their characteristics, osteogenesis in embryological life, and cleft lip and palate repair techniques [10,11]. Jang and Kim reported usefulness of multimedia learning in learning clinical skills and OSCE preparations [12]. Our Surgery II consists of 14 operative sessions in 14 weeks, and 3-4 videos were shown in each session. Comparatively, higher effective rate in our study was made possible by observing and practicing the strategies described below. However, we are disappointed that it was still quite below the full effectiveness. This was partly due to the fact that different faculty members were taking these sessions, with different level of understanding of CTML principles. The learning outcomes for each session were recognition of essential operative steps, identification and uses of different instruments, appreciation of different operative approaches and realization of different operative complications.
1. Coherence principle: We eliminate all possible extraneous materials including theoretical discussion about surgical diseases, review of surgical anatomy and real operative instruments [13].
2. Signalling principle: Mouse pointer is used to indicate a focused area in video.
3. Redundancy principle: No add on-screen text is used to supplement commentary [14].
4. Contiguity principle (spatial): Words labeling, if used, are placed near the images [15].
1. Pre-training principle: Corresponding lecture precedes the operative session, at-least 2-3 days before; surgical anatomy is also revised during lecture. This helped the learners by reducing the intrinsic load, thereby increasing their ability to efficiently construct mental models during these sessions [16].
2. Segmenting principle: 2-hour session is divided into 4 components, with 4 videos (50 minutes total) and intervening breaks. In addition, there were several pauses and start during each video, thus giving the learners, some controlling to pace these sessions [17,18].
3. Modality principle: Words are presented in the form of a running commentary, so as to efficiently use both verbal and visual processing channels [19].
1. Multimedia principle: Words and images are presented together in the form of video (image) and commentary (words).
2. Personalization principle: The commentary also includes interactive discussion by asking direct questions. The interactive environment keeps the students involved in the session with better mental model and understanding [17].
3. Voice principle: Running commentary is provided live by the faculty. Some authors have mentioned this principle as temporal contiguity [20].
All of our subjects were adult learners, but one study have reported multimedia learning in children, as well [21]. Most of the students who have the experience of visiting and observing cases in the OR, found these sessions more beneficial. They hardly see any skin-to-skin case in the OR because of the mismatch in their teaching schedule and OR list; sometimes, they saw the middle portion & at other times start or end of surgery. Further, in open surgery, they just stand behind the surgeon and have no visualization of the operative field. Also, they found surgeons, being more involved with the surgery, not addressing their queries. In contrast, here they see the cases skin-to-skin, and the faculty is describing surgery step-by-step, with immediate attention to any of their queries. Most students feel relaxed and comfortable in the classroom, as compared to OR environment where everyone seems to watch them (and keep saying, "don't touch this", "don't stand here", etc.). However, a few of the students, still feel it as an artificial environment and want real OR exposure. In view of these student's interests, we do encourage them to attend emergency OR in the evening.
The students who showed overall satisfaction performed statistically better in informal retention test (P = 0.026), partly explained by their interests and perceived advantages of these session. However, no significant difference was observed in MCQ exam and OSCE results between these students. The results of operative OSCE stations & MCQs generally match the overall performance of any given student. Good students tend to have meaningful results, whereas poor students have poor results. Issa, et al. found that applying multimedia design principles during medical lectures improves both long- and short-term retention and transfer tests performances [22].
The video-based operative sessions are effective mode of teaching general surgery operation in resource limited setting. It is effective in delivering knowledge irrespective of the prior operating exposure or overall satisfaction.
A major limitation of this paper is lack of proper comparison group. Other limitations of the study are small sample size and single institution, which limits the generalization of the results.
Faculty training is required to achieve greater effective of multimedia design principles.
There is no conflict of interests in this study.