Physics lecturing with audience paced feedback J. Poulis and C. Massen Department of Physics, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands   E. Robens Department of Chemistry, Gutenberg University, Mainz, Germany   M. Gilbert Department of Physics, University College London, Gower Street, London WCIE 6BT, United Kingdom (Received 18 November 1996; accepted 29 August 1997)   For some time we have experimented at the Eindhoven University of Technology with a system which provides students with an electronic feedback path to the lecturer, audience paced feedback (APF). In this paper we describe this APF system, and give indications of its effectiveness. 1998 American Association of Physics Teachers.   I. INTRODUCTION There is a considerable body of evidence supporting the view that traditional lecturing, where the students receive the lecturer's wisdom in full flow with only an occasional lapse into interaction, is flawed.' Attempts have been made in recent years to use technology in the form of computers and multimedia to improve upon the lecture, but, while clearly of growmg importance, these are also encountering constraints. In this paper we describe the use of a different technique to improve lectures.   II. OUTLINE OF THE SYSTEM Audience paced feedback (APF) is the provision, for each student in the lecture theatre, of an electronic handset to allow him/her to answer simple binary questions from the lecturer. A. Handsets On each handset is a single button, which is only pressed to indicate a "yes" response to an inquiry from the lecturer. This is for reasons of operational simplicity: it requires little effort to understand the system, there is no possibility of the wrong button being pressed, and it causes minimal disruption and distraction when used. The students' opinions are then transmitted to the central receiver along hardwired connections. B. Reception and display The responses are then collated and displayed as a simple percentage readout, visible to all participants in the lecture. The system is calibrated using a button on the central receiver; the lecturer asks all present to press their buttons and this number of replies is filed as 100%. In addition, the central unit also has a button to zero the display after each set of responses.   III. FACETS OF AN APF LECTURE The system introduces a more active role for the student in lectures, by giving them a simple feedback path to the lecturer, which should increase their cognitive engagement. This method of interaction is fundamentally different from the traditional "raise your hand to answer" style. The crucial differences are that: All students are involved in answering any given question, The lecturer can ask multiple choice questions allowing the formative nature of misconceptions to be explored, -the students' replies are anonymous (so they need have little fear of being identified and ridiculed by their peers), The procedure moves the format of the lecture away from the chalk and talk model and closer to that of a seminar. A. Question format Questions are asked at frequent intervals and the nature of the inquiry allows the polling of negative replies. The purpose of questions include: (a) exploration, to gauge the opinion of the students of their own understanding (for example, "Have you understood my arguments regarding this equation?");   (b) verification, to allow the lecturer to assess the state of the students' comprehension ("Does this apply to high temperatures?");   (c) interrogation, to test the ability of students to apply the work to specific situations (for example, a multiple choice question for which the student has to decide on the correct equation to use and hence find a solution; where each answer is presented in turn and the students press the button when their solution is presented   (d) organization ("Are you ready for me to continue?"). When a type c problem is presented the lecturer pauses and asks the students to press their buttons when they feel they have an answer (that is, he/she asks an interim type d question) and waits until 60-70% of the students have indicated that they do. This figure is a product of experience, and balances the need to give students time to answer and the boredom of those who have already completed the task. During this period students are free to discuss the problem among themselves. When enough are ready, the lecturer then presents each of the possible answers in turn and the proportion of those answering "yes" to each is seen (the normal method for answering questions of type c). Examination pass rate, %, and sample size, S  Year A Year B Year C Year D Year E Year F Subject  % ........S  % ........S  % ........S  % ........S  % ........S  % ........S Maxwell theory  38.....307  47......259  83.....269  87....230 .....  .....  Vibration and waves  ....  ....  87.....106 87.....122 93.....100 84....115  Optics and Maxwell theory  ....  ....  76.....165  65....180  56....134  57....190  Mechanics and kinetic theory of gasses  42.....72 60......84 93.....116  96.....112  ....  .....  Kirchhoff, Mechanics and vibrations  34......148  39.......124 95.....107 84......98  .....  .....  Energy management  .....  .....  82.....270  74....265  68......85  61....255  Statistical thermodynamic  .....  .....  98.....249  88......261  53.....390  54....578 Table I. The percentage pass rate and sample size around a period where APF was used. Years C and D are those in which APF has been used, shown in bold; years A, B, E, and F are the results of traditional lectures. B. Lecture format At the beginning of each lecture course the students are given a short talk on how the system works, to ensure a minimal level of confusion in its subsequent operation. At the start of a lecture, the students are given photocopies of the overhead projections (OHPs) to be used. These OHPs use a basic color coding scheme: green to signify that the material is being repeated or is expected prerequisite knowledge, red to show questions, blue for the possible answers, and black for new information. The lecture itself begins, as many do, with a recapitulation of what was said during the previous one. Questions are asked of the students to test the understanding and recall of the matter, with response via APF. Throughout the lecture, a difficulty in comprehension is signaled by there being more than 30% incorrect answers to a question of type a, b, or c; or if the time taken for students to signify their readiness is too long for a question of type d (the average wait is around 1 1/2 min, but this, of course, varies with the complexity of the question being posed) , When this occurs, the lecturer takes the students through the problem step by step and then asks a supplementary question (often on the blackboard) to check the new level of student understanding. If it becomes obvious that the students are completely comfortable with a topic,'for example by questions being answered very rapidly and correctly, the redundant questions are discarded. At the end of a lecture, the teacher asks "Who thought that was too fast/too slow/about right?" A desired distribution would be approximately 20% "too fast," 20% "too slow," and 60% "about right," as it is impossible to match everyone's ideal speed of presentation. Overall, a lecture consists of around 20 min of APF functions interspersed between 25 min of conventional lecturing.   IV. RESULTS It is difficult to measure the interest level of students in a lecture empirically, but there is some evidence that they prefer the use of APF. In addition, there are data showing that the use of the system improves examination results: while it is generally accepted that examinations are not a perfect measure of student comprehension, they are a reasonable indicator of understanding. These results are not immune to any Hawthorne effect with respect to the students-they enter a new exciting lecture format and there may be bias from this. However, the system has been in use in Eindhoven for a considerable time (since 1966), so the lecturers using it have become very familiar with the system, mitigating any effects on them. A. Opinions The 288 APF-exposed and 19 790 "ordinary" students were asked on a Likert scale from one to nine (nine indicating a very strong positive): "Do lectures contribute much/ little to a better understanding of the subject?" The mean score for non-APF students was 5.1; for APF students this rose to 6.7. This indicates a preference for the lecture when APF is present, and a positive reaction to it. B. Pass rate The end-of-course examination pass rate for a given course was measured over four years: either two non-APF years followed by two with APF; or two with feedback and the two subsequent traditional years. In each case the academic year of each of the four years' students remained the same. The data are drawn from the period 1979 to 1992, and cover the faculties of industrial engineering and management science, electrical engineering, chemical engineering and chemistry, and applied physics. To ensure consistency in the standard of understanding represented by an examination pass in APF courses during the periods considered, an independent supervisor from each faculty was appointed. It was felt vital that the course present the same amount of material, the supervisor be closely consulted when examinations were designed to ensure their consistency, and he/she be ultimately responsible for the courses' year-to-year equivalence. rate Table I shows that the use of APF improves the pass in most of the variety of physical science lectures in which it has been used. The mean pass rate, Fig. 1, of the APF lectures is significantly higher than that where conventional methods have been employed. Of equal importance is the reduction in the standard deviation of this average, indicating a more consistent level of comprehension throughout any given class, and year on year. This in turn means that decisions on the required level of understanding assumed for future courses can be made with more confidence. Fig. 1. The mean and standard deviation of the individual course-aggregated (data of Table 1) pass rate of 2550 students attending lectures with APF (shaded) and 2841 from traditional lectures.   V. CONCLUSION AND DISCUSSION These results demonstrate that the application of APF in the lecture theater has been of significant use in the students' learning process; both increasing the mean pass rate of individuals exposed to it and reducing the variability between the achievements of different students. These are being tentatively ascribed to four effects, which are, in order of decreasing importance: the removal of the "house of cards" effect, the negation of the inherent passivity of students in lectures, student-student teaching, and a mild Hawthorne-like effect. In a traditional lecture it can be extremely difficult to measure the students' comprehension of a topic. This can lead to a house of cards effect, where the lecturer is explaining a subject to students who have yet to understand its precursor. APF allows the lecturer to ensure that the majority of the student body has understood the material before moving on. In addition to this, the students are given a role in the lecture, and play an active part in it. This increases their cognitive engagement and so material taught to them is considered more closely. The students also spend some time discussing each problem; during this period there is some element of student-student teaching, or consolidation of material presented by the lecturer between students. Finally, there is likely to be a residual Hawthorne-like effect: the students are presented with a situation in which the lecturer has prepared a clear set of OHPs, and where they are given the special attention of the handsets. This cannot be completely discounted as an explanation until far larger longitudinal studies are undertaken. With the results of other independent studies involving student feedback also showing promise, further exploration of the efficacy of lecture feedback will be conducted, with the aim of improving this teaching form.   ACKNOWLEDGMENT The authors are grateful to Professor Sir Eric Ash for a number of helpful discussions. Corresponding author. For example, 1. A. Halloun and D. Hestenes, "The initial knowledge state of college physics students," Am. J. Phys. 53, 1043-1055 (1985); M. Jackson and M. T. Prosser, "Less Lecturing, More Learning," Stud. Higher Educ. 14 (1), 55 (1989); F. Costin, "Lecturing versus other methods of teaching: A Review of Research", Br. J. Educ. Technol. 3 (1), 4-31 (1972). Many are identified in Diana Laurillard, Rethinking University Teaching: A Frameworkfor the Effective Use of Educational Technology (Routledge, London, UK, 1993), pp. 120-178. For example, "Is this equation dimensionally incorrect?" At a trivial level, simply looking to see which students are facing the lecturer only measures that item, and not whether the student is listening. Studied in F. J. Roethlisberger and W. J. Dickson, Management and the Worker (Harvard U.P., Cambridge, MA, 1946), pp. 3-604; further discussed in H. M. Parsons "What happened at Hawthorne?" Science 183, 922-932 (1974); and G. Adair, "The Hawthorne effect: A reconsideration of the methodological artifact," J. Appl. Psych. 69 (2), 334-345 (1984). These data are drawn from the Teaching Evaluation Scheme at TUE. The standard deviations of these are not recorded. Before 1979 the data are no longer available, and after 1992 there are no relevant lecture variations. These data are independent of the data provided by Ref 6 (which was from a separate study regarding the entire university, hence the disparity in sample size). For example, E. Mazur, Peer instruction A Users Manual (Prentice-Hall, Upper Saddle River, NJ, 1997), pp. 3-42.    CLASSICAL MECHANICS Why do we still teach classical mechanics to graduate students in physics? It has been remarked...that when scholars in the humanities use the terms "classic" or "classical" they mean that something endures or is archetypical, but when physicists say something is classical, they mean it's wrong. Stephen Renolds, in a review of Florian Scheck, Mechanics: From Newton's Laws to Deterministic Chaos, American Scientist 80, 391-392(1992). Publications List