QEP Report

QEP Report

History 1305- Clicker Study 2007

David B. Ryden, Assistant Professor

NB: The following is for internal self-evaluation and none of the results should be distributed in light of the human subjects’ committ= ee policy.

For the spring semester o= f 2007, I incorporated “clicker” technology into the curriculum of a si= ngle History 1305 section. For the= last two thirds of the term, I conducted a series of unannounced quizzes, which = were graded and integrated into the students’ final marks. These nine quiz= zes were intended to encourage the students to keep up with their readings and = to study their lecture notes before attending each class.

I simultaneously taught a standard History 1305 section (without clickers) that the following analysis refers to as the “control” group. It would be a mistake to give too = much weight to these results given that that a two-class sample provides a very = thin basis for confident conclusions. I should also highlight that the “test” group (with clickers) started as a 100-student class tha= t had 3 supplemental instructors (SI’s), whereas the “control” group initially consisted of only 48 students and had no SI’s. While = we don’t want to overreach from these limited data, the quantitative res= ults are of interest and they confirm some of my qualitative observations.

The Turning Point System

The clicker software and = hardware package I used is called “Turning Point.” I did not have any special trainin= g and was able to figure out how to record the students’ input thanks to te= lephone conversations with the company’s representatives. They were very helpful and did not= keep me on hold very long. Having = said this, the program itself is a bit cumbersome to use and I do not find it at= all intuitive. I was surpri= sed by the length of time required to prepare a set of questions and by the number= of steps required to begin a quiz session.&nb= sp; There is also some complexity involved in properly integrating the student’s results into an electronic grade book, which I suspect coul= d be streamlined. With my version = of the program, the output data from each session (quiz) is dumped into separate X= cel files. The instructor must therefore merge the results by hand (cut and pas= te) into his/her grade book. Last, the output grade is expressed in an alpha fo= rmat with the “%” sign in each cell. The user must therefore (1) cut-and-paste the data; (2) delete all the percent signs; (3) convert the information from an alpha to a numeric format. I found this all to be a bit compl= icated and unnecessary. While I am t= otally unfamiliar with any other clicker product, it seems that there must be a be= tter system available. I suspect t= hat faculty will resist using this particular package for examination purposes.=

The other complaint I hav= e with the Turning point system is the proprietary policies with regards to their software. One should be able = to download the software for free without any registration codes or any hassle. Given that their busi= ness model is based on charging enormous prices for the actual clicker, the comp= any should be making it as easy as possible for faculty and user support to ins= tall the receivers and software. <= o:p>

Class Results

The clickers definitely facilitated discussion in the big lecture hall. Some of the best students were rea= dy to ask questions and make observations once the aggregate results were instantaneously displayed. The “on-the-fly” histo= grams showing the student’s responses also helped me identify what points I needed to address in my lecture.

The students in the ̶= 0;test” section were more likely to take the exams and to remain registered in the course. Table 1 shows nearly = 20 percent of the 48 students registered for the control section officially dropped the class, whereas the large section only saw nine percent official withdrawals.

While improved retention within the semester might be an indicator of succe= ss, having nearly one-fifth of the control group excluded is likely to account = for that class’s comparatively high test scores throughout the semester.<= span style=3D'mso-spacerun:yes'>

On the multiple choice po= rtion of the first three exams, the control group outperformed the test group. Table 2 shows, however, that = the averages are so close as to be statistically insignificant. Note that students who received zer= os because they did not take the exam are excluded in these calculations.

Turning to the essay port= ion of the exams, the test group’s average grade lagged behind the control group, with a statistically significant difference between the two groups’ first and third essay grades (figure 1). Even if we exclude student papers = that received a zero, there is still a statistically significant difference of f= ive points (a 75 average for the test group and a 79 average for the control).[1] Perhaps too much focus on multiple= choice style thinking distracted the students’ attention away from preparing= for essay-style examination.

Concluding Remarks

It is worth noting two observations in the grade distributions for the test group (table 3) and the control group (table 4). Firs= t, there were proportionally more D’s and F’s in the larger section. This might be a symptom of the ext= ra-ordinarily high proportion of control-group students who withdrew from the course. Second, it is worth noting that th= e test group proved to show a higher proportion of A’s and a respectable proportion of B’s. I noticed that these particular students showed a great deal of enthusiasm and seemed to make the most of the class discussions sparked by the interactive= quizzes. While the results of this first se= mester are ambiguous, I think the concept might prove a success with more experimentation and adjustment.