Student Satisfaction Survey Results

In the Winter term 2016, I taught the University of Calgary’s second logic course from a textbook remixed from the Open Logic Project.  Traditionally, Logic II has used Boolos, Burgess & Jeffrey’s Computability and Logic, and it was taught in Fall 2015 using that book as the required text by my colleague Ali Kazmi, and before that by him, Nicole, and me twice a year from that same book.  One aim Nicole and I had specifically for the OLP was that it should provide a better text for Logic II, since neither we nor our students seemed to be very happy with “BBJ”.

In order to ascertain that the OLP-derived text fares better with students, we did something radical: we asked them what they thought of it.  Ali graciously gave permission to run the same textbook survey in his class, so we have something of a baseline.  A direct comparison of the two books as textbooks for the course is not easily made, since Ali and I used the books differently: I stuck closer to my text than he did to BBJ; I assigned homework problems from the text; and we assessed students differently, so it’s difficult to control for or compare teaching outcomes.  With small samples like ours the results are probably also not statistically significant. But the results are nevertheless interesting, I think, and also gratifying.

We obtained clearance from the Conjoint Faculties Research Ethics Board for the study.  All students in each section of Logic II in F15 and W16 were sent links to an electronic survey.  As an incentive to participate, one respondent from each group was selected to receive a $100 gift certificate to the University of Calgary bookstore. The surveys were started in the last week of classes and remained open for 3 weeks each.  Response rates were comparable (23/43 in F15, 23/42 in W16). The survey was anonymous and developed with the help of the Taylor Institute for Teaching and Learning, who also administered the survey; results were not given to us until past the grade appeal deadline in W16.

We asked 23 questions.  The first three regarded how students accessed and used the textbooks. In the F15 section, the textbook was not made available electronically, but students were expected to buy their own copy (about $40).  Most respondents did that, although almost a quarter apparently pirated electronic copies.  In W16, the OLP-derived text was available for free in PDF and students had the option to buy a print copy at $10. Over half the respondents still opted to buy a copy.  We asked students how they used the texts in hardcopy and electronic form.

When using the text in hardcopy, do you...

Those using the OLP-derived printed text underlined significantly less than those who used BBJ. I’m guessing the OLP text is better structured and so it’s not as necessary to provide structure & emphasis yourself by underlining. In fact, one student commented on BBJ as follows: “Very little in the way of highlighting, underlining, or separating the information. It was often just walls of text broken up by the occasional diagram.”

When using the text in electronic form, do you...

When using the electronic version (both PDF), students did not differ much in their habits between F15 and W16. More students took notes electronically in F15. I suspect it’s because the PDF provided in W16 was optimized for screen reading, with narrow margins, and so there was little space for PDF sticky notes as compared with a PDF of the print book in F15. Also notable: highlighting and bookmarking is not very common among users of the PDF.

The second set of questions concerned the frequency with which students consulted the textbook, generally and for specific purposes.  W16 students used the OLP-derived text significantly more often than F15 students did, and for all purposes.

How often do you consult the text?

The difference is especially striking for the questions about how often students consult the textbook for exams and homework assignments:

Do you read the text in preparation for exams?
Do you consult the text when working on homework problems?

We next asked a series of questions about the quality of the texts. These questions were derived from the “Textbook Assessment and Usage Scale” by Regan Gurung and Ryan Martin. On all but one of these questions, the OLP-derived text scored positive (4 or 5 on a 5-point Likert scale) from over half the respondents. The discrepancy to students’ opinions of BBJ is starkest in the overall evaluations:

How engaging/interesting is the writing?
How understandable/clear is the writing?

The one exception was the question “How well are examples used to explain the material?”:

How well are examples used to explain the material?

This agrees with what we’ve heard in individual feedback: more, better examples!

Lastly, we were interested in how students think of the prices of textbooks for Logic II. We asked them how much they’d be willing to spend, how much the price influenced their decision to buy it. Interestingly, students seemed more willing to spend money on a textbook in the section (W16) in which they liked the textbook better. They also thought a free/cheap textbook was better value for money than the commercial textbook.

Is the price of the textbook too high for the amount of learning support it provides?

We also asked demographic data. Respondents from both sections were similar: almost all men in each (the course is mainly taken by Computer Science and Philosophy majors), evenly divided among 2nd, 3rd, 4th year students plus a couple of grad students in each (Logic II is required for the Philosophy PhD program). Student in W16 expected higher grades than those in F15, but that may well be just an effect of differences in assessment and grading style rather than better student performance.

If you care, there’s an interactive dashboard with all the graphs, and the raw data.

A Few Photos More

I added a few more logician’s photos: Carnap, Herbrand, Kalmar, Lewis, Kleene, Montague, Quine, Wang.

See previous post on how to download/integrate them into your OLP directory.

For the Last Day of Women’s History Month: Biographies of Rózsa Péter and Julia Robinson

I’ve added Rózsa Péter and Julia Bowman Robinson to the biographies chapter, and a high-quality photo of Péter to the photos repository. Thanks to Samara for drafting these sections.

Vote for your Favorite Logician

Suppose we got more illustrations of logicians. Who should we get?

Guess the Logician!

We’ve been working with Calgary graphic designer and illustrator Matt Leadbeater on a series of stylized portraits of logicians. They will be licensed under a Creative Commons Attribution-NonCommercial license (free to use for teaching, but you can’t sell coffee mugs with them on it), and they will be included in the Open Logic Text repository.

Before we make them all available, we’re playing a little game. They’ll be posted without names on Facebook, Twitter, and Google+, and we’ll let you guess who they are!

Props to the Alberta OER initiative for making this happen by providing funding. Making open textbooks visually attractive is an important but overlooked aspect of the OER production process.

 

Logicians’ Biographies

Thanks mainly to Samara‘s efforts, the Open Logic Project has begun to include short biographies of logicians in a new History part (Source on GitHhub, PDF).  So far we have Cantor, Church, Gentzen, Gödel, Noether, Russell, Tarski, Turing, and Zermelo.  We’ve made an effort to appeal to our target audience — undergrad students from a variety of backgrounds — and kept them short and as non-boring as possible.  References are included not just to academic biographies, but also to YouTube videos and podcasts.  As always, feedback is more than welcome, and we take requests for additions!

In order to enable these biographies to contain references, all driver LaTeX files now automatically generate and include a bibliography.  The bibliographies so far are the only texts that contain references (in Natbib author-year format), but they can now be included in other texts as well. Two environments, ‘history’ and ‘reading’ for Historical Remarks and Further Reading sections are now available (defined in open-logic-envs.sty) and material using these will be added to the main text as time goes on. We’re also very close to including portraits!  To keep the main repository to a reasonable size, these will be provided in a separate GitHub repository.

[Photo credits: Alan Turing / National Portrait Gallery CC-BY-NC-ND; Georg Cantor / University of Halle Archives; Kurt Gödel / Institute for Advanced Study Archives; Emmy Noether / Göttingen University Library]

The Logic Gallery now CC-Licensed

David Marans has now licensed his Logic Gallery, a collection of tid-bits of information, thought-provoking quotes, and pictures of figures from the history of logic, as Creative Commons Attribution (CC-BY). Download it as PDF or buy a print copy.

You can also find Howard Pospesel and David Marans’s Arguments: Deductive Logic Exercises (1978), the rights of which have reverted to the authors, available free & online.

University of Calgary Student Paper Profiles OLP

Nicole Wyatt and I were interviewed (and photographed) for an article on open textbooks in the University of Calgary’s student newspaper, The Gauntlet, last month.

[Photo © Louie Villanueva/Gauntlet]

Repository Reorganization

As per issue #48, in order to keep the clutter in the main directory of the repository to a minimum, I’ve moved all the content directories containing texts to a subdirectory content/.  If you have a driver file that includes individual texts (part, chapter, or section) using \olimport, you will have to add “content/” to the optional directory argument.  If you use the starred version \olimport*, you shouldn’t have to do anything, as \olimport* now interprets paths relative to \olpath/content/. In other words, if your file contained the line

\olimport*[first-order-logic]{completeness}

you don’t have to do anything, but if you had, say,

\olimport[/home/zach/openlogic/first-order-logic]{completeness}

you’ll have to change that to

\olimport[/home/zach/openlogic/content/first-order-logic]{completeness}

If you have added your own files to the content directories or have started a new part (really? amazing! let us know!) then the first line of those new files will have to be changed by adding an extra ‘../‘ to the optional argument of the \documentclass command, e.g.,

\documentclass[../../../include/open-logic-section]{subfiles}

Open Textbooks and Reclaiming Rights

I wrote a post about open philosophy (& logic) textbooks over at my blog, which Daily Nous linked to. The comments there also have interesting & useful information on how to get your copyright back from the publisher of your published textbook.