For the sake of sharing, here’s a short version of the results of one of the microstudies I did this semester in research methods. At this point, this may be my Mayinar presentation, but, well, that’s still quite a ways away, and I have plenty of other possible things to draw on (and will have a few more by that time).
Preface 1: This is a chopped-up version of the formal write-up, I’ve cut some sections for length and revised others for voice; if you’re interested in some particular that I glossed over here, feel free to ask.
Preface 2: A lit. review is in the works. It’s mostly complete, but I’d like to flesh it out and revisit my results again before putting it here. \
Preface 3: Consider this your abstract: I studied the differences between how a group of students annotated a text on paper and how they annotated a text on their computers. As far as differences go, there weren’t that many (mostly because of a limited sample). In formal terms, I believe we could say “there was no significant difference between populations” (or something like that).
The university I teach at has a ubiquitous computing initiative that provides each incoming freshman with an HP tablet computer—a laptop with a screen that can be flipped around and laid over the keyboard, allowing students to interact with their computer using a stylus rather than mouse and keyboard. In enabling writing by hand directly on the computer screen, such tablet computers are arguably remediating, or refashioning in a digital format, the medium of pen and paper itself through the stylus and programs such as Microsoft OneNote and Windows Journal. Both programs allow users to handwrite—quickly switching between different pen and highlighter sizes at the click of a toolbar—and type, create hyperlinks, add clips from images and Web pages, and “print” images of documents on an infinite digital workspace.
So, my quasi-experimental study was designed to learn about the differences between using a traditional page and a digitally remediated page for invention—in this case, specifically the practice of text annotation, a commonplace in college classrooms. Even more specifically, my research questions:
- How do students’ annotation strategies differ on printed pages from remediated printed pages?
- Are students more likely to have a greater number of annotations on electronic or printed texts?
- Will handwriting-enabled computer programs such as Microsoft OneNote result in an increased variety of annotations than on a normal printed page?
The participants in this study were 21 undergraduate students, 11 females and 10 males, enrolled in one section of my Composition & Rhetoric course. Students in this course are typically freshman in their first semester of coursework, taking the course to fulfill general education requirements in writing; this section had 19 freshman out of 21. This sample was obviously drawn out of convenience, so generalizations are right out the window–however, as a pilot framework for running questions, operationalizing concepts, and planning for future versions of the study, this population is quite useful.
Briefly, my Methods:
Participants were asked to annotate two short articles, one of which would be the subject of a rhetorical analysis essay assignment; these annotated articles would then be collected at the same time as their completed essays. I collected hard copies of articles collected from 10 students and OneNote files from 5 students. 6 participants did not submit OneNote files (for assorted reasons).
I analyzed the texts, making note of the following features:
- Number of underlines, highlights, symbols, and written or typed comments on electronic and printed versions.
- Variety of interactions (different fonts/colors/sizes of typed comments; different pen or pencil colors/sizes; different highlighter colors; tags, images, hyperlinks, or other types of data) on electronic and printed versions.
- Number of annotations that go beyond the edges of the remediated printed page in OneNote, which places an image of the “printout” on a white background that is infinite in all directions.
- Number of annotations that go to the left or right of the frame of the monitor screen area (the default resolution for student computers is 1280 by 800 pixels).
All of the features in item #1 were operationalized as follows:
- underline: an isolated, complete unit of text with a line (including double/triple underlines) drawn beneath it; one unit may be under a single word, a whole phrase, a whole sentence, or a continuous underline across multiple lines of text.
- highlight: an isolated, complete unit of text colored by a highlighter, whether physical or electronic; same as above.
- symbol: includes numbers, stars, exclamation points or question marks (when independent of text comments), asterisks, or other graphic markers not otherwise defined as underlines, highlights, or comments; one unit may be a combination of symbols (such as “!?!” or a pair of parentheses or brackets), but ordinal numerals (for example, numbered points or paragraphs) should be counted individually; includes single letters (E, P, L, etc).
- comment: an isolated, complete alphabetic text statement; one unit may be a single word, a whole sentence, a list, a paragraph, or multiple paragraphs.
I had an outside observer (another colleague in the English department) help me out and examine a portion of the texts. Because the number of participants in this study is so small (15 students, 6 of whom did not annotate one or more articles), I chose not to use inferential statistics in my analysis.
Speaking of Analysis:
I won’t post up a bunch of data tables right now (again, I’d be happy to do so if you’re interested), and just skip to the big picture: The results of this study indicate little difference in how students’ annotation strategies differ on printed pages from remediated printed pages.
In terms of underlines, highlights, symbols, and comments, there were no telling differences between OneNote and printed copies. The only sizeable difference between OneNote and print scores here was between #Underline and #Highlight (means of 7.7 markings and 8.7 markings respectively). One colleague suggested there was little meaningful distinction between underlining and highlighting, other than the pen used. To explore this, a fifth variable was calculated, which simply added together Underlines and Highlights. The close means for OneNote texts (29.6) and Print texts (28.6) point out that without the pen distinction, the aforementioned “sizeable difference” disappears.
In terms of variety of interactions, OneNote texts showed a slight tendency for a greater variety (a mean of 4 types, compared to a print mean of 2.9–the OneNote texts used a few more colors of highlighters than print texts), but due to the low sample size, this is highly tentative.
Participants didn’t mark much beyond the “page” on their OneNote pages. Again, due to low sample size, there’s not much to say here.
As I mentioned before, the results of this study indicate little difference in how students’ annotation strategies differ on printed pages from remediated printed pages. Participants tended to underline and highlight a lot as compared to other kinds of markings—with slightly more underlining on printed texts and slightly more highlighting on OneNote texts. This may be a result of having easy access to multiple highlighter colors on OneNote. This point is supported by the mild increase in variety of annotations on OneNote texts. However, in terms of comments and other types of annotations, participants tended to do roughly the same things regardless of what type of text they were annotating. Only one student typed comments into their notes, and no students used any of the other capabilities of OneNote.
While lack of specific instruction in the OneNote software undoubtedly played a role here, the low variety of notes is almost certainly a result of students being asked to annotate for specific things in the interest of their rhetorical analysis assignment. That is, comments denoting a section as “logos” or “pathos” were relatively common across text types. Analyzing annotations for different kinds of reading and writing assignments could significantly change these results in future studies, as would analyzing texts by students from multiple instructors. Such a study could see strategies that go beyond the limitations presented by a particular instructor’s teaching and exposition style.
The few areas showing differences nearing significance—variety of annotations and number of annotations going “beyond the page”—do show potential for further study; variety of interactions seems to be a variable worth examining in the future. An exploration of the kinds of notes that typify annotations that go beyond page boundaries could also be worthwhile.
(Gee, that was a snoozer, wasn’t it?)