Annotation in the Open: Part 2

A few days ago, Robin DeRosa – open pedagogy advocate and superstar Hypothesis annotator – shared the following via #digped on Twitter:

I read Robin’s invitation as an opportunity for me to advance my own thinking – and continue my own writing – about INTE 5320 Games and Learning graduate students bloggers, annotators, and players who are working in the open this semester.

Some background: My approach to course design and pedagogy is influenced by an ecological approach to learning (read my related thoughts about designing a DS106 course last summer). I intend for a course, like Games and Learning, to create the conditions for learners to access new ideas and networks, to share information, and to generate knowledge across an ecology of multiple settings. Some of those settings are academic, while others are social; ideally, learning across those settings is connected. Learning, in the best of cases, spans a variety of everyday contexts, from classrooms to online blogs, from LMS platforms to social networks, from neighborhood encounters to interest-driven interactions. Accordingly, my approach to open course design and pedagogy extends public participation to connected and cross-setting agency.

This semester, our learning in INTE 5320 is shared at various public scales – from individually authored blog posts (see our blogroll, at right) to collectively networked conversations via Twitter (follow #ILT5320). We have also begun experimenting with Hypothesis, a tool for open web annotation. I’ll address Robin’s request by sharing some challenges related to public annotation in the open. These challenges are not concerned with the technical affordances of Hypothesis as a tool; rather, they are associated with open annotation as a practice. Over the past few weeks, I’ve begun blogging about our planned annotation-as-discussion, as well as students-as-readers’ initial annotation practices and the playful qualities of their open annotation.

Here are three concerns about working with annotation in the open, and the contingent solutions that have defined the first few months of Games and Learning.

1. Identification and ownership: Some learners were initially concerned about publicly identifying with their open annotation, and subsequently owning the ensuing discussion.

There are 12 graduate learners enrolled in INTE 5320 this semester. How are they publicly identifying via their chosen Hypothesis handle? The course has split equally in thirds. Four have adopted a handle that combines a name initial with either their first or last name; with a little digging, a savvy reader could discern the individual associated with the given Hypothesis handle. Four have chosen a handle that combines their first with last name (as have I, remiholden). And four have chosen a handle combining either a personally meaningful or entirely random assortment of letters and numbers; in these instances, it is nearly impossible to identify who is authoring the associated annotations.

I did not provide recommendations about how annotators should create their Hypothesis handle. Chosen handles indicate a range of preference – from assured anonymity to full identification. It appears as if personal identification (and, in some cases, concern) may be related to comfort owning learning in the open: the greater an individual’s concern, the more likely she may annotate anonymously; less, or no, concern often leads to more public ownership of annotation.

My contingent solution: Honor learners’ decisions as they embrace a range of identities to annotate in the open. While I know each of my annotator’s Hypothesis handles (for course administrative purposes), I must respect their desire for public work to sometimes mean anonymous expression.

2. Assessing annotation: Learners have been concerned about my assessment of their open annotation.

How frequently should readers annotate a course text? By what standards am I assessing the quality of either a single annotation or a collection of annotations? And might learners be assessed favorably if they annotate with more than text (for example, if they link to related resources, or embedded images or GIFs)? I am grateful that my class has raised these critical, honest, and necessary questions.

And no, I have not mandated a quantitative frequency for annotation – whether of a given text, or throughout a two-week reading cycle. And no, I did not create an a priori rubric to assess either a single annotation, or a reader’s annotation practice (and any rubric would invariably be co-constructed, like last year’s “crowdsourced” rubric for the Games and Learning affinity space project). And no, I had little expectation about the emergent semiotic qualities of annotation. The messier the media(tion), the merrier.

My decision not to formally assess learners’ participation in open web annotation is informed by own experience reading – and writing inside – books. When in college and graduate school, notes I made in a book’s margin most frequently served as a means to synthesize information or to express an opinion. Perhaps I shared annotations with a peer when studying for a quiz. Certainly I referenced my notes when writing a paper. In this respect, annotation was both a generative and a formative practice. No professor ever asked me to photocopy my annotations and submit them for approval. No professor ever required that I count and report a summary of highlighted lines of text to measure my comprehension. When the practice of annotation moves into the open – and becomes social and networked – should a formative and self-directed practice become a means for summative assessment? I think not.

My contingent solution: I encourage learner annotation as a practice that engages curiosity, pursues interest, and promotes experimentation – all without fear that this social practice will be quantified into a measure of some irrelevant objective.

3. Facilitating annotation: Learners have expressed concern about how best to turn open annotation into substantive discussion.

INTE 5320 adopted open annotation as a replacement for LMS-based threaded forum discussions. Not only is a single reader annotating a given text, readers are collectively discussing ideas, engaging questions, and sharing resources through their networked annotation. I previously wrote about the rationale for annotation-as-discussion as a shift that moves:

From the privacy – and primacy – of LMS (specifically Canvas) discussion forums to the public “playground” afforded by Hypothesis;
From the formality of pre-determined questions (which can privilege the scope and purpose of reading) to open-ended and less formal (re)action and exchange; and
From an instructor’s authority to center and control textual discourse to a de-centering of power through a fracturing of attention, interest, and commitment.

Our shift towards discussion as public, more open-ended, and de-centered has not, however, replaced the utility of active facilitation. I presumed readers would annotate text, but I was uncertain about the extent to which such annotation might remain isolated or disjointed. How, then, to design for more substantive annotation-as-discussion in service of shared critique or debate? Such dialogic annotation would likely require elements of planning, response, and encouraged collaboration. As our course began, I shared a set of generic annotation-as-discussion facilitator guidelines that – for better or worse – were largely modeled after LMS-based discussion expectations. Here they are, slightly edited:

  1. Annotate readings with thoughts, questions, highlights, confusions, and related resources.
  2. Present annotations that are both insightful and informal, and that invite others to contribute and respond.
  3. Ask follow-up questions during the back-and-forth of annotation.
  4. Reference complementary resources, recommended readings and media, and/or other experiences and insights that both deepen and broaden our collective engagement with course material.
  5. Respectfully challenge your peers’ lines of argumentation, helping us all to address blind spots in our logic or perspectives, to confront our biases, to check (if not also work against) our privileges, and to be a critic in the most encouraging sense.

During the course’s first two-week cycle, I facilitated our annotation activities to model these fairly traditional discussion practices; my annotations asked questions, shared opinions, established connections amongst texts and ideas, and prompted responses to key ideas and general themes. The second cycle featured our first pair of learner-facilitators. Kirklunsford and LisaDise (yes, those are public Hypothsis handles) capably adopted many of the practices I modeled; they successfully sustained a discussion around three rather complex texts (complex in further introducing sociocultural learning theory and ethnographic descriptions of game play). Our third cycle of annotation-as-discussion concludes today. A second pair of learner-facilitators – SusannahSimmons and Hoffmaca – are continuing these practices. They have also embedded five scavenger hunt-style clues among the texts (here’s Clue #1). Peer response to this playful layer has been positive. This is another indicator that playfulness may “appropriate” open annotation (something I’ll write more about in a forthcoming post).

It may not help mitigate some learners’ concern that I have – and will continue to – avoid articulating so-called “best practices” for facilitating open annotation-as-discussion at the graduate level. And by the way, Hypothesis – to their credit – has done a great job developing education resources, including these annotation tips for students. However, these tips are just different than naming practices for discussion through annotation – and particularly for graduate learners. We’re less than two months into our shared endeavor – why cramp people’s creativity? I’m committed to describing how certain qualities of annotation emerge and are socially negotiated (such as playfulness). But I’m not very interested in making definitive claims about the relative effectiveness of this or that facilitation strategy. Our annotation-as-discussion is improvisational; this open experiment is not intended to build a decontextualized method.

My contingent solution: I will continue supporting learner experimentation with varied approaches to annotation-as-discussion. Open annotation can spark fascinating expressions of conversation, from playful flash mobs to civic annotatathons. In our open work, I anticipate continued ambiguity, confusion, and even frustration as we (re)shape this mashup of more formal academic discussion facilitation with informal and emergent social annotation.



Playful Annotation in the Open

Note the Feb 21 update below, and read this page, at the dedicated URL and with Hypothesis, if you really want a playful learning experience.

Games and Learning turns one-month old today. Among many highlights from our first month, in this post I’ll discuss one of my growing curiosities – playful annotation in the open. And it looks something like this:


Curious about what’s happening here? Let me briefly sketch some context. First, INTE 5320 Games and Learning is an online graduate course at the University of Colorado Denver’s School of Education and Human Development. Second, the course is designed so that student learning occurs across networked and open settings and practices; from our use of Twitter (follow #ILT5320), to student blogs (and this blog, too, as our public home), to our use of Hypothesis (an open annotation platform that we use for annotation-as-discussion of course readings). And third, I’ve begun writing about students’ deep dive into the practices of open annotation. As an antidote to the (dying) discussion standards of online education, most students readers have responded rather favorably to Hypothesis. One student recently wrote to me:

This format [Hypothesis] is much better for me as far as encouraging participation. With the old discussion format that listed all the readings then posed questions for group discussion, I felt a bit overwhelmed by the long responses people offered and had a hard time jumping into the conversation. With, I can offer my thoughts as I go, which I find to be much more effective in my assimilation of the information.

Like writing in the margins of a book, I too appreciate how easily Hypothesis allows me to author and share “my thoughts as I go” – and to do so for a broader audience (anyone who installs the browser extension), and through a greater range of expressive representation (including text, hyperlinks, and embedded media). In this sense, open annotation is a means for readers to share spontaneous, messy, and sometimes humorous responses. Given these technical and social affordances, students’ open annotation is be(com)ing playful. As I observe student playfulness – and because I’m a course designer, games and play researcher, and learning scientist – I am now interested in the following question:

What are the playful qualities of learners’ open and socially networked annotation?

Which brings us back to this post’s opening image, a screenshot of open – and playful – annotation. Last cycle’s course readings about affinity spaces and participatory culture included Reed Stevens and colleagues’ (2008) ethnography “In-game, in-room, in-world: Reconnecting video game play to the rest of kids lives.” The ethnography describes how a small group of children and youth play video games together, and through eight vignettes examines “how video game play is tangled up in other parts of kids’ lives, including their relationships with siblings, parents, schools, and their own futures” (p. 44). The fifth vignette focuses upon cheating and players’ use of designed “cheats.” At the mention of “cheat codes,” one Games and Learning graduate student, bjauw, added a rather unusual annotation (also pictured above, with my own added visual emphasis). Again, bjauw’s annotation in full:

↑ ↑ ↓ ↓ ← → ← → B A start

Look familiar? bjauw’s annotation is the Konami Code. The Konami Code is both a cheat and also an Easter Egg. As an annotation, bjauw’s Konami echoes Stevens and colleagues’ argument that game play activities are connected across multiple settings. Game play both occurs within – and is simultaneously connected across – the designed reality and co-constructed narratives of a video game (“in-game”), the material confines of a room where bodies and conversation circulate (“in-room”), and also realms of personal interest and meaning-making (“in-world”).

So, too, does bjauw’s Konami annotation operate across settings and meanings. We might say it is “in-text, in-platform, and in-world.” This cheat-as-annotation circulates in conversation with the ideas of Stevens and colleagues’ ethnography (“in-text”), within the open annotations mediated by the Hypothesis platform (“in-platform”), and also amongst a discourse in gaming culture about the importance of cheats (“in-world”). Like the Konami Code in a game, bjauw’s annotation is intentional, context-sensitive, and subversive; it is literally and conceptually playful.

Of course, not every open annotation authored by graduate students learners in Games and Learning is so expressly playful. And not every open annotation, even in a course about games and play, should be so playful. I welcome a conversation about the extent to which (online) discussion of academic literature should be playful, and how best to create the conditions for such playful learning.

What I am beginning to appreciate about playful (online) learning is that open annotation appears to be a very promising practice, and that Hypothesis is a particularly deft tool. In working to articulate the potential for playfulness afforded by open annotation, I’ll borrow a few concluding thoughts from Miguel Sicart’s profoundly wonderful Play Matters. Among the book’s strengths, Sicart distinguishes the practice of play from the attitude of playfulness. He observes: “Playfulness is a physical, psychological, and emotional attitude towards things, people, and situations” (p. 21). Later he qualifies this disposition, adding: “Playfulness assumes one of the core attributes of play: appropriation. To be playful is to appropriate a context that is not created or intended for play” (p. 27). Through open annotation, students in Games and Learning are appropriating contexts not designed for play – including graduate education, online learning, and asynchronous text-based discussion. And in doing so, students are beginning to deeply cultivate an attitude of playfulness towards things (like Hypothesis, games, and digital media), people (including their peers and networks), and situations (namely their own interest-driven learning).

And it appears the playful adventures of open annotation are just beginning:

Feb 21 Update:

When life gives you an annotation flashmob… well, best help advance the discursive meaning-making.

Yesterday I awoke to the pleasant surprise of some very smart people – including tellio, jeremydean, nomadwarmachine, dogtrax, and onewheeljoe, among others (those are Hypothesis handles, which in most cases are Twitter handles, too) – exploding this blog post via Hypothesis through a meta-conversation that served as an act of annotation-as-play. In the 24 hours since, many others have added their annotations, too (and thank you for doing so!). And while this has been pretty awesome, a useful (that is, a readable) representation of this impromptu, playful annotation about/atop a blog analyzing playful annotation didn’t appear possible. Terry Elliot (@tellio) give it a try via Storify (see his Saturday Morning Feldgang), but as he noted:

OK, we have all of it here, but it is so damned opaque. It doesn’t tell a story, it doesn’t feel open, it doesn’t feel curated. WTF. I want to make some kind of statement here about the road we walked together this morning. Part of me says not to worry. It’s a process with intrinsic value. Another part of me wants to blaze the damned trail so others can follow if they wish.

And then along came Jon Udell. As Terry noted, there is a desire “to blaze the damned trail so others can follow if they wish.” But how to show this “field walk” to other readers – especially interested readers who are not using Hypothesis, but who may be curious about reading open annotation outside the platform? And if that seems counterintuitive (why not just jump in?!), well, that’s another conversation for another day.

Jon (judell/@judell), who currently works for Hypothesis, created a visual representation and meta-layer of conversation titled Annotations on Playful Annotation in the Open. And it shouldn’t come as a surprise that as of this morning, judell’s page now features various annotations-as-discussion, too. Here’s what annotation about annotation atop annotation is beginning to look like (and yes, I’ve added red text and arrows atop this image, yet another layer of annotation, as a further guide for readers new to Hypothesis):

annotation on annotation.jpg

A bit meta? Yes. Playful? Definitely. And as nomadwarmachine notes: “We can annotate the annotations, then annotate the annotations of the an notations, then …”

Happy trails.

Cycle 3 Readings

Since our course began a month ago, graduate students in ILT5320: Games and Learning have read and openly annotated game-based learning literature from leading scholars in the fields of digital media and learning, games studies and play, and situated learning (see our Readings for additional detail). These researchers and learning scientists have shared important insights with us; the following synthesizes a few key ideas:

  • Game-based learning is both situated – in social and cultural practices – and also ecological – that is, games and play spans settings and contexts.
  • Games and game play may be understood as socially accomplished, as influenced by cultural and expert practices (including those of formal academic disciplines, as well as pop culture), and as reliant upon contingent (and sometimes improvisational) configurations of tools, routines, and interactions.
  • The cultural expressions of games – including fan communities and affinity spaces, more generally – exhibit distinct patterns of participation, and these patterns contribute to individual and collective identity development and knowledge generation. Game play is one means of fostering a more participatory culture through networked media, people, and ways of being.
  • And any discussion of games and game play surfaces related conversations – and concerns – about equity. Who has access to networks and knowledge – and who doesn’t? How are people differentially supported in participating – or prohibited from participating? What skills and identities are developed – and honored – through play?

As we move through our course’s third two-week cycle we turn from the descriptive toward the dynamics of design. We have read authors who describe learning theories, the characteristics of affinity spaces, and media literacy skills. Now, we begin to read authors who are also concerned with design – from the design of games, to the design of meaningful learning experiences and environments through game creation and play, to the design of playful everyday experiences.

Some of our readings this cycle are expressly critical, with authors demonstrating a healthy skepticism about the ways in which games are designed, as well as the purpose for playing (and learning) in the first place. Consider the idea of gamification. Too frequently are points, leaderboards, levels, and boss challenges superficially slapped onto some experience. Moreover, should routine aspects of our everyday be “played” as if we’re all immersed in some big life-as-game? Posts by Bogost (@ibogost), and the highly recommended reading from Heller (@nathanheller), offer useful criticism. Alternatively, the article from Nicholson (@snicholson) and video lecture by Werbach (@kwerb) suggest ways that gamification can be more meaningful in the contexts of education and business. Moving away from the contested practice of gamification, learning scientists like Peppler (@DrPeppler) and Kafai (@katyaskit), as well as Games, identify diverse learning environments and practices that support youth creation of games as a means of learning specific skills and principles. It shouldn’t come as a surprise that we have much to learn about the design of game-based learning from those who research learning.

As always – here’s our usual reminder reading links, websites, and PDFs can be found on the course Readings page.

Cycle 2 Readings

As the first cycle of Games and Learning readings wrapped this past Sunday, I wrote at length about our course’s emerging and open annotation-as-discussion practices. I observed how certain types of annotation both transcend and also echo social practices found in more traditional LMS-based discussion forums. Just as annotation may support peer-to-peer questioning – a routine familiar to many who facilitate online learning – so too can annotation express the playfulness and personality of readers. I’m curious about how our annotation-as-discussion practices will continue to grow and change this cycle. What norms will subsist? What annotation practices will diminish? And what will emerge?

This post is less analytical and more administrative as I introduce readings for our course’s second cycle focused on affinity spaces and participatory culture. Students in Games and Learning will soon begin their semester-long affinity space project – an opportunity to join, observe, and participate in a games-related space of their choosing. This cycle’s readings provide a number of theoretical and empirical entry points for learning about affinity spaces, addressing questions like: What are affinity spaces? How is knowledge and expertise generated, and circulated, in affinity spaces? And where and how can students begin participating in the social and cultural practices of such spaces?


  • Gee & Hayes (2008) introduce the concept of affinity spaces within the context of game-based learning. They provide over a dozen features that distinguish how these spaces support learning, social cohesion and development, and the “nurturing” of expertise. Students will likely revisit this reading throughout the semester as they join an affinity space and analyze their own participation and learning.
  • Jenkins and colleagues’ (2006) seminal MacArthur report articulates what has become a theoretical pillar in the broad field of digital media and learning – the idea of participatory culture. Students are encouraged to read at least the report’s first half (from the “Executive Summary” through the section titled “What Should We Teach? Rethinking Literacy”), and to then focus on other sections based upon personal interest (for example, one or more of the “Core Media Literacy Skills”).
  • Stevens and colleagues (2008) present a highly-regarded ethnography of “everyday cognition” and social interaction, illustrating how children’s game play spans multiple settings and types of activity, with consequences for their divergent identity and skill development. The chapter is included in Salen’s (2008) edited volume The Ecology of Games.


  • The interview with Jenkins (2008) delves more deeply into fan communities and affinity spaces.
  • The piece from Squire (2006), and accompanying video, previews the importance of design within game-based learning, a topic that our course will explore in greater detail during Cycle 3.
  • And the short video from Gee attaches a personality to one of our course’s central authors.

Here’s another reminder that links to reading websites and PDFs can be found on the course Readings page.