Composition Forum 22, Summer 2010
http://compositionforum.com/issue/22/
Broad, Bob, Linda Adler-Kassner, Barry Alford, Jane Detweiler, Heidi Estrem, Susanmarie Harrington, Maureen McBride, Eric Stalions, and Scott Weeden. Organic Writing Assessment: Dynamic Criteria Mapping in Action. Logan: Utah State UP, 2009. 167 pp.
Will this book’s argument offend? It certainly could, as the co-authors and especially Bob Broad present his Dynamic Criteria Mapping (DCM) process as preferable to the ways many people assess writing within their own colleges and universities. First introduced in Broad’s 2003 book, What We Really Value: Beyond Rubrics in Teaching and Assessing Writing, the DCM concept provides an alternative to traditional rubrics and scoring guides. From Broad’s perspective, despite the relative popularity of those documents, they fall alarmingly short of representing the actual beliefs people hold when they engage in discussions about writing and the teaching and assessing of it. DCM responds to the limitations of those assessment instruments, Broad explains, by providing “a method of inquiry into and negotiation of rhetorical values in classrooms and writing programs” (x). In What We Really Value, DCM is expressed as an assessment process informed by the work of those who value site-based, context-sensitive, qualitative inquiry. Since the publication of that book seven years ago, however, the arenas of writing assessment have become more charged than ever, often through loud cries for accountability, efficiency, and financial responsibility. The challenge of assessing writing can seem daunting to even the best-intentioned of faculty and administrators. Fortunately, after reading Organic Writing Assessment (2009), they may come away hopeful: the five DCM projects cover a wide breadth of approaches for developing local assessment initiatives. Riffing from a garden basket of earthy descriptions, the authors explicate DCM’s applicability to assessment projects in departments, programs, and general education settings, and to institutional varieties that range in size and kind. Through their attention to assessment work in several universities and a community college, the authors demonstrate the generative potential of DCM.
The chapters of Organic Writing Assessment will provide readers with argumentative grist for rejecting contemporary evaluation corporations that, as Broad asserts, are in the business of delivering “fast-food-style offers to make assessment faster and simpler by splitting it off from the rest of our work as educators” (Broad et al. 2). Broad and his co-contributors lead their readers through agricultural and ecological figures of thought to emphasize that just as people who aspire to healthy living must know and understand the complexity of food production, so must those involved with writing assessment know and acknowledge the richness of their institutional contexts. Readers of the book will witness explicit and implicit arguments for the value and legitimacy that results when educators themselves are the ones to articulate the many variables of their assessment projects. To varying degrees, the Organic writers also highlight pertinent assessment history and resources that could be helpful to readers interested in developing home-grown assessments. Broad’s introductory chapter deserves particular credit in this regard, as it skillfully contextualizes the efforts of composition specialists such as Richard Haswell, Brian Huot, and the contributors to Organic Writing Assessment. As the architect of DCM, Broad also invokes the work of assessment experts such as Egon G. Guba, Yvonna S. Lincoln, and Pamela Moss. This wider reach further reinforces Broad’s forceful call for faculty and administrators to assume responsibility for the writing assessment work affecting their students, colleagues, and institutions.
According to Broad, “we need to recall and listen to other voices urging us to re-capture, re-coup, and harness organic, localized assessment to nourish productive teaching and learning” (Broad et al. 2). Indeed, Broad and his colleagues have shaped a substantive argument for nourishing home-grown opportunities for assessment. Organic Writing Assessment actually functions less as a sequel to What We Really Value than an application and elaboration of Broad’s earlier ideas. Broad takes care to provide the necessary DCM background within the first chapter of Organic Writing Assessment. For that reason, it would be helpful but probably not essential to read Broad’s solo project before delving into the co-authored publication. Faculty teaching graduate seminars in writing studies would likely appreciate the addition of Organic Writing Assessment to the literature on writing assessment. So might others who study assessment theory and practice, and I believe the book will be of particular interest to writing program administrators and faculty, general education specialists, and writing across the curriculum consultants. Patient administrators could also benefit from considering the multiple examples of complex but thoughtfully designed approaches to DCM.
Though Organic Writing Assessment definitely functions as a stand-alone text, the work does not constitute an edited collection. It is not supposed to, as Broad himself refers to his colleagues whose scholarship appears in the book as “co-authors.” And no editor is listed, even though Broad’s Chapters 1 and 7 do function, at some levels, like introduction and conclusion, respectively. Even in those sections of the book, Broad exercises restraint. He avoids the potential pitfall of over-reminding readers that much of the work described in this book is an outgrowth of his ideas about assessment and specifically What We Really Value. Preferring to contextualize the multiple chapters of Organic Writing Assessment rather than enforce an editorial agenda, the closest Broad comes to shepherding occurs in a useful table in Chapter 7 in which he provides a “synthesis and summary of each of the five projects, including overlapping and harmonizing innovations, discoveries, and benefits achieved in each setting” (154).
The book’s conceptual design further supports the collaborative approach, most notably in the co-author responses integrated throughout each chapter. In most instances, these brief responses merit a reader’s attention because the writers’ commentary reflects their thinking about the content under consideration in the particular chapter at hand. In other instances, however, the examples seem less particular in their reinforcement and instead run the risk of drawing the reader’s focus away from the chapter and back to the wider conceptual frame of DCM. The writers likely reasoned that the testimonials would help readers understand the collaborative compositional strategy and confirm the coherence of Organic Writing Assessment. That makes sense, and often the additional voices do contribute to the immediate discussion. Regrettably, though, the document design of the book often privileges compression over breathing room. This can result in occasional moments of temporary confusion about whose voice is whose. The brief interludes, in those instances, disrupt the coherence of the chapter’s thought line.
Of greater concern is the relative lack of discussion about the aims of the collaborative approach itself. I found myself regretting that, apart from the testimonials, the contributors to this book did not seem to really offer much evidence of collaboration at the level of the overall project. Broad seems to carry more than his share of the responsibility for defining what he calls “the interplay among DCM projects” (Broad et al. 9), and though the DCM concept works well to unify those projects at a macro level, the specific interrelationships at times seem forced, as suggested in the comment above regarding the inserted comments. Other readers will likely appreciate the freedom of the collaborative approach, though, and find it welcoming to see a book that breaks form and embraces multiple voices working together. I would, too, if only the execution throughout the project sustained the high level of intellectual energy that emerged intermittently. Because of the diversity of projects represented—a positive attribute, in most circumstances—the co-authors share mostly large-scale themes and impulses and experiences. It seems the vitality of the collaboration might lessen if readers expect to find multiple layers of intertextuality. The inclusive design of DCM works across boundaries, and that accounts for a sincerely marvelous strength of the process, but the ways in which it works across boundaries could be a topic addressed in more depth in the next DCM publication. Aside from Broad’s two chapters, I found the chapters to range so far in scope that it was hard to reconcile the book as a tightly conceived project that coheres in interesting ways. Perhaps the very fact that DCM functions best in what Broad calls “home-grown, inductive assessment” (Broad et al. 6) accounts for the slight difficulty I had in appreciating the distinction between Organic Writing Assessment and, say, a genuine edited collection of essays on assessment projects that apply DCM to discreet institutional contexts.
Could readers have benefitted even more if the book had been conceived and produced as a full-scale edited collection under the direction of DCM architect Bob Broad? Perhaps, but Organic Writing Assessment exists as it does, I suspect, precisely because Broad and his colleagues are seeking to democratize the entire discussion about writing assessment, not just the individual projects they embark upon. Broad’s efforts to see this book to fruition reflect his overarching philosophy about working together in and among the necessary chaos of writing assessment. More than likely, he and his co-collaborators shared a vision for a book that modeled the sort of communal scholarship they advocate through their work with DCM.
Really, it may not matter how a reader characterizes the book, for whatever (all) it may be, Organic Writing Assessment certainly establishes a useful body of practical applications for DCM. And in Broad’s estimation, his co-contributors “have vividly and lovingly illustrated how much more flexible, adaptable, broadly applicable, and variable the DCM process can be” (Broad et al. 154) than what he had envisioned in What We Really Value. Having published a review of that earlier publication, I heartily agree that the DCM concept has been enhanced and matured by the later work. The labor and critical thinking of the co-authors embody a remarkable statement about the benefits of working together to engage in fair and meaningful assessment. Returning now to my original question of whether this new book’s argument will offend might seem as if I am anxious to create a tension between the contributors to Organic Writing Assessment and those who do not follow the sort of rich and often alternative approaches informed by DCM. If such a tension exists, I believe the writers have created it, and I would give them credit and respect for doing so. They have exhibited the courage to move into the intricacies of their work with patience, confidence, and intellectual curiosity. And despite my reservations about the effectiveness of Organic Writing Assessment as a co-authored project, the book offers readers a wide variety of perspectives on and approaches to writing assessment that are well worth reading. I admire the writers’ zeal for DCM and their interesting projects inspired by Bob Broad’s thinking. Organic Writing Assessment effectively follows up What We Really Value, but it also complements a range of other recent publications that acknowledge the incredible bounty that can come from home-grown writing assessment.
Works Cited
Broad, Bob. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing. Logan: Utah State UP, 2003. Print.
Eliason, John. Rev. of What We Really Value: Beyond Rubrics in Teaching and Assessing Writing, by Bob Broad. Composition Studies Online. 32.2 (2004): n. pag. Web. 10 May 2010. <http://www.compositionstudies.tcu.edu/bookreviews/online/32-2/eliason.html>.
Review of Broad et al., ORGANIC WRITING ASSESSMENT from Composition Forum 22 (Summer 2010)
Online at: http://compositionforum.com/issue/22/organic-writing-assessment-review.php
© Copyright 2010 John Eliason.
Licensed under a Creative Commons Attribution-Share Alike License.
Return to Composition Forum 22 table of contents.