Composition Forum 46, Spring 2021
http://compositionforum.com/issue/46/

Revising Reflection for Results in Teacher Research

Adrienne Jankens and Amy Ann Latawiec

Abstract: In this article, we argue that using students’ reflective writing to understand specific aspects of their classroom experience requires that researchers systematically integrate into the curriculum reflections that responsibly attend to both students’ learning and the focus of classroom research. Informed by recently published articles on reflection and collaborative writing and learning, this argument contributes to recent Composition Forum discussions (e.g. VanKooten; Fiscus; Winzenreid et al.; Jankens, Learning How to Ask). We also aim to demonstrate that the process of learning how to do research “right” is a recursive endeavor. Addressing the challenging results of our study, we consider ways that more systematic reflection in our paired courses might have brought collaborative learning even more to the surface both for students and our research. We pose that, in retrospect, had we developed a series of reflective writing assignments that explicitly prompted students to describe learning as part of a social process within the classroom, this reflective writing would likely have better highlighted collaboration for our research and for students’ meta-awareness of their learning processes.

“And, if things do not go just as you hoped in your research, then you will likely need to go back and plan certain aspects of it all over again.”
—Ann Blakeslee and Cathy Fleischer, Becoming a Writing Researcher, p. 75.

Introduction

For a long while, we considered how best to present the research story we tell here. In 2014, we set out to study how we could understand students’ experiences with collaboration through the reflective writing they did in a basic writing (BW) class and a first-year writing (FYW) class that we taught at the same time, in adjacent classrooms. We designed the classes as opportunities for students to 1) study writing and learning spaces (and literature about such spaces) and 2) meet our general education learning outcomes of practicing reading, writing, analysis, research, and reflection. To emphasize the social processes of learning and composing, we strategically brought students from both classes together throughout the semester to discuss readings, drafts, and proposals. As a service-centered outcome of this work, we compiled a collection of students’ researched proposals for redesigning these two classrooms to better serve future students and instructors, and we shared this collection with institutional stakeholders (including other composition instructors and our Director of Composition). Two additional goals were layered onto these teaching efforts: our research goal of understanding how students experienced these collaborative learning moments as preparation for future learning and a long-term program goal of constructing peer mentoring opportunities for BW and FYW students. One of our central research questions, and the one we ultimately devoted our writing energy to was: how do reflective writing assignments designed to make students think about course learning objectives help them understand the ways in which they find collaboration to be an integral part of achieving the course learning objectives?

What drives this version of the story of our research is the lesson we learned about how to work with reflective writing for classroom studies. Through an IRB-approved study of students’ learning in these classes, we hoped to understand students’ experiences with collaboration through their reflective writing, especially their end-of-semester reflective essays, and we hoped that students would use their reflective writing to articulate an understanding of collaborative learning. We carefully planned the study, crafting focused research questions, selecting an appropriate research site, and matching data collection strategies to our research questions. However, we found that our study yielded challenging results: when students referenced peer and teacher feedback and influence in their end-of-semester reflective essays, they did not meaningfully or explicitly identify collaborative knowledge-making. Instead, they typically referenced other classroom moments or mentioned other students, but not in a way that demonstrated awareness of collaboration—a learning strategy they used often in class. In this article, we explore this result and its most pressing implication: how best to study a topic through reflective texts.

To position our description of our study and analysis, we review literature in reflection and collaborative writing and learning. We describe our instructional strategies in our BW and FYW courses and phases of analysis after the courses were completed. Then, we consider ways that more systematic reflection throughout our paired courses may have brought collaborative learning even more to the surface both for students and our research. We explain how, in retrospect, we see that had we developed a series of reflective writing assignments that explicitly prompted students to describe learning as part of a social process within the classroom, this reflective writing would likely have better highlighted collaboration for our research and for students’ meta-awareness of their learning processes. We argue based on this experience, and in light of recently published articles on reflection and the construction of writing knowledge (e.g. VanKooten; Fiscus; Winzenreid et al.; Jankens, “Learning How to Ask”), that using students’ reflective writing to understand specific aspects of their classroom experience requires that researchers systematically integrate into the curriculum reflections that responsibly attend to both students’ learning and the focus of classroom research. We also emphasize that the process of learning how to do research “right” is a recursive endeavor.

We hope that teacher researchers reading our contribution to scholarship on teaching and studying reflection will consider their own plans to pilot and engage in research on student learning in the writing classroom in light of our experience. Further, we hope that our work contributes meaningfully to forwarding the work of the recent Composition Forum publications cited above, which all seem to gesture or point confidently toward the same pedagogical and research practice: using systematic reflection to support students’ developing writing knowledge and to study that learning.

Literature Review

At their conception, the study and classroom work we describe here responded to an array of discussions in composition pedagogy and writing assessment, including conversations about knowledge transfer, service-learning, peer mentoring, and teaching and learning in BW and FYW classrooms. In this paper, however, we work to explore specifically how a significant learning tool in our classes—reflective writing—functioned as a research instrument for understanding learning strategies and classroom behaviors outside of the course learning outcomes. Here, we present a brief literature review to both position our work and leave space to tell the story of our data analysis. We return in the conclusion to examine how our work, in the company of several other recently published studies, presents an example of why writing teachers need responsibly integrated reflective writing as data for understanding students’ developing writing knowledge.

In designing our courses, we were particularly interested in drawing on the social elements of the classroom to support students’ engagement and learning. For both FYW and BW students, the collaborative classroom can impact decision- and knowledge-making (Shor; Bruffee; Wallace and Ewald) and can help them develop an “interdependence” and accountability (Bruffee xiii; Shor 125). And, because students may be resistant to learning from each other (Bruffee 14-15; Keating 64), the discursive structure of the classroom impacts the degree to which collaboration is successful (Wallace and Ewald), including the ways that teachers convey messages about why they value collaborative opportunities like peer review (Keating 61). Teachers need to consider how best to integrate dialogic and written spaces in which students can engage in consensus (Bruffee), examine difference (Trimbur; Shor; Leverenz; Higgins et al.), have “dialogic contact with a diversity of experience” (Keating 69), and engage opportunities for coauthoring (Eodice and Day 191-192). Lorraine Higgins et al. note the importance in community-based projects, like ours, that provide participants with resources to help them think about the perspectives of those who, though not present, have a stake in the outcomes of the project. Misty Anne Winzenried et al. examine students’ conversations in a first-year writing and theater cohort, learning that students build “domain-specific knowledge” together through “co-telling” and “co-constructing.” This scholarship highlights the ways that classroom talk and collaboration in various forms is integral for knowledge-building, development of new perspectives, and new ways of presenting ideas in writing (Duffy 422).

Often, the writing knowledge developed through learning strategies like collaboration is captured in students’ reflective writing. Kathleen Blake Yancey’s description of the ways reflective writing can be integrated into writing classrooms has influenced our work with students for over two decades, and more recent scholarship demonstrates the different ways that reflection-in-action, constructive reflection, and reflection-in-presentation manifest in writing classrooms (e.g. VanKooten; Fiscus; Jankens, Learning How to Ask). Important in each of these studies is a conception of reflective writing that is “systematic” (Yancey et al.). This systematic reflection may be a series of reflections focused on different aspects of the course that, together, create a whole picture (see, for example, Jankens, Responsive Classroom Ecologies; Guinot Varty) or it may be a series of reflections that ask students to iteratively articulate their developing knowledge about writing (e.g. Ash and Clayton; Sommers, Reflection Revisited; Yancey et al.). Reflective writing is an important part of students’ self-assessment of writing practices, but it does not come easily to all students (Taczak), so space and time must be scaffolded for students to see this writing demonstrated, to practice it, and even to receive feedback that improves the opportunity for constructive metacognition (Allan and Driscoll; Sommers, Problematizing Reflection; Trimble and Jankens). In all, our collective scholarship about reflection tells us that reflective writing is a powerful tool for learning if it is done purposefully and reiteratively.

As ubiquitous as studying reflective writing may feel to those of us who use it both to assess our classrooms and to develop larger theories about teaching and learning, we still find ourselves, as a field, in the process of articulating how reflective writing works as a way that both students and researchers learn about learning and develop writing knowledge (in A Rhetoric of Reflection, Yancey calls this a “third generation” of reflection (9)). Understanding how to do this well is important to us, in part, because reflective writing helps us access students’ meta-awareness of writing knowledge; the use of reflective writing as a research tool can help us conduct larger studies of this writing knowledge by providing us with snapshots of meta-awareness across multiple student samples. Identifying four ways that meta-awareness manifests in students’ reflective writing (through discussion of process, techniques, rhetoric, and intercomparativity), Crystal VanKooten calls for “a more robust theorization of meta-awareness of composition.” Our collective qualitative studies are working to help us build an understanding of that meta-awareness and how we can support students’ development of writing knowledge. For example, in Writing Across Contexts, Yancey et al. describe a reframed purpose for writing reflection: not to arrive at and perform an end-of-term reflective assignment, but to support the development of a theory of writing (4). Members of the Writing Transfer Project find that a nuanced understanding of genre, with an awareness of how conventions relate to a genre’s audience and purpose, predicts writing growth in general education writing courses (Driscoll et al.) and argue that a meta-awareness of their own cognitive and metacognitive strategies may help students to grow as writers (Gorzelsky et al.). As we continue to build that knowledge, our collective theory/theories of using reflective writing to understand students’ meta-awareness of writing knowledge will be as robust as our present use of reflective writing in the classroom.

The Story of our Study

In this section, we use the story of our study to highlight our teaching and research goals in those Winter and Fall 2014 semesters, as well as to share our struggles with analyzing students’ reflective writing to learn about how collaboration impacted their learning. While we take only a few pages here to describe this research journey, our story spans several years. Framing this teacher research study as a story allows us to demonstrate how we used planning, teaching, analysis, writing, and re-writing to come to our conclusions about how best to use reflective writing for classroom research. In short, as we describe a study based in students’ reflections, we also provide a window into our own reflective inquiry in practice.

Collaboration Between a BW Class and an FYW Class

In the Winter and Fall 2014 semesters, students in one section each of our university’s BW and FYW courses worked together to research effective writing and learning spaces and to propose aesthetic and functional changes to two computer classrooms regularly used for teaching composition courses. We asked students to explore three research questions throughout the semester: 1) What is a positive, rich learning environment? 2) How does environment impact learning? 3) What can we do to enhance our writing/learning environment? The computer classrooms our classes were held in were next to each other in an older building on campus that houses most of our program’s general education composition courses (and all three of our writing program-dedicated computer labs). With analytical writing as a key learning outcome of Amy’s BW class, students learned how to and then performed observational analyses of learning spaces of their own choosing (on and off campus). Because research is a significant learning outcome of the FYW course, students in Adrienne’s class conducted secondary research on learning spaces to craft their proposal arguments for changes to these classrooms. In the collaborative class sessions that we held four times each semester, students from both BW and FYW reviewed one another’s work with a particular focus on whether and how the BW students’ analyses would be useful for the FYW research projects. During collaborative class sessions, we used fishbowl activities and peer response to facilitate BW and FYW students’ discussions of their analyses and research. In our individual classes, we worked with students on meeting all other course learning outcomes distinct to each course. As we describe below, the only assignment the classes had in common was the final reflective argument essay, because it was the artifact our composition program used for assessment.

Our goals in facilitating collaboration between our classes was to support BW students in traversing a more novice-expert line as they moved from BW into FYW, and to allow students in both classes, through the composition classroom expertise they were developing, to understand what content, discourse community, and writing knowledge they had to contribute to each other{1}. As described above, this project relied on teaching and learning strategies derived from literature on collaboration and reflection and aimed to respond to efforts in the field to “teach for transfer,” a central goal of our program’s composition courses. This was done, in part, by prompting students’ motivation to “[contribute] something to others” (Bransford et al.) and by helping them gain a sense of cross-sequence coherence, which we hoped would improve students’ perceptions of the value and purpose of work completed in the writing courses.

Reflective Assignments in the Courses

While the dialogic space of the composition classroom can be designed to support student learning, like we did through class discussions and peer feedback, writing assignments also need to support this work. We integrated written reflection assignments to support the development of students’ learning and writing strategies and to help them think about how to recontextualize prior knowledge for new writing assignments (Nowacek; Yancey et al.). We also used these assignments to help students articulate their personal goals, motivations, learning objectives, and self-assessment of their learning (Yancey, Reflection; Ash and Clayton; Latawiec). Finally, Amy designed a reflection assignment in the BW course to allow students to imagine how skills learned in BW would be useful in FYW. We hoped this work would make the projects, processes, and learning objectives of both groups more visible.

For reflection to be useful for learning, it needed to be scaffolded thoughtfully throughout the courses (Allan and Driscoll; Ash and Clayton). Sarah L. Ash and Patti H. Clayton’s Articulated Learning prompt, a kind of reflective writing students in the FYW course posted for each project, provided a useful construct for engaging students in thinking about the academic, personal, and civic outcomes of their work in our inquiry-guided course. The prompt asks students to identify their own learning objectives and to describe why this learning matters and how it may be applied in the future (Ash and Clayton 234). Ash and Clayton describe these structured reflection assignments, which occur “continuously...and in multiple forums” as a means of helping students work through problems and decision-making in a collaborative learning environment (231-232).

The post-project reflective prompts for students in the BW course also foregrounded taking ownership of one’s own learning. Specifically, students wrote their reflective assignments with their own personal learning objectives (PLOs) in mind (Latawiec). In these assignments, students were asked to call upon the PLOs they developed at the beginning of the course and monitored throughout the course. In this way, these assignments were designed to give students ownership over their learning to determine whether and to what extent the course and their learning aligned with their academic goals.

The reflective argument, the final assignment in both courses, adapted from Edward M. White’s description of “Phase II” portfolio scoring, also allowed students in both the FYW and BW course to make some decisions about how to address their learning through different rhetorical strategies. The reflective argument genre, as we still use it in our program, asks students to make a claim about their achievement of course learning outcomes, and then to provide and analyze evidence from their coursework that shows their growth or progress as a writer. Students prepared portfolios and discussed learning outcomes throughout the semester to prepare for this final assignment. This space for students to reflect on their overall experiences in a course, with attention to how these experiences helped them make progress on course learning outcomes, seemed especially valuable to us for both our students’ self-assessment of experiences in the class and our own teacher research. Through the reflective argument artifact, both students and teachers could consider whether and how particular strategies, like collaboration, impacted learning.

The reflective argument assignments for both courses asked students to make slightly different moves as they constructed their arguments about writing. The assignment for BW asked students to “reflect on the work completed throughout the course of the semester,” which included their “writing and writing practices.” Along with this direction, students were urged to “consider the course learning objectives reflected on throughout the semester” noting that students could “recall [their] own personal learning objective(s) as well as those listed in the syllabus.” The assignment for FYW asked students to use their formal and informal work from the semester as evidence for their “progress and/or growth as a writer, reader, researcher, and reflector.” These reflections were posted on FYW students’ individual blogs, and students were encouraged to hyperlink from the reflective essay to relevant artifacts from the course. Because both prompts asked students to consider formal and informal work in the courses as evidence, these reflective argument texts became central to our analysis of students’ reflective writing.

Working Through Our Study of Collaborative Learning in BW and FYW

Our two-semester, IRB-approved study of students’ collaborative learning in our BW and FYW classes across two semesters included 12 BW participants and 16 FYW student participants. Students consented to participate in the study early in the semester through a third-party recruiter, and as participant observers we did not know which students participated in the study either semester until after we submitted final grades. For both the Winter and Fall 2014 semesters we collected students’ reflective writing, including in-process and post-project reflections and reflective argument essays. We used qualitative text-analysis to understand whether and how students saw the collaborative course design as a key part of their learning.

It is important here to reiterate that our research question was centered on how students wrote about collaboration in their post-project and capstone reflections. We could have selected other research methods to understand this—for example, we could have elected to focus solely on our own observations and field notes of collaboration in action. However, had we done so, there is still significant collaboration (as defined in our codes, described below) that we would have missed and that we would not have been able to identify with our own eyes and ears. Additionally, we were experienced in analyzing students' reflections; we saw reflections as both a learning and research tool that could provide us insight into our questions through students' own voices, and close to the moment of the experience, unlike interviews, which would happen months later, after final grades were submitted and we would learn who participated in the study. Finally, we felt it was important not to ask participating students to do anything above and beyond what was required for these classes; we felt the service-learning oriented research we were requiring of them, including formal presentations to institutional guests at the end of the term, was request enough beyond the general education class. Any teacher researcher makes considerations like these in planning a study. Blakeslee and Fleischer point out, “all of these factors—your research questions; the setting and context for your research; your background, experience, and comfort level; and your positionality and theoretical stances—will influence your decisions about what tools you will use” (99){2}.

Beginning with the Winter 2014 student texts, we worked through open coding on four reflective argument essays to understand whether and how students addressed collaborative writing tasks. We developed codes inductively from the data, following a standard qualitative procedure (Merriam). This open coding process was driven by our primary research question: How do reflective writing assignments designed to make students think about course learning objectives help them understand the ways in which they find collaboration to be an integral part of achieving the course learning objectives? Looking for moments in the essays where students wrote about interaction with others in the class, we identified the codes listed and defined in Table 1.

Table 1. Collaboration codes identified through inductive coding

Parent Code	Child Code	Example
Collaboration{3}
	Whole class collaboration The development of knowledge for or about writing through whole-class work.	“During Project one I felt like I was taught a majority of the project through our class discussions like how to properly explain an area that no one else has seen before in detail or working through the readings to understand how to take field notes in a descriptive manner, which makes decoding your notes a lot quicker and easier” (Amber).
	Small group collaboration{4} Development of knowledge for or about writing through small-group work	“The first thing we did was reread the line and truly evaluate the print. We then discussed the idea with one another and concluded that it indeed was appropriate to integrate in our paper. We found out that it coincided with our belief that the layout of the classroom correlated with the ability to learn, so we included it. We continued this group discussion every time we were evaluating information we obtained from research. Our goal was to avoid implanting research that was irrelevant to our topic” (Nurin).
Feedback A direct reference to what a teacher or peer told a student to do, and, thus, what resulted from that feedback (explicit)
	Peer feedback	“Second, I brought the rough draft to class and had my partner read it, while reading it (aloud) I noticed many parts that were confusing, and many parts of my paper that just didn’t flow properly. After receiving constructive criticism from her I decided to go back and fix what she pointed out before making an appointment with my instructor” (Brooke).
	Teacher feedback	“...at first I was very crazy about writing everything down so I thought I was doing it wrong. Then I asked my professor and she explained to me how little things would make the greatest impact and I would later see that when it came to code and finding patterns in the notes” (Alina).
Influence An indirect reference to what a teacher or peer did in the classroom that impacted the student to change their processes (implicit)
	Peer influence	“I never felt confident about my writing and hated to share my writing with the peers but when I got to this class the friendly and honesty reposes that I got from the peers on my paper has changed me to be more confident about my writing” (Asa).
	Teacher influence	“That was one of the main things [my teacher] talked about while teaching us about annotating. You can’t just highlight your text without knowing why you did it, if someone asks you about a certain section you highlighted you should be able to explain to them what you found this important about it” (Anna).
Social motivation An expression of a sense of class activities contributing to a common good.		“For Project 3, all the students came together for the common goal of making room 337 of State Hall a better learning environment, this common goal made us all a part of a valuable discourse community that will hopefully impact the learning experience of all students who will be in that classroom in future semesters” (Beth).
Attitude An expression of a disposition toward group work or collaboration; an expression of a change in attitude based on group work.		“Starting off I picked a group that would help me but at the same time that I could help. In the beginning, I was hesitant to rely on any of them because this is my grade. As the project progressed and others were putting their parts in I understood at that point I could trust them. I did whatever I could to help” (Nadia).

After identifying these codes, we focused mostly on students’ reflective argument essays for our analysis. We believed the reflective argument essay, as a student’s evaluation of their work throughout the semester, would be the appropriate assignment to code, because, as described above, the assignment requires the student to choose the information they present in order to craft an argument about their success in the course. If students thought collaborative learning impacted how they met course learning outcomes or accomplished their own learning objectives, we felt it was likely that they would write about collaboration in the reflective argument. While evaluative reflection is explicitly prompted in the reflective argument essay, though, students were not explicitly prompted to discuss the collaborative practice that was central to the course. Because collaboration was a significant part of students’ weekly experiences in the class, however, we wanted to understand to what degree students wrote about this collaborative practice when they were not explicitly asked to do so. Learning about their experience with collaboration through the reflective argument could help us understand whether, how, and to what degree we were highlighting the social processes of learning to write and developing writing knowledge in our instruction. This strategy could also help us understand the degree to which students saw these collaborative moments as integral to their development of writing knowledge, not just incidental. Ultimately, we were able to apply five of the eight codes to BW reflective argument essays, and all eight codes to FYW essays (see Table 2).

Table 2. Number of applications of codes in reflective argument essays

Code	Number of Applications in Reflective Argument Essay
	Basic Writing	First Year Writing
Whole-class Collaboration		2
Small-group collaboration	1	21
Peer feedback	2	8
Teacher feedback	8	11
Peer influence	2	3
Teacher influence	4	5
Social motivation		4
Attitude		5

After reviewing coded excerpts of reflective arguments, we discovered that in FYW artifacts, coded passages fell in the section of the reflective argument focused on the “research” learning outcome. In this section of the essay, students often discussed the collaborative, researched proposal assignment in which they worked together to write proposals for changing the functional and aesthetic aspects of our program’s computer classrooms. Because of this finding, we decided to review students’ reflective writing across the semester to understand why, when collaboration was integrated throughout the course, it was largely only appearing in the “research” section of the final reflection where students had to write about the collaborative research essay as the most viable evidence. Figuring out how to present our findings was difficult. We lacked a compelling data set across participants, but discussion of collaboration in reflective writing was not wholly absent. So, to understand how to present the most common, albeit limited, discussion of collaboration we found, we analyzed the work of two students—one in FYW and one in BW. Working through these two cases, we considered whether and how the reflective assignments in each class provided reasonable opportunity for students to write about collaboration. We selected Nurin, an FYW student, because her coded passages on small-group collaboration in the reflective argument were especially specific. We selected Anna, a BW student, because hers was the only reflective argument in the BW corpus coded for collaboration.

Tracing Anna and Nurin’s reflective writing through the semester, we worked to understand the degree to which collaboration codes or other codes could be applied to their earlier reflective writing, as well as what we could learn from the scaffolding of this reflective work across the semester. Our findings from studying these two cases more closely showed us that though both classes engaged in collaborative activities, neither Anna nor Nurin referenced any of these activities as connected to collaborative knowledge-making within the first two projects of the course. While they did write about teacher and peer feedback, as well as teacher and peer influence, two sets of codes we used, they did not write about explicit collaborative knowledge-making. This limited reference to collaboration in both students’ reflections suggested to us something important about our use of the reflective argument both for research and as a capstone assignment in these particular assignment sequences. It also left us with a major sticking point: despite integrating both collaboration and reflection throughout our classes, we could not find significant evidence of collaboration in students’ reflections. What conclusions could we present and how?

A Way Forward

Our story’s contribution to knowledge-building about reflective writing echoes Ann Blakeslee and Cathy Fleischer’s reminder that “if things do not go just as you hoped in your research, then you will likely need to go back and plan certain aspects of it all over again. Planning is productive work, but it is also challenging and ongoing work” (75). As teacher researchers, we must keep reviewing our questions, thinking through data, reading literature in the field; we must be ready to make changes as needed (75). So, considering the multiple studies of students’ reflective talk and writing cited above, how might we re-conceive systematic reflection in a course like this to emphasize collaboration or any other learning strategy or classroom behavior we might want to study? How could scaffolding reflection throughout a course both leave students the opportunity to express their wide array of developing writing knowledge and help teacher researchers effectively study aspects of students’ learning?

In this final section, we present strategies for readers looking to learn from our writer/researcher challenges. These strategies may help them plan a way forward if they sense a need to re-strategize their own research plans, or if they, too, struggle with a research story that seems hard to tell. We present our conclusions in a few areas: scaffolding reflective assignments systematically, dealing with perceived programmatic constraints when planning research, and reviewing the use of reflections in pilot courses as a framing tool for future research. While we focus this implications section on speaking to studies that use students’ reflective writing as a central research artifact, we believe this section carries lessons for qualitative and teacher researchers more broadly.

Foremost, we understand that scaffolding reflective assignments throughout a semester may be helpful in introducing and demonstrating the value of reflective writing to students who may need time to practice reflective genres and develop writing knowledge through this activity. Reflective prompts have the potential to engage students in thinking through any number of learning experiences and in describing their rhetorical decision-making across projects. However, we also know that these smaller reflective assignments might not explicitly build toward what is asked in a more prescriptive capstone assignment (like the reflective argument essay) that is often used for the purposes of program assessment. Yancey gets at this tension between representing the social and the individual aspects of learning and writing, describing how commonly used writing assessments do not attend well to the “messy” nature of our social writing ecologies (It’s Tagmemics and The Sex Pistols 200).

In our study, we saw brief insights into students’ collaborative work in some of their early reflection assignments. Ultimately, though, while these earlier reflections potentially provided evidence that collaboration happened across the semester, these references to collaboration did not manifest in the final reflective argument as thoroughly as we hoped. While we worked to develop a scaffold of reflection in our courses, we did not anticipate how the constraints of the final reflective prompt might stifle students’ ability to understand how to draw from those earlier reflections. While reflective writing is systematically integrated into our program’s composition courses, and while we did scaffold reflection throughout the semester, we did not systematically scaffold reflection about how collaboration as a learning strategy informed or influenced students’ developing writing knowledge. Our assumption was that open-ended reflections would allow students the space for writing about whatever learning experiences were most impactful, and this may be what manifested. However, because of the evaluative, individual-centered nature of our final reflective writing assignment, we cannot be sure. The prompts in both courses moved from broad, open reflective invitations to specific questions about particular assignments and learning objectives. When the shift from broad to specific reflective prompts happened, we saw a move away from discussing collaborative work and toward an emphasis on individual work in some students’ reflections. While we did notice one exception when students wrote about the collaborative research essay in the “research” section of the final reflection, we found that a focus on individual work held true for most student texts in the study.

The second implication we consider is how our story draws attention to the need to deal with perceived constraints in selecting research methods and developing artifacts for analysis in line with specific research questions. In teaching and researching our paired classes, we desired as much as possible to study our courses using our planned curricula and without additional research instruments like interviews or surveys. Perhaps this intention came as much from already having done the labor of developing new curricula for these courses as it did from being confident that we could learn what we needed to about students’ learning from the artifacts required in the course. Even as non-tenure-track teacher researchers, we enjoyed the freedom to responsibly adapt our course plans and assignments for our research. However, as central teacher leaders in our composition program, we also wanted our courses to be part of the annual assessment conversation; we wanted our approach to teaching these two classes to inform that conversation, and so using the common assessment artifact, the reflective argument essay, seemed the best approach to both conducting our research and demonstrating a fidelity to program assessment. We will note, though, our reading of more recent studies suggests that scaffolding more focused reflection would not have hampered that fidelity. For example, Ruth Boeder’s 2017 study of students’ research and writing processes in FYC, conducted using the same program-wide final assessment assignment our study employed, also used scaffolded journal prompts on research and writing. Boeder’s analysis of a journal prompt specifically focused on students’ thinking about potential research sources results in clear conclusions about seeming disconnects between students’ thinking about author and audience, and their reluctance in research writing to engage authors with whom they disagree. These conclusions, developed from the analysis of a specifically-targeted journal prompt, lead to programmatically-viable instructional interventions.

A third implication or lesson worth exploring as we reflect on our story is the value of piloting reflection assignments prior to use in formal studies. Literature in composition scholarship is replete with descriptions of specific pilot studies, but not with texts focused on explaining more generally how to construct pilot studies. So we turn to cross-disciplinary design texts, like John W. Creswell and J. David Creswell’s Research Design, in which the one explicit reference to “pilot testing” relates to developing and testing quantitative research instruments, like surveys (154). From our disciplinary neighbors in Education, Susan McKenney and Thomas C. Reeves’ Conducting Educational Design Research discusses pilot testing in more depth and with more possibilities for the kinds of interventions that may be explored and the kinds of data collection that may occur in the pilot: “video review, discourse analysis, structured (non-)participant observation, interviews, questionnaires, assessments, participant logbooks, focus groups, document analysis, and computer logs” (176). The authors define the term “pilot” as a “field testing of interventions in settings that approximate, but do not completely represent, the target context” and in which some of the conditions or contextual factors are “non-natural” (176). McKenney and Reeves’ definition reflects an intention at the core of our 2014 study: we hoped our experiment with these two paired classes would help us envision, in part, how similar pairings or other modes of peer mentoring could develop in and among the composition courses at our institution.

Framing our initial study as a pilot test in this way may have better helped us also choose how we would study this collaboration using reflection, as we would need to craft reflective assignments that could be easily understood and integrated by instructors in other sections, should we anticipate a wider study or larger integration of the teaching approach. The “non-natural” conditions of our study included the service-learning project we had integrated into the courses: students’ research and proposals regarding the physical condition of the computer classrooms we were using. This was a teaching and learning context that could not be replicated across multiple sections in a wider study. We did not conduct a pilot study before a more formal version, though Amy had used the reflections for her BW class in previous courses, and we had both worked enough with scaffolded reflection and the reflective argument essay in our classes to understand how to prepare students to write in response to those reflective prompts. Reflecting on our own story, however, helps us see the value of piloting a study even when the research tools feel very familiar, as reflective writing and collaboration felt for us. And our suggestion of pilot testing does not negate the value of discovery through teacher research. Do we hope for compelling results from our teacher research that will inform others’ teaching? Yes. Do we also know this research will reshape our own classrooms and our approaches to research because of what we both can and cannot plan for? Yes. We know that in classroom-based composition research, our study of students' learning through reflective writing is more focused and productive once we have conducted pilot tests that help us refine our frameworks and understand better how to integrate that reflective writing to benefit both students' learning and our understanding of that learning.

We also encourage our readers to remember that even after meticulously planning a study, you may realize that you have not done everything needed to answer your research questions or may feel like not everything has been done right. We followed the “textbook” steps to preparing our study, did our mindful, pedagogical best as teachers in the classroom, used models from scholarship to design reflective writing activities for students, worked through a careful open coding process to develop codes related to collaboration from students’ reflections, and then worked through the corpus of reflections to develop analytical conclusions. However, even though all the pieces seemed like they would come together to help us finish our puzzle, they didn’t. The missing piece, we think, is the systematic reflection we have described. Once we began studying students’ reflections, a more defined, student-driven vocabulary and set of concepts for how to talk about collaboration as a learning strategy began to emerge and is now represented in the coding schema described above. We can imagine that if we were to conduct the study again, using more systematic reflection on collaboration, perhaps students’ reflections would suggest that collaboration was, indeed, more integral to their learning in these two classrooms than we see in this present set of artifacts. As a result, we would be able to see more clearly how that collaboration supported students’ developing writing and rhetorical knowledge. But we don’t see this yet; we need Blakeslee and Fleischer’s encouraging reminder that this is part of the process of becoming writing researchers.

From the conception of this project to this final draft of writing about our research study and experiences, we have learned several lessons. As teachers, we learned about integrating collaborative research and ultra-localized service-learning and the challenges of meaningfully enacting institutional change in the short time frame of the semester. As participants in our composition program’s annual general education assessment, we learned that the reflective argument essay was helpful for learning about how students could write about their progress on the course learning outcomes, but was not as helpful for learning about other influential moments in the class that fell outside of that framework. As researchers, we learned that our enthusiasm for studying our classes and sharing our findings needed to be balanced with the time it takes to craft a story that is critical and contributes to the larger teacher researcher conversation (Holmsten). Thus, as writers, over time, we have learned about the complexity of presenting a project with so many threads in a way that is compelling and useful. Moving forward as composition teachers and researchers, we are confident our lesson can meaningfully inform future classroom research, especially research that centers on understanding student learning through reflective writing.

Notes

1. In part, this version of a “team-taught” set of classes (in which we designed and reflected on the curriculum together, and planned and facilitated several class sessions where our students came together to work through writing and discussion), was a way for us to wade into learning about peer mentoring, to see how students in different composition courses and with different research, writing, and learning experiences might be able to teach and learn from each other. This work helped us set the foundation for what is now the Composition Learning Community, a unique, program-centered learning community at our institution, now in its seventh year. (Return to text.)

2. Blakeslee and Fleischer also remind us that “Even experienced researchers grapple both with what tools to use and how best to use them” (99). (Return to text.)

3. In our open coding, we developed a code for “teacher collaboration,” following the codes developed for teacher feedback and teacher influence. However, teacher collaboration, or the development of knowledge for or about writing through dialogue with the instructor, was never applied as a code to students’ reflective argument essays. (Return to text.)

4. The weight of small group collaboration codes was assigned based on whether students were merely dictating a process the small group worked through (e.g. “we looked through sources and then we documented them in our paper”), OR whether students provided any degree of insight about or analysis of the collaboration (description only = 1, degree of analysis or insight = 2-5). (Return to text.)

Works Cited

Allan, Elizabeth G., and Dana L. Driscoll. The Three-Fold Benefit of Reflective Writing: Improving Program Assessment, Student Learning, and Faculty Professional Development. Assessing Writing, vol. 21, 2014, pp. 37-55.

Ash, Sarah L., and Patti H. Clayton. Service-Learning: Integrating Inquiry and Engagement. Teaching and Learning Through Inquiry: A Guidebook for Institutions and Instructors, edited by Virginia S. Lee, Stylus Publishing, 2004, pp. 229-244.

Blakeslee, Ann, and Cathy Fleischer. Becoming a Writing Researcher. 2nd ed., Routledge, 2019.

Boeder, Ruth. Looking to the Research Paper and Beyond: Information Literacy and Rhetorical/Composition Perspectives on Student Research and Writing Processes. LOEX Quarterly, vol. 45, no. 2, 2018.

Bransford, John, et al. Learning and Transfer. How People Learn: Brain, Mind, Experience, and School: Expanded Edition. National Academy of Sciences, 2000.

Bruffee, Kenneth. Collaborative Learning: Higher Education, Interdependence, and the Authority of Knowledge. 2nd ed., Johns Hopkins UP, 1999.

Creswell, John W., and J. David Creswell. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 5th ed., Sage, 2014.

Driscoll, Dana Lynn, et al. Genre Knowledge and Writing Development: Results from the Writing Transfer Project. Written Communication, vol. 37, no. 1, 2020, pp. 69-103.

Duffy, William. Collaboration (in) Theory: Reworking the Social Turn’s Conversational Imperative. College English, vol. 76, no. 5, May 2014, pp. 416-435.

Eodice, Michelle, and Kami Day. Learning from Coauthoring: Composing Texts Together in the Composition Classroom. Teaching with Student Texts: Essays Toward an Informed Practice, edited by Joseph Harris, John D. Miles, and Charles Paine, Utah State UP, 2010, pp. 190-199.

Fiscus, Jaclyn M. Genre, Reflection, and Multimodality: Capturing Uptake in the Making. Composition Forum, vol. 37, 2017. http://compositionforum.com/issue/37/genre.php.

Gorzelsky, Gwen, et al. Cultivating Constructive Metacognition: A New Taxonomy for Writing Studies. Critical Transitions: Writing and the Question of Transfer, edited by Chris M. Anson and Jessie L. Moore, WAC Clearinghouse and UP of Colorado, 2016, pp. 215-246.

Guinot Varty, Nicole. Ecological Awareness: Enacting an Ecological Composition Curriculum to Encourage Student Knowledge Transfer. 2016. Wayne State University. PhD dissertation, Digital Commons.

Higgins, Lorraine, et al. Community Literacy: A Rhetorical Model for Personal and Public Inquiry. Community Literacy Journal, 2006.

Holmsten, Victoria. My World Is Made Up of Stories. Feminist Empirical Research: Emerging Perspectives on Qualitative and Teacher Research, edited by Joanne Addison and Sharon James McGee, Boynton/Cook, 1999, pp. 40-52.

Jankens, Adrienne. Learning How to Ask Writing Questions with Rhetorical Reflections. Composition Forum, vol. 41, 2019. http://compositionforum.com/issue/41/rhetorical-reflections.php.

---. Responsive Classroom Ecologies: Supporting Student Inquiry and Rhetorical Awareness in College Writing Courses. 2014. Wayne State University, PhD dissertation. Digital Commons.

Keating, Benjamin. ‘A Good Development Thing: A Longitudinal Analysis of Peer Review and Authority in Undergraduate Writing. Developing Writers in Higher Education: A Longitudinal Study, edited by Anne Ruggles Gere, U of Michigan P, 2019, pp. 56-79.

Latawiec, Amy Ann. Self-Directed Learning and the Development of Self-Efficacy in Basic Writing. 2016. Wayne State University, PhD dissertation. Digital Commons.

Leverenz, Carrie Shively. Collaboration, Race, and the Rhetoric of Evasion. JAC, vol. 16, no. 2, 1996, pp. 297-312.

McKenney, Susan, and Thomas C. Reeves, editors. Conducting Educational Design Research. 2nd ed., Routledge, 2019.

Merriam, Sharan B. Qualitative Research: A Guide to Design and Implementation. Jossey-Bass, 2009.

Nowacek, Rebecca S. Agents of Integration: Understanding Transfer as a Rhetorical Act. Southern Illinois UP, 2011.

Shor, Ira. When Students Have Power: Negotiating Authority in a Critical Pedagogy. U of Chicago P, 1996.

Sommers, Jeff. Problematizing Reflection: Conflicted Motives in the Writer’s Memo. A Rhetoric of Reflection, edited by Kathleen Blake Yancey, Utah State UP, 2016, pp. 272-287.

---. Reflection Revisited: The Class Collage. Journal of Basic Writing, vol. 30, no. 1, 2011, pp. 99-129.

Taczak, Kara. Reflection is Critical for Writers’ Development. Naming What We Know: Threshold Concepts of Writing Studies, edited by Linda Adler-Kassner and Elizabeth Wardle, Utah State UP, 2015, pp. 78-81.

Taczak, Kara, and Liane Robertson. Reiterative Reflection in the Twenty-First Century Writing Classroom: An Integrated Approach to Teaching for Transfer. A Rhetoric of Reflection, edited by Kathleen Blake Yancey, Utah State UP, 2016, pp. 42-63.

Trimble, Thomas, and Adrienne Jankens. Features of Reflective Writing: Using Taxonomies of Metacognitive Behaviors to Analyze Student Reflection and Improve Teaching Practice. Pedagogy: Critical Approaches to Teaching Literature, Language, Composition, and Culture, vol. 19, no. 3, Oct. 2019, pp. 433-454.

Trimbur, John. Consensus and Difference in Collaborative Learning. Cross-Talk in Comp Theory: A Reader, edited by Victor Villaneuva and Kristin L. Arola, 3rd ed., NCTE, 2011, pp. 441-458.

VanKooten, Crystal. Identifying Components of Meta-Awareness about Composition: Toward a Theory and Methodology for Writing Studies. Composition Forum, vol. 33, 2016. http://compositionforum.com/issue/33/meta-awareness.php.

Wallace, David L., and Helen Rothschild Ewald. Mutuality in the Rhetoric and Composition Classroom, Southern Illinois UP, 2000.

White, Edward M. The Scoring of Writing Portfolios: Phase 2. College Composition and Communication, vol. 56, no. 4, June 2005, pp. 581-600.

Winzenried, Misty Anne, et al. Co-Constructing Writing Knowledge: Students’ Collaborative Talk Across Contexts. Composition Forum, vol. 37, Fall 2017. http://compositionforum.com/issue/37/co-constructing.php.

Yancey, Kathleen Blake. Introduction: Contextualizing Reflection. A Rhetoric of Reflection, edited by Kathleen Blake Yancey, Utah State UP, 2016, pp. 3-20.

---. It’s Tagmemics and the Sex Pistols: Current Issues in Individual and Programmatic Writing Assessment. Academic and Professional Writing in an Age of Accountability, edited by Shirley Wilson Logan and Wayne H. Slater, Southern Illinois UP, 2018, pp. 199-212.

---. Reflection in the Writing Classroom. Utah State UP, 1998.

Yancey, Kathleen Blake, et al. Writing Across Contexts: Transfer, Composition, and Sites of Writing. Utah State UP, 2014.