Composition Forum 27, Spring 2013
http://compositionforum.com/issue/27/
Appendix 5 to Intractable Writing Program Problems: Results of Fall 2009 Portfolio Assessment
This is an appendix to Intractable Writing Program Problems, Kairos, and Writing about Writing: A Profile of the University of Central Florida's First-Year Composition Program.
Raters read portfolios and then drew upon the entire portfolio to score 12 specific characteristics that the assessment group had created and modified using sample portfolios.
General Summary of ENC 1101 Portfolio Analysis
Fall 2009
|
|
New curriculum 19 |
New curriculum 25 |
Traditional curriculum 25 |
|
|
% of portfolios at excellent or good |
% of portfolios at excellent or good |
% of portfolios at excellent or good |
|
All major papers demonstrate college-level thinking and exploration of ideas and issues. |
48% |
45% |
30% |
|
All major papers demonstrate a level of polish (editing, formatting, etc) appropriate for college-level writing. |
55% |
51% |
48% |
|
In at least one paper, writer demonstrates the ability to rhetorically analyze complex texts written by others. |
67% |
38% |
17% |
|
In at least one paper, writer demonstrates the ability to carefully consider an idea or issue. |
63% |
50% |
51% |
|
In reflective comments, revision memos, or papers, writer demonstrates the ability to carefully reflect on her own writing processes and practices. |
58% |
55% |
6% |
|
In all papers that include outside sources, writer uses correct in-text citations. |
52% |
50% |
29% |
|
In all papers that include outside sources, writer includes correctly formatted works cited pages. |
49% |
45% |
31% |
|
In at least one paper, writer carefully integrates outside sources into her own text. |
63% |
47% |
33% |
|
Writer includes two or more outside sources per paper |
67% |
67% |
19% |
|
There is evidence of peer and/or teacher feedback on drafts |
94% |
94% |
79% |
Statistical Analysis of ENC 1101 Portfolio Results, Fall 2009
(category titles have been truncated from above)
Ten of twelve items showed significant differences among the three groups, with both sizes of WAW courses showing the highest scores in each case, and the WAW sections of 19 scoring the highest of the three groups on all but one measure. Only items examining polish and works cited format did not show any significant differences between groups at alpha = 0.05. On the remaining ten measures, there were various statistically significant differences between groups. The following table lists the 12 items judged for each portfolio, the mean scores for each group, the observed significance levels (p values) for the One-Way ANOVA F test simultaneously comparing the mean scores for the 3 types of instruction, and the level of interrater reliability using Cronbach’s Alpha.
|
|
Observed Significance level |
WAW Sections of 19 Mean |
WAW Sections of 25 Mean |
Traditional Sections of 25 Mean |
|
Thinking |
0.0255 |
2.74 |
2.48 |
2.37 |
|
Polish |
0.2276 |
2.88 |
2.65 |
2.68 |
|
Rhetorically analyze |
0.0001 |
3.01 |
2.42 |
1.12 |
|
Consider idea |
0.0047 |
3.04 |
2.61 |
2.66 |
|
Reflective |
0.0001 |
3.07 |
2.71 |
0.52 |
|
In-text citation |
0.0255 |
2.64 |
2.49 |
2.13 |
|
Work cited |
0.1713 |
2.46 |
2.33 |
2.08 |
|
Outside sources |
0.0004 |
2.96 |
2.73 |
2.34 |
|
Two plus sources |
0.0001 |
2.97 |
2.85 |
2.04 |
|
Feedback |
0.0001 |
3.77 |
3.75 |
3.15 |
|
Macro revision |
0.0032 |
2.07 |
2.07 |
1.52 |
|
Micro revision |
0.0160 |
2.63 |
2.58 |
2.12 |
David Nickerson, Professor and Chair of the Department of Statistics at the University of Central Florida, conducted all statistical analysis.
Appendix 5 to “Intractable Writing Program Problems” from Composition Forum 27 (Spring 2013)
Online at: http://compositionforum.com/issue/27/ucf-appendix5.php
© Copyright 2013 Elizabeth Wardle.
Licensed under a Creative Commons Attribution-Share Alike License.
Return to Composition Forum 27 table of contents.