Skip to content

Composition Forum 27, Spring 2013
http://compositionforum.com/issue/27/

Appendix 5 to Intractable Writing Program Problems: Results of Fall 2009 Portfolio Assessment

Elizabeth Wardle

This is an appendix to Intractable Writing Program Problems, Kairos, and Writing about Writing: A Profile of the University of Central Florida's First-Year Composition Program.

Raters read portfolios and then drew upon the entire portfolio to score 12 specific characteristics that the assessment group had created and modified using sample portfolios.

General Summary of ENC 1101 Portfolio Analysis

Fall 2009

New curriculum 19

New curriculum 25

Traditional curriculum 25

% of portfolios at excellent or good

% of portfolios at excellent or good

% of portfolios at excellent or good

All major papers demonstrate college-level thinking and exploration of ideas and issues.

48%

45%

30%

All major papers demonstrate a level of polish (editing, formatting, etc) appropriate for college-level writing.

55%

51%

48%

In at least one paper, writer demonstrates the ability to rhetorically analyze complex texts written by others.

67%

38%

17%

In at least one paper, writer demonstrates the ability to carefully consider an idea or issue.

63%

50%

51%

In reflective comments, revision memos, or papers, writer demonstrates the ability to carefully reflect on her own writing processes and practices.

58%

55%

6%

In all papers that include outside sources, writer uses correct in-text citations.

52%

50%

29%

In all papers that include outside sources, writer includes correctly formatted works cited pages.

49%

45%

31%

In at least one paper, writer carefully integrates outside sources into her own text.

63%

47%

33%

Writer includes two or more outside sources per paper

67%

67%

19%

There is evidence of peer and/or teacher feedback on drafts

94%

94%

79%

Statistical Analysis of ENC 1101 Portfolio Results, Fall 2009

(category titles have been truncated from above)

Ten of twelve items showed significant differences among the three groups, with both sizes of WAW courses showing the highest scores in each case, and the WAW sections of 19 scoring the highest of the three groups on all but one measure. Only items examining polish and works cited format did not show any significant differences between groups at alpha = 0.05. On the remaining ten measures, there were various statistically significant differences between groups. The following table lists the 12 items judged for each portfolio, the mean scores for each group, the observed significance levels (p values) for the One-Way ANOVA F test simultaneously comparing the mean scores for the 3 types of instruction, and the level of interrater reliability using Cronbach’s Alpha.

Observed Significance level

WAW Sections of 19 Mean

WAW Sections of 25 Mean

Traditional Sections of 25 Mean

Thinking

0.0255

2.74

2.48

2.37

Polish

0.2276

2.88

2.65

2.68

Rhetorically analyze

0.0001

3.01

2.42

1.12

Consider idea

0.0047

3.04

2.61

2.66

Reflective

0.0001

3.07

2.71

0.52

In-text citation

0.0255

2.64

2.49

2.13

Work cited

0.1713

2.46

2.33

2.08

Outside sources

0.0004

2.96

2.73

2.34

Two plus sources

0.0001

2.97

2.85

2.04

Feedback

0.0001

3.77

3.75

3.15

Macro revision

0.0032

2.07

2.07

1.52

Micro revision

0.0160

2.63

2.58

2.12

David Nickerson, Professor and Chair of the Department of Statistics at the University of Central Florida, conducted all statistical analysis.

Return to Composition Forum 27 table of contents.