The Discourse Community: Unisa’s Practical English Course

Get Complete Project Material File(s) Now! »

The Empirical Study

In a carefully controlled experiment involving Unisa’s Practical English students, conducted in 1995, the level of improvement in the revised paragraph resulting from the five divergent marking strategies discussed in Section 1 of this chapter was statistically examined. The following appendices are so directly relevant to this section of the research that they have been printed after this chapter rather than in at the end of the thesis. They are:
• The ESL Composition Profile (p 168-69);
• Instructions for the Marking of Assignment 01 (p 170-72);
• Self Assessment Tutorial Letter (p 173-75);
• Instructions for the Marking of Assignment 04: The Revised Paragraph (p 176);
• Instructions to Markers for the Second Examining of Research Scripts (p 177) .
• A photostat copy to demonstrate the final appearance of the A3 pages is printed as Addendum 10 at the end of the thesis.
To avoid unnecessary repetition, information printed in these documents has not been repeated in the text of this chapter and readers are requested to refer to the relevant documents when they are mentioned in the text.

The Aim

The empirical study was designed to compare the effect on student revisions of five divergent marking strategies. The four experimental methods tested (Correction Code (CC), Minimal Marking (MM), Taped Response (TR) and Self Assessment (SA)) are formative 13 evaluation techniques designed to improve performance through feedback. In contrast, the control group (CG) is restricted to summative feedback in that they were only awarded a mark. The empirical study was designed to determine the relative effectiveness of the marking methods and was not a study of assessment itself as reflected in rankings, percentage grades, or test scores. Ranking was used to grade the original and the revised version of the student’s writing for the sole purpose of comparing the two ratings to allow statistical measurement of the ‘improvement’ resulting from the marking method. For the purposes of this study the effectiveness of the marking method could be determined by subtracting the score allocated to the original paragraph from the score given to the revised version. For example, ifthe student received 12 out of 25 for the original work and 14 for the revised paragraph, the improvement would be +2. Expressed as a percentage, the mark had improved by 8% (2 times 4). Groups receiving the divergent marking treatments could be compared as the manipulation of data remained constant across the five groups.

Hypotheses Tested

The following hypotheses were tested:
• Hypothesis 1:Each of the four experimental groups will have a content improvement greater than that of the control group.
• Hypothesis 2:Each of the four experimental groups will have a form improvement greater than that of the control group.
• Hypothesis 3:Each of the four experimental groups will have a mean total improvement greater than that of the control group.
• Hypothesis 4:Because practice, without any lecturer intervention, promotes fluency, the control group’s mean improvement for the revised paragraphs will reflect a higher mark than that awarded to their original scripts.
• Hypothesis 5:Because of the form-related emphasis inherent in the Correction Code, the mean improvement with respect to grammar will be greater than the mean content improvement for this group.
• Hypothesis 6:Because of the farm-related emphasis inherent in Minimal Marking, the mean improvement with respect to grammar will be greater than the mean content improvement for this group.
• Hypothesis 7:Because students struggle to self-correct grammatical features,the Control Group’s mean improvement with respect to content will be greater than that recorded for form.
• Hypothesis 8: Because the Taped Response allows lecturers to place greater emphasis on the larger issues of discourse, the mean improvement
• Hypothesis 9: with respect to content will be greater than the mean form improvement for this group.
Because students struggle to self-correct grammatical features,
the Self Assessment group’s mean content improvement will be
greater than that recorded for form.

Chapter One: Introduction
1. Aims
2. Statement of the Problem
3. Hypotheses
4. Field
4.1 The Role of Writing
4.2 The Communicative Approach
4.3 Response Strategies
5. The Discourse Community: Unisa’s Practical English Course
6. Scope
7. The Gap in Response Theory
8. Chapter Outline
8.1 Chapter One: Introduction
8.2 Chapter Two: Writing, Reading and Frameworks
of Response
8.3 Chapter Three: Research Overview
8.4 Chapter Four: The Text
8.5 Chapter Five: The Students’ Perspectives
8.6 Chapter Six: The Readers’ Perspectives
8.7 Chapter Seven: Conclusions and Recommendations
9. Conclusion
Chapter Two: Writing, Reading and Frameworks of Response
1. Writing Theory
1.1 Formalist Axiology: The Current-Tr~itional Paradigm
1.1.1 Formalist Axiology: Implications for Responding to Student Writing
1.2 Emotive Axiology: The Expressivist Paradigm
1.2.1 Emotive Axiology: Implications for Responding to Student Writing
1.3 Mimetic Axiology: The Cognitive Paradigm
1.3.1 Mimetic Axiology: Implications for Responding to Student Writing
1.4 Rhetorical Axiology: The Socio-Constructionist Paradigm
1.4.1 Rhetorical Axiology: Implications for Responding to Student Writing
2. Reading Theory
2.1 Paradigmatic Shifts in Reading Theory
2.2 Reading Student Texts: Rhetorical Distortions
3. Frameworks of Response
3.1 Perry’s Taxonomy
3.2 Dualistic Frameworks of Response
3.3 Taxonomising a Middle Ground
3.4 Frameworks Linked to a Theoretical Orientation
4. Conclusion
Chapter Three: Research: Response to Student Writing
1. Procedure
2. Reviews of Research Findings
3. ‘Appropriation’ of Student Texts
4. Error
4.1 Towards a More Balanced View of Error
4.2 The Effects of Error Correction
4.3 Lecturers’ Attitudes Towards Error
4.4 Students’ Perspectives on Error
4.5 Attitudes Towards Error of Content-Subject Specialists
5. Content Versus Form
6. The Effect of Praise
7. Reader Roles
8. Grades
9. Lecturers’ Commentary
9.1 The Nature of Lecturers’ Commentary
9.2 Length of Feedback
10. South African Studies
11. Conclusion
Chapter Four: Empirical Study: The Effect of Marking Strategies
1. The Marking Strategies Tested
1.1 Method A: Correction Code (CC)- Direct Method
1.2 Method B: Minimal Marking (MM) – Indirect Method
1.3 Method C: Mark Only- Control Group (CG)
1.4 Method D: Audiotaped Response (TR)
1.5 Method E: Self Assessment (SA)
1.6 The Markers’ Opinions
2. The Empirical Study
2.1 The Aim
2.2 Hypotheses Tested
2.3 The ESL Composition Profile
2.4 Procedure
2.4.1 The Original Paragraph
2.4.2 The Marking of the Original Paragraph
2.4.3 Inter-Rater Reliability Study
2.4.4 Marking of the Revised Paragraphs
2.4.5 The Second Marking of Both the Original
and Revised Paragraphs
2.5 Findings
2.5.1 Levels of Improvement in the Revised Paragraphs
2.5.2 Content Improvement in the Revised Paragraphs
2.5.3 Form Improvement in the Revised Paragraphs
2.5.4 Total Improvement in the Revised Paragraphs
2.5.5 Content I Form Improvement
2.5.6 Proficiency Group Variations
2.5.7 Codes Allocated by Markers
2.5.8 Statistics Omitting Students who Submitted Virtually
Identical Revised Paragraphs
2.6 Limitations of the Study
2.7 Conclusion
3. Appendix to Chapter Four
3.1 Adaptation of the ESL Composition Profile
3.2 Instructions for the Marking of Assignment 01: The Original Paragraph
3.3 Self Assessment Tutorial Letter
3.4 Instructions for the Marking of Assignment 04:
The Revised Paragraph
3.5 Instructions to Markers for the Second Examining
of Research Scripts
Chapter Five: The Students’ Perspectives
1.Closed Research Questions
1. 1 General Information
1.2 How do Students View the Mark and Tutorial Commentary?
1.3 How do Students Perceive the Lecturers’ Role?
1.4 Who Takes Responsibility for Improvement?
1.5 Is There a Gap Between the Feedback Students Want and
the Feedback They Receive?
1. 6 How do Students Respond to Feedback?
1.7 How Open are Students to Alternative Forms of Feedback?
1.8 How Theoretically Sound are Students’ Perceptions About Writing?
1. 9 How do Students Rank the Five Marking Strategies?
1. 9 .1 Ranking by Students With Experience of the Marking Method
1. 9 .2 Ranking by All Students
2. Open-ended Research Questions
2.1 How Useful is the Lecturers’ Commentary?
2.2 How Do Students Define the Term ‘Revision’?
2.3 How Do Students React to the Various Marking Strategies?
3. Conclusion
Chapter Six: The Readers’ Perspectives
1. Analysis of Global Commentary
1. 1 The Sample Group
1.2 The Parsing of Responses
1.3 Developing a Taxonomy for Analysis of Response Styles
1. 3. 1 Axiological Orientation
1.3.2 Focus of Commentary
1. 3. 3 Locus of Control
1.3 .4 Cognitive Level
1.3.5 Focal Point
1. 3. 6 Mode of Response
1.3.7 Grammatical Structure of the Response
1. 3. 8 Positive I Negative
1.3.9 Global I Local Sequence
1.3.10 Length
1.3.11 Focus
1.3.12 Overall Purpose
1. 3 .13 Reader Impression I Reader Roles
1.3.14 Form I Content
1.4 Findings
1.4.1 Axiological Orientation
1.4.2 Focus of Commentary
1.4.3 Locus of Control
1.4.4 Cognitive Level
1.4.5 Focal Point
1.4.6 Mode ofResponse
1.4.7 Grammatical Structure of the Response
1.4.8 Positive I Negative
1.4. 9 Global I Local Sequence
1.4. 10 Length
1.4.11 Focus
1.4.12 Overall Purpose
1.4 .13 Reader Impression I Reader Roles
1.4.14 Form I Content
1. 4. 15 Conclusion
1. 5 Individual Marking Styles
1. 5 .1 First Example
1.5.2 Second Example
1.6 Samples of Various Approaches
1.6.1 Marking as Editing -Response Reduced to a Handwriting Exercise
1.6.2 Extreme Formalism
1.6.3 Question in Emotive ParadigmFormalist Response Style
1. 6. 4 Conflicting Instructions
1.6.5 Responding Within the Cognitive Paradigm
2. Conclusion
Chapter Seven: Conclusions and Recommendations
1.What is happening now?
2.In what sense is the situation problematic?
3. What can be done about it?
3. 1 Audience
3.1.1 Self Assessment
3.1.2 Peer Evaluation
3.1.3 Unisa Learning Centres
3.1.4 Study Groups Throughout the Country
3.2 Transparency
3.3 Ownership
3.4 Innovative Approaches
3.5 Appropriacy of Feedback
3.6 Training
3.7 Integrating Assessment and the Teaching Programme
3.8 Enhancing the Marking Code
3.9 Breaking the Anonymity
4. Conclusion
Addenda
Addendum 1: Assignment 01 of 1995
Addendum2: 1986 Revision Assignment
Addendum 3: Marking Codes: Tutorial Letter JOI
Addendum4: Error Awareness Sheet
Addendum 5: Markers’ Questionnaire
Addendum 6: Reliability Study 1
Addendum 7: Reliability Study 2
Addendum 8: Reliability Study 3
Addendum 9: Student Questionnaire
Addendum 10: Sample A3 Script
Bibliography