SILC Showcase

Showcase July 2011: Effectiveness of Student-Constructed Diagrams and Self-Explanation Instruction

Share this article using our bitly.com url: http://bit.ly/1m6zehZ

Effectiveness of Student-Constructed Diagrams and Self-Explanation Instruction  Open .pdf document

Jennifer Cromley, Nora Newcombe (PI), Bradley Bergey, Shannon Fitzhugh, & Theodore W. Wills

Temple University

 

For the archival version of this research, and the preferred citation, see:

  • Cromley, J. G., Bergey, B. W., Fitzhugh, S., Newcombe, N., Wills, T. W., Shipley, T. F., & Tanaka, J. C. (2013). Effects of three diagram instruction methods on transfer of diagram comprehension skills: The critical role of inference while learning. Learning and Instruction, 26, 45-58. [DOI]

Abstract

Fostering student self-explanation in drawings and teaching students to construct their own drawings have both been tested and shown to be effective methods for increasing comprehension of figures in science text. However, it is unclear whether student-constructed drawing treatments are beneficial because they are generative, because they are visual, or because they increase inferential processing. We compared a student-constructed visual (SCD-Vis) treatment to a student-constructed verbal (SCD-Verb) treatment and a self-explanation (SelfExpl) treatment with 137 high school biology students. Results suggest equal, significant, positive effects of all treatments on biology diagram comprehension, but significantly less growth in biology knowledge and geoscience diagram comprehension (transfer) for the SCD-Vis treatment. All treatments were associated with increases in literal and inferential biology diagram comprehension. Our results suggest that the benefits of student-constructed drawings found in prior research are likely due to increasing inferential activity rather than due to generativity per se. Contrary to expectations, both self-explanation and student-constructed verbal treatments show benefits on posttests that use a verbal modality and on posttests that combine verbal and visual modalities. Findings suggest that simultaneously building biology knowledge and encouraging inference-making during diagram comprehension underlies the greater success of these two treatments. We close with implications for theory and instruction.

Purposes

Students in science classes are faced with more than one visual representation per textbook page, yet research consistently finds poor comprehension of these diagrams, graphs, maps, photographs, and tables (Canham & Hegarty, 2010; Florax & Ploetzner, 2010; Seufert, Schutze & Brunken, 2009). Instruction in comprehension of visuals therefore seems warranted as a means to increase student learning of important science topics.

Teaching students to construct their own drawings and fostering student self-explanation in drawings have both been tested and shown to be effective methods for increasing comprehension of figures in science text. However, it is unclear whether student-constructed drawing (SCD) treatments are beneficial a) because they are generative, b) because SCD interventions are visual interventions whose effectiveness is tested with visual posttests, and/or c) because they foster inferential processing. Our purpose was therefore to compare visual highly-generative (Student-Constructed Diagrams-Visual), verbal highly-generative (Student-Constructed-Diagrams-Verbal), and verbal low-generative (self-xplanation) interventions in order to test hypotheses related to matched visual modality, generativity, and inferential processing.

Theoretical Framework

Student-constructed drawing is thought to be effective because of the generative nature of learning (Wittrock, 1974); in creating a new representation in the course of learning, rather than simply processing a representation created by another (as in self-explanation), students should develop a deeper understanding of concepts and better learn and remember the material. Students who are instructed to make their own drawings while reading often—but not always—show better comprehension of scientific text than uninstructed students (van Meter & Garner, 2005). Self-explanationis a multifaceted activity that includes gap-filling inferences, bridging inferences, knowledge elaboration, metacognitive monitoring, and fix-up strategies (Roy & Chi, 2005). Students who are prompted to self-explain while learning from diagrams show better comprehension than students not prompted to self-explain (Ainsworth & Th Loizu, 2003).

In both the generativity hypothesis and the self-explanation paradigm, students must actively engage with the to-be-learned information and make inferences to connect information in memory with to-be-learned information. One critical difference, however, between self-explanation and student-constructed drawings (SCD) as a means for learning from visual representations, is that the former is an almost entirely verbal process whereas the latter is almost entirely visual. This has important implications for design of future interventions in classrooms, since results could point towards a focus on generativity or towards a focus on instruction in visual skills.

Hypotheses from the generativity hypothesis. We expected that if the benefit of student-constructed diagrams is due to generativity, the Student-Constructed Diagrams (SCD) Verbal and Visual conditions would show the same results on student learning. Specifically, both SCD conditions should outscore SelfExpl on knowledge gains and inferential diagram questions because according to the generativity hypothesis the creation of a new representation is more generative than the creation of a written explanation.

Hypotheses from modality match. A different type of explanation for the success of SCD interventions is that students create a visual representation and are then tested on visual representations (i.e., diagram comprehension items). We refer to this as modality match hypothesis A. An alternative modality match hypothesis is that students on the posttest answer verbal questions that ask them to make sense of visual material. Modality match hypothesis B therefore predicts that the verbal instructional conditions should outscore the visual instructional condition. The predicted pattern of results is the same for biology knowledge, biology diagrams, and geoscience diagrams.

Hypotheses from inferential processing. A third alternative explanation is that both SCD and SelfExpl interventions increase inferential processing. If this is equally true for all interventions, then all three treatments should show effects on inferential measures at posttest. However, as reviewed by Van Meter and Garner (2005), SCD may show no effects on literal comprehension.

Method

Research design. We used a quasi-experimental design with random assignment of 12 classes within teachers to the three conditions.
Participants were 143 students in 12 intact 9th grade biology classes, Mage = 14.5, 60% female, 59% White, 17% Black, 14% Hispanic, 10% other/mixed race. Median maternal and paternal education was a high school diploma. This was a relatively low-achieving district, as 37% of students tested partially-proficient (i.e., Below Basic + Basic) on a low-stakes statewide science exam and 23% below basic on a high-stakes reading comprehension exam.
Measures for students included demographics, biology knowledge, geoscience background knowledge, biology diagram comprehension, and geoscience diagram comprehension. See Table 1 (below) for the number of items, reliability and validity data.
Fidelity of Implementation was measured with a researcher-developed checklist based on the Pressley and Harris (2006) strategy instruction model.
Intervention Materials consisted of one of three workbooks (Self-explanation in diagrams, Student-Constructed Diagrams Verbal and Student-Constructed Diagrams Visual), which students completed in pairs with teacher scaffolding, followed by class-wide discussion. Every diagram from 4 chapters in the students’ textbook (Postlethwait & Hopson, 2006) was scanned, and we wrote instructions for the three conditions. The topics covered in the 4 chapters were: DNA structure, protein synthesis, and inheritance patterns, and DNA technology.

Figure 1

Figure 2

Figure 3

For pragmatic reasons related to research in intact classes, and in consultation with the teachers at the school, we made slight modifications to procedures that are typically used in lab settings. For the SCD conditions, rather than giving students a blank sheet of paper, we provided partially-completed figures (with either parts of the figure or with words deleted) and asked students to complete them. For the self-explanation condition, we created a confused character (a guinea pig), who verbalized an incomplete or incorrect explanation of what was shown in the figure and asked students to discuss and then write down a better explanation in the workbooks.

Procedures. After obtaining parent/guardian consent, student assent, and student demographics, we pretested students in intact classes in single, whole-group sessions of about 45 min on the 2 measures of science knowledge and 2 measures of diagram comprehension. The intervention was implemented over 6 weeks. Students received a copy of the workbook in which they wrote answers to the workbook questions. Workbooks were stored in the classroom between class meetings to control for student exposure to the workbooks. The teacher provided his or her usual instruction and introduced each workbook exercise whenever he or she reached the corresponding image in the textbook. Students worked in pairs on the worksheets (26 to 28 double-page exercises total), with scaffolding available from the teacher, for approximately 5 minutes. This was followed by teacher discussion of and feedback on student answers, lasting another 5 minutes. Students were posttested on the same biology knowledge and diagram comprehension measures after 27 school days of daily instruction.
Data scoring and analysis. Student workbooks were scored for the maximum level of inference demanded by the workbook questions (SCD-Vis < SCD-Verb < Self-Expl) and the level of inference actually engaged by the students as a proportion of possible (SCD-Verb > SCD-Vis = Self-Expl). Interrater reliability for workbook scoring was high, Cohen’s kappa = .95 (exact agreement = 96%).

Screening showed no violation of assumptions of normality (univariate or multivariate), linearity, or independence of observations within classrooms, nor were any outliers detected. Since the ICC for pairs was low (< .05), no adjustments were needed to protect against risk of Type I error. Analyses used 2 (Time) x 3 (Condition) repeated-measures ANOVAs evaluated at an alpha level of p < .05.

Results

Statistical results are presented in Table 2 (see below).

For biology knowledge, students in the SelfExplanation and SCD-Verb condition showed significant growth in biology knowledge from pre- to posttest, but the SCD-Vis group showed no growth. Effect sizes for growth from pre- to posttest were d = .63, .04, and .68, for SCD-Verb, SCD-Vis, and SelfExpl, respectively. These findings are consistent with the treatment-testing modality match hypothesis, which predicted larger gains on a verbal posttest from the two verbal conditions, compared to the visual condition. These findings are not consistent with the generativity hypothesis, which predicted equal gains from the two generative conditions (SCD-Verb and SCD-Vis).

Chart 1  Chart 2
a) Growth in biology knowledge                                         b) Growth in biology diagram scores

Chart 3
c) Growth in geoscience diagram (transfer) scores

For biology diagram comprehension overall scores, students in all three conditions showed significant growth. Effect sizes for growth from pre- to posttest were d = .30, .32, and .22, for SCD-Verb, SCD-Vis, and SelfExpl, respectively. For biology diagram literal subscale scores, students in all three conditions showed significant growth. Effect sizes for growth from pre- to posttest were d = .32, .27, and .17, respectively. For biology diagram inferential subscale scores, students in all three conditions showed significant growth. Effect sizes for growth from pre- to posttest were d = .32, .21, and .24, respectively. Effects were the same for literal and inferential subscales as for the measure as a whole, supporting the inference hypothesis. These findings for the biology diagram measure are inconsistent with the treatment-testing modality match hypothesis, which predicted smaller gains on a visual posttest from the two verbal conditions, compared to the visual condition. The findings are, however, consistent with the generativity hypothesis, which predicted equal gains from the two generative conditions (SCD-Verb and SCD-Vis). With respect to our hypothesis regarding inference, it appears that all three treatments do increase scores on the inferential subscale of diagram comprehension.

For geoscience diagram scores, students in the SCD-Verbal and SelfExpl conditions showed significant growth in geoscience diagram comprehension. Effect sizes for growth from pre- to posttest were d = .37, <.01, and .28 for SCD-Verb, SCD-Vis, and SelfExplanation, respectively. These findings are inconsistent with the treatment-testing modality match hypothesis, which predicted smaller gains on a visual posttest from the two verbal conditions, compared to the visual condition. These findings are also inconsistent with the generativity hypothesis, which predicted equal gains from the two generative conditions (SCD-Verb and SCD-Vis).

Discussion

Implications for theory. The focus of our research was on comparing effects of self-explanation—a low-generative verbal technique which prompts inferences—to student-constructed verbal completion—a high-generative, verbal technique—to student-constructed visual completion—a high-generative, visual technique. We hypothesized that if the benefit of student-constructed diagrams is due to generativity, the Student-Constructed Diagrams Verbal and Visual conditions would show the same results on student learning. The two conditions showed equal effects only for the biology diagram subscales, but not for biology knowledge or geoscience diagram comprehension (far transfer). We also hypothesized that if the benefit of student-constructed diagrams is the visual nature of both the instruction and the posttest measure, then Student-Constructed Diagrams Visual would outscore both the Student-Constructed Diagrams Verbal and the Self-explanation treatment on visual posttests. This hypothesis was supported only for the biology knowledge measure, suggesting that it is not a tenable explanation for effects of diagram comprehension instruction. Finally, we hypothesized that if inference is the key to the success of both self-explanation and student-constructed diagram approaches, then all three treatments should show effects on inferential measures at posttest, but might not show effects on literal diagram comprehension items. This hypothesis was supported by data from the inferential subscale of the biology diagram comprehension measure, which showed equal, significant growth across all three conditions. However, we saw equal, significant gains on literal items as well. Overall, none of the findings are consistent with the generativity hypothesis. The common success of SCD Verbal and Self-explanation is most consistent with Inferential processing. Consistent with the self-explanation model, increasing student inferences—whether with SCD-Verbal or Self-explanation—can yield increases in knowledge, literal and inferential diagram scores, and transfer to diagram comprehension in an uninstructed domain.

Implications for practice. Our results suggest that medium-to-low ability students can benefit from direct instruction in diagram comprehension, but this instruction is lacking in most classrooms (Roth, et al., 2006). Despite the popular notion of students being “visually literate,” these high school students were not very skilled at learning from visual representations, and became more skilled once they were taught how to reason with visuals. Regarding student-constructed drawing, the workbook scores suggest that our treatment did not demand as much inferential activity—and students did not engage in as much inferential activity—compared to the other treatments, and this may explain the lack of growth in biology knowledge and the geoscience diagram transfer measure. Future interventions should increase the cognitive demand of student-constructed drawing conditions. In all conditions, teacher scaffolding of student learning was a prominent component, consistent with Van Meter and Garner's (2005) literature review. Teachers need to be aware of the specific difficulties that students may have with each textbook figure, and this takes some effort on the part of curriculum developers and teachers. Finally, classroom discussion was an integral part of the intervention; the discussion in pairs during workbook completion was complemented by group discussion, where some students updated their answers after reflecting on the discussion. Researchers and schools wishing to implement such interventions should be sensitive to the need for curriculum materials, teacher professional development, and time for both workbook completion and classroom discussion.

Table 1 (Power Point Slide)

 

Table 2 (Power Point Slide)

You are here: SILC Home Page SILC Showcase Showcase July 2011: Effectiveness of Student-Constructed Diagrams and Self-Explanation Instruction