Effective Access: Teachers’ Use of Digital Resources

(Research in Progress)

Bethany Carlson and Sharon Reidy
Gender, Diversities & Technology Institute at EDC
Education Development Center, Inc.

ABSTRACT

In this paper, the authors describe the need for a better understanding of the cognitive frameworks used by high school science, technology, engineering, and mathematics (STEM) educators as they search for digital resources and conceptualize the integration of digital resources into their teaching. We outline our research methodologies, summarize preliminary results, and discuss how our outcomes will influence how digital resources are developed for educators across all disciplines in the future.

AUTHORS

Bethany Carlson is a Research Associate at the Gender, Diversities, and Technology Institute at EDC.  She brings experience in both engineering and urban middle school science classrooms to her research. Ms. Carlson has conducted extensive focus groups and interviews with educators on technology use and school reform, and has led participatory research sessions with middle school aged children. She has also written business and technology lessons for high school students. Her current projects include two NSF funded digital library projects, Effective Access and CaREN: Career Resources Education Network.

Sharon Reidy is a Technology Associate at the Gender, Diversities, and Technology Institute EDC. Her work on design and development of   digital  libraries  which  encourage  the participation of underrepresented  populations  in  science,  technology,  engineering,  and mathematics education and careers.  Projects include two NSF funded digital libraries, the Gender and Science Digital Library (www.gsdl.org) and CaREN: Career Resources Education  Network.   Previously, Ms. Reidy worked in informal education and as an instructional designer.  She earned her Ed.M. in Technology in Education from Harvard's Graduate School of Education.

Keywords: User Centered Design,  End  User, Digital  Resource, Digital  Library, Educational  Technology, Usability Evaluation, Usability, Interface Design, K-12 Teachers, STEM, Science, Technology, Engineering, Mathematics, Developers, Educators, Information Storage and Retrieval

The overwhelming volume of resources available on-line can make finding resources with high-quality content and features suitable to the K-12 environment nearly impossible.  Digital libraries offer a way to bring scattered digital resources together in a coherent and accessible framework [1].  However, most digital collections and search mechanisms were not created for K-12 users and present a number of challenges for educators in both formal and informal settings.  These challenges include the overwhelming variety and scope of the collections; the fragmentary nature of material; search engines not designed for average users; teachers’ lack of experience in using non-textual resource; and the ongoing struggle between in-depth inquiry and curriculum breadth [2].  Teachers cite inappropriate materials and the lack of knowledge about using the Web effectively as further obstacles to using the Internet to find digital resources [3].  A study of library media specialists showed that their mental models of teaching with digital resources are often based on their models of information retrieval and teaching with print resources [4].  We hypothesize that this behavior is true for other groups of educators.  In order to develop effective digital resources and search mechanisms, developers need to better understand and apply educators’ mental models for retrieval and integration of these materials.

Educational digital libraries are making progress with respect to accessibility.  The National Science Digital Library’s (NSDL) K-12 task force is working to bring the needs of teachers and students into discussions about this collection.  In addition, experiments in interfaces and navigation tools such as “Science Guy”, a type of mascot-avatar-agent for the NSDL, currently being explored by the Syracuse Virtual Reference Team, or the searching aids on the MERLOT Web site are the beginning point for the development of comprehensive services to help K-12 educators search for appropriate digital materials.  The Effective Access project is furthering this work by paying particular attention to the self-perceived needs of teachers who are struggling with the difficulties of selecting quality materials and tailoring those materials to meet the needs of their diverse student populations.  The project has initiated an examination of the cognitive frameworks used by teachers in order to understand how they search for digital resources and how they conceptualize the integration of digital resources into their teaching.  In conjunction with a parallel study of the cognitive frameworks used by developers in the creation of digital resources, Effective Access hopes to inform the future of how digital materials are developed and distributed to educators.

Effective Access research focuses on high school level science, technology, engineering, and mathematics (STEM) educators, as their subject areas are often the focus of education reform. Digital libraries can offer these educators access to a multitude of digital resources to build student interest and achievement in STEM.

FOCUS ON DIVERSITY

Voices, perspectives, and interests of women, people of color, and other under-represented groups are not often reflected in the design and development of Web-based educational resources since the majority of those working in the Internet world are white and male[5]. Finding resources that reflect and support inclusive and equitable education with high expectations for all students continues to be a critical component in teaching and learning.  Lessons about inclusive and equitable STEM education, learned through projects such as the Gender and STEM Digital Library (GSDL), and the Gender Education in Mathematics and Science (GEMS) research at the Education Development Center Inc., are informing Effective Access research.  This research explores how male and female educators from different racial/ethnic groups perceive and use digital resources and how they use diversity as a lens to align resources with their students’ needs.  By examining these issues, we hope to stimulate the development of educational tools that meet diverse needs, interests, and learning styles of our heterogeneous society.

METHODOLOGY
Research Questions

At the highest level, our research questions are:

• What are STEM educators doing with educational resources and what would they like to be doing?
• What kind of technical and educational supports do educators want and need?
• How do teachers find Web-based resources?
• What design features meet the needs of STEM educators?
• What is the relationship of gender/race/ethnicity to educators’ perceptions and use of digital resources?

Data Collection

In order to answer our highest level questions, we began with an initial review of related project evaluations and existing research. This served as a basis for the conceptualization and development of a survey for high school level STEM educators and a separate survey for developers of Web-based educational resources. The two surveys are aligned in order to elucidate the discrepancies and parallels between developers’ understandings of teachers’ needs and what teachers express as their needs. The surveys were created on-line using a Web-based survey tool. Several question formats were used including: multiple choice, rating scales, short-answer, and single choice.

We designed focus group protocols in order to delve further into the needs of educators. The guiding question of the educator focus groups is “How do you FIND Web resources and INTEGRATE those resources into your teaching?” Participants also work together to describe ideal features and services of a STEM site, and they reflect on how Web resources could better meet the needs of individual students. Throughout the focus groups, we monitor the influences of gender and race (of the participants and of their students) on participant responses.

Since our survey is primarily Web-based, teachers who do not use the Internet for professional activities are likely to be excluded. Focus groups and interviews can elicit the opinions of these teachers, and our first focus group did just that. Focus groups will be followed by in-depth interviews or case studies with educators that will include Human-Computer Interaction research and explorations of various digital resources. Data will be triangulated to confirm key findings.

Participants

On-line and paper versions of the educators’ survey have been piloted and refined, and are now being distributed through a number of EDC projects, including the Women’s Education Equity Act (WEEA) Resource Center, Center for Science Education, and other groups that will give us a broad distribution of diverse educators in different settings, and with various levels of technology capacity. Additional surveys will be distributed through NSDL projects, NSTA, and NCTM until we have about 700 responses. Developers will be sought through the NSDL and professional organizations for technology developers.

PRELIMINARY RESULTS

A preliminary analysis of the first 25 survey responses from subjects who are currently teaching a STEM subject to 14-18 year olds has been completed. Although limited in scope, the responses of these first survey takers have revealed similarities which have provided some insight into our aforementioned research questions and will guide us through the remainder of our research. Survey respondents are being actively solicited and will continue to be recruited until we have reached our goal sample size of 700 educators. We predict that some of the trends which have emerged during our initial analysis will carry through this research project; however, the small sample size used in our preliminary analysis cannot in any way be generalized to the teaching population as a whole.

What are STEM educators doing with educational resources and what would they like to be doing?

Survey takers were asked a variety of questions about their use of educational resources for curriculum planning and for use during instruction. After answering a series of questions about their general use of educational resources, respondents were asked to focus on their use of Web-based resources. All of the subjects in this sample have sought out educational resources on the Web. All of the respondents “sometimes” (42%), “frequently” (37%) or “always” (21%) use Web sites or electronic documents for background information during their curriculum planning. Sixty-nine percent of participants use on-line lesson plans or tutorials “sometimes,” “frequently,” or “always”; 32% never use these resources for their planning. When asked to choose the two resources that they use most for curriculum planning, from a given list, a majority of educators chose textbooks (61%) and Web sites (61%).

However, a majority of educators (84%) spend less than 50% of their time using Web-based resources during instruction. Sixty-three percent spend less than 25% of their instructional time incorporating Web-based resources. Educators that do use Web-based resources during instruction use on-line articles, downloadable handouts, and on-line activities (e.g. simulations or virtual manipulatives) most frequently. (see chart 1) Half of the respondents (n=18) indicated that they typically customize resources to suit their needs based on time constraints, the needs of their special education students, or to supplement the on-line resources. When asked to choose the two resources that they use most during instruction, the majority again chose textbooks and Web sites.

Chart 1 Use of Digital Resources

Educators use textbooks and Web sites most frequently during instruction, yet 63% of respondents use Web-based resources during 0-25% of their instructional time. One may conclude that in general, high school level STEM educators use textbooks most frequently during instruction. Two of the three Web-based resources that educators use most frequently (e.g. on-line articles, downloadable handouts) are very similar in nature to textbooks. These Web-based resources may be used most often because of the comfort level educators have with text based resources, and the fact that articles and handouts downloaded from the Web afford more up-to-date information (one of the top reasons mentioned for why educators value educational resources) than textbooks. These results reflect the stages of technology integration that other researchers such as Sandholtz et al. have outlined through their research with the Apple Classrooms of Tomorrow [6]. Focus groups and interviews will be used to clarify how educators are actually using Web-based resources in their classroom. For instance, are educators downloading and printing articles and handouts before class, or are students actually responsible for finding these resources during class time? Do students react differently to resources they are told were downloaded from the Web, or would educators be better served to simply use newspapers and magazines for up-to-date information?

However, 78% (n=18) of participants indicated that Web-based educational resources have changed the way they plan or structure their teaching. Educators commented that they were “…able to get real data for statistics that is meaningful to the students, ”that using the teaching ideas and background information available through Web-based resources made their work more efficient, and that they were able to access tutorials and simulations that interested their students.

When asked what instructional tools they would most like to find on a Web site devoted to educational resources, 65% (n=17) responded that they would most like to see “lesson plans and activity ideas; 53% indicated that they would like “raw data;” 47% selected “pictures and graphics,” and 41% choose “simulations.” Again, this data suggests that our survey sample is looking for Web-based resources that replicate what they could find through newspapers, magazines, and textbooks. The use of the Internet for these resources may be more a product of educators’ lack of time to look through text based resources (although they complain about the amount of time it takes to locate Web-based resources) than of the affordances of Web-based resources.

Of the 10 focus group participants, eight used the Internet for lesson planning and activities at least some of the time, while two never used the Internet in their teaching. The teachers who used the Internet cited the abundance of information available as their main reason. The two teachers who did not use the Web each gave a different reason; one teacher claimed lack of knowledge and skills around Internet use, and the other said that the curriculum had no room or need for extra resources.

The teachers mentioned problem sets, interactive simulations and sets of real-life data as three specific Web resources they value for their mathematics classes. Teachers who frequently use these types of resources wished that they could find a greater variety on-line.

Even though interactive Web activities were mentioned often during the discussion, the “ideal” resource that the focus group participants described supports the idea that teachers are using Web resources in the same manner as they would print materials. They envisioned a single mathematics site, divided into sections by math concept and student skill level, which could generate problem sets to support a lesson topic.

What kind of technical and educational supports do educators want and need?

Two-thirds (67%) of participants (n=18) are most often supported by media specialists and/or technology coordinators when they require technical or curriculum integration support for using Web-based educational resources. None of the participants listed “Curriculum Specialist” as one of the three most supportive staff members who provide technical or curriculum integration support. It is not clear if all participants have access to a curriculum coordinator to begin with. However, participants may not feel that their curriculum coordinator is technically savvy enough to help them integrate Web-based activities and simulations into their curriculum. Since our survey sample indicated that downloadable handouts and on-line articles are frequently used in the classroom, educators may not feel that they need the support to integrate these resources into their teaching. Support may be a function of technical problems which require the assistance of seemingly more technically savvy media specialists or technology coordinators.

Chart 2 Professional Development Topics

More than half of the respondents have participated in professional development sessions that focus on using the Web to find educational resources. The majority found this to be a useful topic. Just under half of the participants have had some professional development in using technology during STEM instruction. Of those who have not taken such a course, the majority would be interested in doing so. A majority of participants have not taken professional development in equity in STEM education and are not interested in doing so.

How do teachers find Web-based resources?

When asked to identify how they typically find out about new educational resources, respondents (n=23) answered in the following manner:

78% Professional Development Sessions/ Conferences
70% Colleagues
65% On-line sources
65% Professional Journals/Readings
48% Newsletters

The majority (70%) of participants collaborate with their colleagues to share resources once a week or more. Of those participants, 32% are sharing resources daily.

When asked specifically how they find Web-based educational resources, all of the respondents (n=19) stated that they use search engines. Ninety-five percent of subjects indicated that they followed links from other Web sites, while 79% indicated that they learned about particular Web-based educational resources from colleagues or professional development sessions/ conferences.

The majority (5 out of 8) of the focus group participants who use the Internet regularly for their teaching said that the time required to find usable/appropriate resources on-line was their biggest frustration. They characterized their search for Web resources as always having to go from site to site rather than being able to find resources in one place; they begin their searches using a search engine (Yahoo or Google) or at a favorite educational site (NCTM or Math.com). Three teachers were frustrated most by difficulties getting computer access for their class, either because of scarce computer lab availability or because of frequent equipment breakdowns.

When reflecting on finding resources appropriate for individual students, the focus group teachers most often described the Web as a source of supplemental practice problems for a struggling student and/or as a way to increase student engagement.

What design features meet the needs of STEM educators?

The design features educators would most like to find on a Web site for educational resources include searching capabilities, methods for submitting questions, links to related resources, and assurance that the Web site is supported by a reputable source. These responses are not surprising in light of the fact that 71% of respondents indicated that the top challenge they face in seeking in using Web-based resources was the great amount of time it takes to locate appropriate resources on-line.

Design features most commonly associated with Universal Design for Learning, including audio options, text-only options, and adjustable font size and color, were among the design features that educators indicated that they wanted least. This may be due to this sample’s lack of familiarity with Universal Design features: 87% of this sample had not had any professional development about equity issues in technology.

What is the relationship of gender/race/ethnicity to educators’ perceptions and use of digital resources?

The demographic make-up of our initial survey sample makes it impossible for us to draw any conclusions regarding educators’ perceptions of gender, race, or ethnicity. Ninety-six percent of this survey sample (n=25) were Caucasian and 76% worked with predominantly Caucasian students. We are addressing this lack of diversity in our efforts to recruit additional subjects. Survey announcements have been sent to areas of the country with large concentrations of diverse populations such as Chicago, Los Angeles, and areas of the southern United States. Sixty-one percent of survey takers were female, and our next steps include disaggregating survey data by gender. However, since “Nearly three out of every four public school teachers are female, and 89 percent are white, whereas only 7 percent are black and 2 percent are Hispanic” [7] we see this as an on-going challenge.

The 10 focus group participants all were high school mathematics teachers who ranged in experience from one year to over 20. 90% taught in rural schools. 90% were white, and 60% were female. Although we tracked issues surrounding race and gender of the teachers and students in this focus group, no trends emerged from such a small sample. As we conduct more focus groups across the country, we will continue to analyze the resulting data with a race and gender lens.

IMPLICATIONS FOR THE FUTURE

Through our investigation of how teachers search for and use educational resources, we hope to create a bridge between the needs of teachers and the work of developers. We anticipate that some perceptions held by developers about teachers’ technological fluency and use of Web-based resources will be very similar to what teachers express. Other ideas will be glaringly different. By synthesizing the similarities and differences among the teachers’ and developers’ responses, we will create a framework for developing effective digital resources. This framework will help developers create Web-based educational resources that better meet the needs of STEM teachers in high schools and informal education settings.

Ultimately, this research will improve the ways in which STEM educators and students can access and use digital materials. The NSDL and the Core Integration project have already begun to focus on the needs of K-12 teachers as an important and under-served audience. Effective Access will provide leadership in that effort: the comprehensive frameworks and guidelines developed as a result of this research will shape the future work of digital libraries as developers attempt to meet the unique needs of K-12 teachers.

Acknowledgements: This project is a collaborative effort between two centers at the Education Development Center Inc: The Gender and Diversities Institute at the Center for Education, Employment, and Community and the Center for Online Professional Education. The principal investigators are Katherine Hanson and Pamela Buffington. Yael Sucher, formerly of EDC, made significant contributions to this project and paper. This project is supported by the National Science Foundation under grant No. DUE-0226483. Opinions, findings, or recommendations expressed are those of the authors and do not necessarily reflect the views of the NSF. This work is part of an ongoing collaborative to build a National STEM Education Digital Library at nsdl.org.

REFERENCES

[1] Examples of digital resources include electronic text, virtual applets, audio clips, video, digital images, and animations.
[2] Hawkins, J. Panush, E.M. & Spielvogel, R. National Study Tour of District Technology Integration (Summary Report). New York: Center for Children and Technology, Education Development Center, December 1999.
[3] Net Day. The Internet, Technology, and Teachers: Survey Results, May 2001. Available at http://www.netday.org
[4] Tallman, J. and Henderson, L. “Constructing Mental Model Paradigms for Teaching Electronic Resources” 1999. Available at http://www.ala.org/aasl/SLMR/vol2/mental.html
[5] ITAA Task Force Reports. Building the 21st Century Information Technology Workforce: Underrepresented Groups in the Information Technology Workforce. Available at http://www.itaa.org/workforce/studies/recruit.htm
[6] Sandholtz, J. Ringstaff, C. Dwyer, D. Teaching with Technology: Creating Student-Centered Classrooms 1997.
[7] Feistritzer, C. E. Profile of teachers in the U.S. Washington, DC: National Center for Education Information, 1996.