ADL Newsletter for Educators and Educational Researchers

Academic
ADL Co-Lab logo

Advanced Distributed Learning for Educators and Educational Researchers

September 2007

IN THIS ISSUE

The Knowledge Economy, SCORM, and Design-Based Research

Postscript to a Debate Between Constructivists and Supporters of Explicit Instruction

Improving Courses
Traci Sitzmann

EDITOR'S CORNER

A stimulating think piece by Michael Bush is the lead article in this issue of the Newsletter (No. 6). For background on the Sharable Content Object Reference Model (SCORM), referred to in the article, please consult Newsletter Issue 1. SCORM describes the conventions in Advanced Distributed (ADL) for instructional objects. A number of interesting ideas are discussed in the articles that go way beyond the specifications to make instructional objects SCORM conformant for ADL. We welcome discussions of the article, as well as responses, expansion, or disagreements with Michael’s views, so write on and send them to me at sig.tobias.ctr@adlnet.gov

Newsletter subscribers may recall that several scholars responded to a debate about the efficacy of constructivism in Issue 3 and Issue 4. We had a debate about the effectiveness of constructivism at the AERA convention last April, and some final thoughts about that controversy also appear in the present issue. Again, please submit discussions, responses, or disagreements with the views expressed so that we can consider publishing them.

Finally, in this issue Traci Sitzman adds a short notice offering to help trainers and instructional designers develop courses that maximize student learning and satisfaction. If interested, please contact her.

Sig
sig.tobias.ctr@adlnet.gov

 

 

Prior Articles

About CORDRA (Issue 4)

E Learning and ADL in Korea (Issue 2)

Effectiveness of Web Based Training (Issue 2)

Games, Learning, and Society Conference (Issue 3)

Introduction to ADL (Issue 1)

Kirschner, Sweller, Clark Paper discussion (Issue 3)

Multi Media Lab in Taiwan (Issue 2)

Newsletter Purpose (Issue 1)

Newsletter Archive

 

 

 

 

 

 

 

The Knowledge Economy, SCORM, and Design-Based Research

Michael Bush
Brigham Young University

The Knowledge Economy and SCORM

The concept of “knowledge economy” has been around for a few years as a means of providing a context for the idea that work force productivity in the Information Age is more mental than manual. Because knowledge implies learning, it is reasonable to assume that learning objects should play a role in such an economy. As a result, it is important to explore the standards and specifications of the Sharable Content Object Reference Model (SCORM), used to specify conventions in Advanced Distributed Learning, to help define the role of learning objects and the parameters within which this role is carried out.

It is assumed that understanding how these ideas fit within the “knowledge economy” paradigm and its connection to learning could provide a broad framework for understanding the interaction between instructional technology and various elements of education. In particular, it seems important to not only understand the potential already present but also to explore how to increase the impact of instructional technology.

Instructional Technology’s Potential

There is ample evidence that “instructional technology” can have a beneficial impact on learning outcomes (Bunderson & Abboud, 1971; Bunderson, et. al. 1984; Kulik & Kulik, 1987; and Fletcher, 2003). Such a short list of references provides only meager documentation for technological implementations occurring over the last 35 years. In 2000, the hype reached the pages of Business Week:

Enthusiasm is also running wild in the emerging dot.com education sector. ''There will be a tremendous migration away from classroom learning to online learning,'' predicts Howard Block, an analyst at Banc of America Securities. Some are even calling education the next ''killer app'' for the Internet (Symonds, 2000).

Unfortunately, the dot-com bubble burst not only for business but also for education. As is abundantly clear in a high percentage of classrooms, implementation of instructional technology has failed to meet expectations. Furthermore, questions concerning the effectiveness of technology-based learning still cast a shadow over its proven potential (Dynarski, 2007). To date, for classrooms and learners everywhere, it appears that the potential for effectiveness has regrettably been overshadowed by questions about educational technology.

The Gap between the Technologies of Leisure and Learning

While the use of technology for learning languishes, a glance through any publication from the popular press shows that technology for leisure surpasses all expectations. The disparity between what is seen in education and the available capabilities for computer-delivered entertainment is glaring. On the one hand, much of the leisure time of today’s world is filled with gadgets like iPod, Zune, and a seemingly infinite variety of MP3 players, not to mention the hot-and-connected iPhone, creating opportunities for content providers to develop incredible amounts of multimedia content, interactive and otherwise. On the other hand, education proceeds apace with the tried and true elements of its centuries-old model for delivery… teachers and textbooks, creating an astounding technology gap between leisure and learning. Indeed, minimal reflection upon the state of technology-enhanced education is more than enough to bewilder any self-respecting technology aficionado who wants to see students learn in the most cost-effective and efficient manner.

Defining “Knowledge Economy”

Explaining this situation is clearly more complex than can be found by even the best formulated Google search. An expansion of the concept of “knowledge economy” can perhaps provide a framework to understand where things are and how they might change for technology in learning and education. Hodgins (2002) connected the concepts of learning objects and standards to the vision of “knowledge economy,” providing a foundation for the examination of strategies for moving from vision to reality.

To elaborate on this notion and to make a useful examination of the principles involved possible, it is essential first to define some terms. In the most fundamental sense, an economy is a system of production, distribution, and consumption. In other words, an economy exists when producers make their goods and services available to consumers in some measurable quantity or package. Conventionally, exchange of the product is accompanied by some form of payment from the consumer to the producer, at a time and place amenable to both parties. The place where the transaction takes place has usually been defined as a market, but increasingly the Internet is the space where producers and consumers meet. Conventionally, payment takes the form of in kind trade or transfer of monetary value in compensation for the product that is then transferred to the consumer.

Increasingly, however, the exchange involves products comprising software or digital media. Payment might well happen conventionally with funds, but at times it takes the form of goodwill or enhanced reputation, as happens with open source software such as Linux. The amount paid is an essential element of the inherent market mechanism of the economy for evaluating the product’s value and quality. This evaluation method depends on the situation of the exchange, which in turn reflects the demand that exists for the products in the marketplace.

To define “knowledge economy,” it is essential to first identify those elements that exist in parallel economic models. An economy has sectors, and within the “knowledge economy,” the business, government, and formal education sectors would need to be considered in the context of a complete exploration of how learning objects fit into the big picture. The present discussion considers only the formal education sector, but several principles also apply to business and perhaps to self-learning. Possibly business and government will benefit from innovations in the production and delivery of learning content before other sectors, but education is the topic at hand.

The Challenge to the Educational Sector in a Knowledge Economy

In a simplified analysis, the producers in the formal education sector of the “knowledge economy” can typically be as viewed as publishers (with the help of teachers and professors), while teachers and professors (referred to as teachers from here on) are a central element of the delivery of learning content (product) to the students (the consumers) via the classroom. As reasonable as this might seem on the surface, it should be clear that teachers are part of the production side of the typical market transaction (production to distribution to consumption), part of the distribution system, AND part of the consumption side of the transaction

Because some teachers actually play a role in all three aspects of the educational sector of the knowledge economy, some would say that this creates a conflict of interest at several levels. At times they are part of the production side, helping to write textbooks, but they also assume some of the normal role of the typical consumer. As such, teachers make decisions about which specific products learners should acquire (textbooks) and the instruction to be received (assignments and classroom activities). At a minimum, such an arrangement creates a dynamic in the educational marketplace that is hard to quantify or explain, especially at the distribution to consumption side of the transaction.

Online learning is viewed by many teachers as a replacement for what is considered to be normal activity in education, therefore acquisition and instructional design decisions are often reduced to a simple question, “Will I choose to acquire something that will reduce my individual role in the system?” The answer of course is often, “No!” as witnessed by the failure of eLearning to make the progress in education that was anticipated by many instructional technology pioneers. This creates something of a conundrum, given that a tool’s proven value is virtually ignored in a sector where every possible bit of help is needed.

Addressing the Challenge with SCORM

Finding a solution to the problem involves considering the law of supply and demand, perhaps the most fundamental of all economic laws. If teachers do not perceive the value of using eLearning technology, then there will be no demand. If production costs are too high, then prices will limit demand as well. On the other hand, if there is perceived value in the marketplace of the educational sector of the knowledge economy, then it is safe to assume that demand will increase, given that prices are at an acceptable level. Because prices are in large part dependent upon production costs, then producers who can spread their fixed costs over a large volume of output and corresponding consumption will be able to reduce prices to a level that will in turn contribute to an increase in demand.

Specifically, therefore, SCORM can play a significant role in developing the business model that is implied here by:

  1. Providing the mechanism to spread fixed production costs over larger numbers of consumers (teachers and learners) than is possible through proprietary, online delivery solutions.
  2. Clarifying opportunities for research that can enhance the perceived value of instructional content by the key decision makers (teachers and administrators in formal settings and individual learners in self-learning settings).

Both of these developments are possible to some extent right now. In particular, several of the key goals of SCORM, i.e. interoperability, content reusability, etc. address the first issue. Just as the creators and purveyors of entertainment leverage their sales numbers to pump out billions of dollars of content each year, educational publishers could also benefit from a significant change in their way of doing business. Currently, about 77% of the cost of the typical college textbook goes to production and distribution that could be eliminated by electronic distribution (Bradshaw & Crutcher, 2006). Obviously, this possibility merely addresses a shift in the production and delivery mechanism, which would benefit the producer’s bottom line even as it would potentially increase the supply of online educational content. It says nothing about opening the possibility of presenting the same content interactively, which would enable learners to reap the documented benefits of instructional technology.

One market mechanism that is not yet at the required level is the means for providing feedback on the quality of products. This is accomplished in the ideal economy by the market itself: good products sell and bad products don’t. While publications like Consumer Reports and product reviews in various magazines help, the corresponding approach in educational software has yet to have much impact. It could also be argued that there is still a lot to be learned about what are the best approaches for designing and creating online materials.

One of the biggest untapped advantages for SCORM lies in identifying opportunities for research. Unfortunately, there are technical and political problems that make it difficult for such an effort to succeed. Technical issues deal with the fact that the current SCORM model is agnostic about what an LMS does with data from previous attempts by a learner on a Sharable Content Object (SCO). Ignoring these data and keeping only the final attempt might be appropriate for assessing the capabilities of the learner, but it is inadequate for determining the usefulness of online materials.

Because a SCO can be created at any level of granularity, it is probable that a great deal of useful online content will be buried inside complete online courses, inaccessible for use in situations where the whole course is not needed, significantly reducing reusability. While a full course on Basic Electronics might attract quite a few learners, smaller units of instruction, say on “Understanding and Applying Ohm’s Law” would no doubt reach even more.

The main political issue springs from the fact that professors in higher education are the biggest, primary source of learning content, mainly through the avenue of textbook publication. It is safe to assume that they would be the source of any extensive development effort for online materials. Unfortunately, the “Publish or perish!” concept is still the guiding doctrine for academic promotion within institutions of higher education, and professors’ contributions to instructional technology are often ignored. To help address this situation, a well-developed, SCORM-based infrastructure with adequate data collection on learner interactions could help education move beyond the question of whether instructional technology works or not to the more interesting issues of the types of online instructional interactions that work better than others.

Design-based research is perhaps one avenue for changing this situation, given that it is an avenue for connecting theoretical insights to educational practice (Sandoval & Bell, 2004). The development of a SCORM-conformant research platform to facilitate design-based research would open up opportunities for research and publication for professors, enabling all quarters of the knowledge economy’s educational sector to benefit.

It is not clear whether changes to SCORM itself would be necessary for this potential to become a reality, or whether the development of a research-oriented, best practices guide would suffice. What is clear is that just as engineering science enables the implementation of scientific principles in structural design projects, so would instructional materials science benefit from the contributions that such research would make in the area of online learning.

Conclusion

SCORM provides a tool addressing many challenges in implementing instructional technology, and its success should bode well for education. To date, however, the educational economy has not benefited substantially from demonstrated capabilities. These capabilities range from cost-effective development and delivery that is computer platform independent to a means of avoiding technological obsolescence, enabling SCORM to play a positive role in the market mechanisms of the knowledge economy.

Fulfilling this role becomes possible in two ways. First, increasing the base of “compatible” educational users, who could incorporate content from numerous sources into their learning environment, would increase the probability that developers would reach a suitable return on investment. Second, an increase in the opportunities for design-based research would enable developers to learn about learning, as well as leading to the determination of the sorts of interactions that best facilitate learning. In addition, reports on such research would provide invaluable information as to how well various products work, a solid harbinger of deserved product success.

With respect to needed improvements, it is not clear whether changes are necessary for the SCORM specification itself, or whether developing some sort of “Best Practices” guide, which` would be based on actual experience, would suffice. In any case, it is clear that as a specification for interoperability, SCORM is proving its mettle. Furthermore, at Version 2004, SCORM is now quite solid, given that it enables fairly sophisticated principles of instructional design to be implemented. Addressing the issues raised here would further strengthen SCORM’s value and provide significant benefits to education’s knowledge economy through increased implementation of instructional technology.

References

Bradshaw, G. L. & Crutcher, R. J. (2006, December). Beyond the Printed Page: Style Suggestions for Electronic Texts. MERLOT Journal of Online Learning and Teaching , 2(4). Retrieved August 2, 2007, from http://jolt.merlot.org/vol2no4/bradshaw.htm.

Bunderson, C. V. & Abboud, V. C. (1971, February). A Computer-Assisted Instruction Program in the Arabic Writing System. Technical Report No. 4. Austin, TX: The University of Texas at Austin. (ED052603).
Bunderson, C. V., Olsen, J. B., Baillio, B., Lipson, J. I., & Fisher, K. M. (1984). Instructional effectiveness of an intelligent videodisc in biology. Machine-Mediated Learning, 1 (2), 175-215.
Degen, B. (2001). Capitalizing on the Learning Object Economy: The Strategic Benefits of Standard Learning Objects. Retrieved July 23, 2007, from http://www.learningobjectsnetwork.com/resources/LONWhitePaper_StrategicB...

Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., Campuzano, L., Means, B., Murphy, R., Penuel, W., Javitz, H., Emery, D., and Sussex. W. (2007). Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort, Washington, D.C.: U.S. Department of Education, Institute of Education Sciences. Retrieved July 23, 2007, from http://ies.ed.gov/ncee/pdf/20074006.pdf.

Fletcher, J. D. (2003). Evidence for learning from technology-assisted instruction. In H. F. O’Neil, Jr. & R. Perez (Eds.) Technology applications in education: a learning view (pp. 79-99). Hillsdale, NJ: Lawrence Erlbaum Associates.
Hodgins, W. (2002). The future of learning objects. Retrieved July 23, 2007, from http://www.reusability.org/read/chapters/hodgins.doc.

Kulik, J. A. & Kulik, C. L. C. (1987, July). Review of recent research literature on computer-based instruction. Contemporary Educational Psychology, 12, 222-230.
Sandoval, W. A. & Bell, P. (2004). Design-Based Research Methods for Studying Learning in Context: Introduction. Educational Psychologist, 39(4), 199–201.
Symonds, W. C. (2000, January 10). Industry Outlook 2000 – Services: Education. Business Week. Retrieved August 3, 2007, from http://www.businessweek.com/2000/00_02/b3663149.htm

Back to top


Postscript to a Debate Between Constructivists and Supporters of Explicit Instruction
Sigmund Tobias
          

The publication of the article by Kirschtner, Sweller, and Clark (2006) led to a series of rejoinders to that paper (Schmidt, Loyens, Loft & Paas, 2007; Hmelo-Silver Duncan, & Chinn, 2007; Kuhn 2007; Sweller, Kirchner, & Clark, 2007). You may recall that the original article was also discussed in Issues No 3 and 4 of this Newsletter. In addition, the original article stimulated the scheduling of a debate at the 2007 conference of the American Educational Research Association regarding the paper’s main assertion that approaches to minimally guided instruction (constructivism, discovery learning, problem based learning, etc…) had failed. Debate participants included two critics of minimal instructional guidance: Paul Kirschner, the first author of the article assisted by comments from the paper’s second author John Sweller, and Barak Rosenshine, a well known critic of minimally guided approaches to learning. Defenders of constructivism included two noted scholars identified with that orientation: David Jonassen, and Rand Spiro.

The purpose of this note is to report a narrowing of the differences regarding the issues posed in the Kirschner et al. (2006) article in the debate as well as in some of the rejoinders to the article (see Hmelo-Silver et al. 2007; Schmidt et al., 2007) and to suggest some research and theoretical development that may ultimately resolve the issues posed.

In the debate the constructivists agreed that some form of direct, or explicit instruction (Sweller et al., 2007) may be useful for students with little prior knowledge of well structured domains. Similarly, Kuhn (2007) acknowledged that there is a place for explicit instruction, and Schmidt et al., (2007), as well as Hmelo-Silver et al. (2007) agreed that some instructional guidance was necessary. During the debate, the constructivists also agreed that some form of guidance, or instructional support (Tobias, 1982) more generally, was essential in all instructional approaches. The constructivists suggested that instruction in ill structured domains, such as problems in medical diagnosis, advanced engineering design, even teaching students the meaning of a complex concept such as “justice” were difficult if not impossible with explicit instructional approaches and could only be managed with instruction following a constructivist orientation. The critics of the constructivist position acknowledged that ill structured domains presented difficult problems and ought to be approached by a variety of means.

A consensus developed in the debate suggesting an interaction among prior knowledge, organization of the subject matter, and instructional method similar to the multivariable aptitude treatment interaction research recommended by Cronbach (2002. The constructivist debaters especially suggested that students’ prior knowledge was a key variable in these interactions, confirming expectations regarding the importance of prior knowledge for instructional adaptations (Tobias, 1989, 2003; Gustaffson & Undheim, 1996). It should be noted that while prior knowledge has been extensively studied in previous aptitude treatment interaction research, there were relatively few studies varying domain structure, a variable that could be profitably followed up by future research.

Instructional Support and Domain Structure

In order to achieve progress in the clarification of these issues it is important for theorists to be more precise about the definition of two terms: guidance or more generally instructional support, and the structure of a domain. Sweller et al. (2007) acknowledged the importance of greater specificity about what guidance meant. Clarification of all forms of instructional support more generally requires specification of a hierarchy so that terms like “minimal” support could be anchored in specific instructional actions rather than vague terms. For example, does asking students to repeat an answer provided by the instructor consist of more or less instructional support than providing knowledge of results, or “worked examples”? Similarly, where do forms of support such as providing explanations, whiteboards, prompts, or hints, to mention some of the supports described in the articles, fit in a hierarchy of instructional support?
It would also be useful to have specific descriptions of how domain structure may be determined. Clearly subjects like the multiplication tables, for example, constitute a well structured domain, and medical diagnosis of rare disease entities is an ill structured domain. Perhaps a second hierarchy ranging from ill structured to well structured domains should be specified, and instantiated by examples such as those given above, to facilitate research on the interaction among prior knowledge, instructional support, and domain structure.

Specification of Cognitive Processes

It should be noted that all instructional approaches need to specify the psychological processes engaged by teaching methods. Even though learning may be strongly influenced by the communities with which learners affiliate, and perhaps even by the environments in which learning occurs (Tobias, 2003), the psychological processes presumably engaged or enhanced by participation in communities need to be specified. Do constructivist activities enhance such processes as attention, retention in working memory, or storage and retrieval from long term memory, to mention only some examples? Similarly, which of these processes, or others, are invoked by explicit instructional methods? Furthermore, which of the approaches stimulates more frequent processing of instructional material either with the processes suggested or by others?

The accuracy of Kirschner et al. (2006), the rejoinders to it (Schmidt et al., 2007; Hmelo-Silver, et al., 2007; Kuhn 2007), and the response to the rejoinders (Sweller, et al., 2007) will ultimately be decided by research, rather than by rhetoric or debates. As noted some time ago (Tobias, 1982), different instructional approaches can lead to varying outcomes only if they either engage different cognitive processes controlling learning, or invoke those processes more or less frequently. Now, that some of the different views have been somewhat clarified by the publications and by the debate, would be a good time to conduct research concerned with the cognitive processes engaged by different teaching methods.  Such investigations will ultimately settle the issues discussed in the debate and in the publications dealing with the issue. I join Sweller et al. (2007) in urging that further experimental research is needed, and would add that in order to be most useful, such research should aim to identify and examine both the cognitive processes engaged by the instructional methods used, and the frequency with which the cognitive processes are used by different instructional approaches.

References

Cronbach, L.J. (Ed.) (2002). Remaking the concept of aptitude: Extending the legacy of Richard E. Snow. Mahwah NJ: Erlbaum Associates:.

Gustaffson, J. & Undheim, J.O. (1996). Indivdidual differences in cognitive functions. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 186–242). New York: Macmillan Reference

Hmelo-Silver, C., Duncan, R., & Chinn, C. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 99-108.

Kirschner, P. A., Sweller, J., & Clark, R. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist,discovery, problem-based, experiential and inquiry-based teaching. Educational Psychologist, 41, 75–86.

Kuhn, D. (2007). Is direct instruction the answer to the right question? Educational Psychologist, 42, 109-114.

Schmidt, H., Loyens, S., van Gog, T., & Paas, F. (2007). Problem-based learning is compatible with human cognitive architecture: Commentary on Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 91-97.

Sweller, J., Kirchner, P.A., & Clark, R.E. (2007) Why minimally guided teaching techniques do not work: A reply to commentaries. Educational Psychologist, 42, 115-121.

Tobias, S. (1982). When do instructional methods make a difference? Educational Researcher, 11(4), 4-9.

Tobias, S. (2003) Extending Snow’s conceptions of aptitudes. Contemporary Psychology, 48, 277-279


Improving Courses
Traci Sitzmann
Research Scientist
Advanced Distributed Learning (ADL) Co-Laboratory

Would you like detailed information regarding course stressors, preparation for class, feedback received on assignments, and more? Participating in this project will give you the information to improve your course design.  As part of ADL’s mission, we are helping trainers and course designers develop courses maximizing student learning and satisfaction. By participating in this project, you will:

  • Receive detailed feedback on students' perceptions of your courses
  • Receive information not included in typical course evaluations such as whether students felt the course was stressful, whether sufficient feedback was given, and which instructional methods were used for learning
  • Enhance the effectiveness of courses

To participate in this project, students will need to answer an online questionnaire, requiring 15 minutes or less, to assess their course satisfaction and perceptions of the course design. Additionally, the course instructors/ designers will need to fill out an online questionnaire (requiring 10 minutes or less) to assess their perceptions of the course design. In order to receive useful feedback, as many students as possible should complete the questionnaire.

You can participate in this project at any time by clicking on the following instructor/course designer questionnaire link and forward the student questionnaire link to your students.

Instructor/course designer questionnaire: http://adlevaluation.googlepages.com/instructor

Student questionnaire: http://adlevaluation.googlepages.com/student

At the end of the questionnaire there is a place for the instructor's/course designer's e-mail, which will ONLY be used to send feedback. The instructor/course designer will receive a customized free feedback report. While it is understood that no computer transmission can be perfectly secure, reasonable efforts will be made to protect confidentiality and student names will not be collected. Also, the feedback report will ONLY be sent to the instructor/course designer and these individuals will be the only ones outside of the ADL Training Evaluation Team to receive a copy of the report.

Please contact Traci Sitzmann (traci.sitzmann.ctr@adlnet.gov) for further questions about this project or participate whenever you are ready. Also, please forward this message to colleagues who may be interested in participating. We look forward to hearing from you.

Back to top

 

Academic ADL Co-Lab
222 W. Washington Ave.
Suite 470
Madison, WI 53703
Phone: 608.263.6657
Fax: 608.265.0070
www.academiccolab.org