Lit Review Outline

Here is a rough outline for my literature review…

• Introduction
  • The technologization of China
  • Internet use in China
• Web-Based Instruction in Chinese Education
  • Initiatives from the Ministry of Education
  • Preliminary results from educational initiatives
  • Characteristics of student and teacher users
• Barriers to adoption of WBI in China
  • The Chinese classroom
  • Student learning approaches
  • Instructional approaches
  • Influence of college entrance examinations
• Cultural Dimensions in Chinese Pedagogy
  • Survey of Hofstede’s cultural dimensions
  • Chinese cultural dimensions and relationships to pedagogy
• Rethinking the Chinese Learner
  • Alternative views of Chinese learning and instruction
  • Criticism of cultural deficit views
  • Findings from the research
• Mastery Learning, WBI. and Chinese Pedagogy
  • Survey of mastery learning
  • Bloom’s model for mastery learning
  • Synergy with WBI
  • Compatibilities with Chinese pedagogy
• Research on Mastery Learning in China
  • Survey of current research
  • Need for mastery learning used in conjunction with WBI

This is shaping up to be a pretty long literature review, so I may consider dropping or trimming some sections.

 

Summaries on New Readings, Part 1

As part of this iteration, I’m also doing another search in the literature and updating some of my references.

The two main searches I did were to beef up my references on Bloom’s mastery learning approach and its relationship with web-based instruction (WBI) and computer-assisted instruction (CAI); and to search for any new, good references on the use of WBI in China in the last year or so.

After reading through the articles, here are summaries are what I found in my search for mastery learning articles:

• Kim, S. (2005). The relationship between enactive mastery experiences and online-course self-efficacy (OCSE). Retrieved from ERIC on September 4, 2007.

This turned out to be not exactly relevant. This is a study about how using mastery learning can facilitate self-efficacy, where the curriculum is actually using computers; not where computers are used as an instructional tool. Interesting discussion here, but it’s not something that would go into this literature review.

• Brothen, T. Transforming instruction with technology for developmental students. Journal of Developmental Education, 21(3). Retrieved from ERIC on September 4, 2007.

This was a commentary, not a research study. Nevertheless, it was interesting to me, because the kinds of assertions it poses are the main impetus for me to look more into the issue of contextualizing Western-based WBI in a Chinese setting. For example, it asserts, “[As opposed to lectures,] discussion, for example, is clearly superior at fostering student retention of information, transfer of knowledge, problem solving, thinking ability, attitude change, and motivation.” Really? Is that so sure of a fact, even in Western settings? If a teacher were to use discussion instead of lecture as the instructional method in China, he might find that students actually have much more difficulty learning, as many Western teachers who have taught in China have found.

The commentary also includes a discussion on Bloom’s model and Keller’s Personalized System of Instruction, saying that computer technology has opened up new possibilities for these models of instruction, as they are a good fit for computer-assisted instruction and the individualized tutoring that such tools can facilitate. I mostly agree with this, but perhaps fall short of the optimism that the author has on how easily computer technology will be able to be integrated and used. The author points to teachers being able to write their own courseware that will help students get individualized tutoring, by using new course authoring tools. Having spent more than half a year on writing a single tool for a limited purpose, that could hopefully be the precursor to an authoring tool that a non-programmer could use — well, I’m not sure it’s that easy. Moreover, I’ve made tons of little decisions about administrative issues — how classes are organized, assessed, scheduled, etc. that are totally outside the realm of even the curriculum — that would need to be supported and customizable as well.

• Dalton, D. W. & Hannafin, M. J. (2001). The effects of computer-assisted and traditional mastery methods on computation accuracy and attitudes. Journal of Educational Research, 82(1), 27-33.

This study was a great source of insights on the use of CAI in mastery learning. The study’s literature review was also very informative. It addresses one of the difficulties with mastery methods due to applying it in group settings — also known as the “time-achievement-equality dilemma.” The basis of this is the mastery methods may can slow down able learners for the sake of less able learners, or can leave behind less able learners so that able learners achieve more. In essence, it’s a synchronization issue — because there is (usually) only one teacher following one schedule that the entire class has to follow, either people are going to be left behind or slowed down. Or both. The benefit of CAI is that now instruction can be more individualized, and that there are now opportunities for able learners to go faster or less able learners to go slower, since the teacher is not the “instructional bottleneck” anymore.

An interesting finding in this study is that “while both traditional and computer-based delivery systems have valuable roles in supporting instruction, they are of greatest value when complementing one another.” This finding echoes many studies that have been done in studying blended learning approaches, and it bodes well for my proposed approach for my tool of combining in-class instruction with individualized tutoring/drilling with the system.

• Pape, L. (2006). From bricks to clicks: Blurring classroom/cyber lines. School Administrator 63(7). Retrieved from ERIC on September 4, 2007.

This was not a research article, but just an informational article mostly about hybrid classrooms/blended learning – the use of online and face-to-face teaching approaches. It contains some sections that delve into the role of mastery, and how the instructional variable becomes time, and unit mastery as the constant, rather than the other way around with the face-to-face approach alone. The article doesn’t have a bibliography, so it was pretty much a dead end.

• Lyuben, P. D., Hipworth, K., & Pappas, T. (2003). Effects of CAI on the academic performance and attitudes of college students. Teaching of Psychology, 30(2, 154-158.

This study starts with the idea of mastery learning and CAI, but goes looks beyond how assessments are traditionally performed with this approach. Specifically, this study is concerned about the idea of fluency — that is, the rate at which a learner can answer questions correctly, and the durability of the learner, which enables the learner to solve problems even when distracted. This is an interesting angle to look at, but it’s beyond the scope of what I am looking at, so most likely I won’t be including this in the literature review.

 

#36 of 40 Add Lesson Table and Test Data

Today’s story involves adding instructional units to the database. Since that’s a mouthful and might be confusing terminology to people outside the education academia realm, we’re going with a simpler term, “Lesson.”

The purpose of the lesson table will be to store the definition of the lesson — the lesson name, as well as its data relevant to the mastery learning method. In particular, there will be a start date and a date for the 2 assessments. Any practice that the students do will occur between these dates.

Also, in this lesson we specify the number of possible points for assessment 1 and 2, as well as define what level of achievement (as a percentage) constitutes mastery.

The data fields are pretty straightforward, with the LessonID obviously being the primary key and being auto-numbered, and a foreign key relationship with the Class table.

The rest of the time for the story was for setting up test data with some simple lesson records for the Arithmetic class, which is mostly for helping testing, and also a lesson record for vectors in the physics class, which will be the more “realistic” lesson module.

 

#5 of 40 Problem Statement

My advisor has asked me to write a problem statement to help ground my project and define its scope, and especially since things have changed since I’ve started. Here it is…

Problem Statement

Education in China has seen a rapid proliferation of Web-Based Instruction (WBI) over the past decades. The government’s Ministry of Education has instituted far-reaching initiatives for primary, secondary, college, and adult learning, backed by tremendous amounts of capital, infrastructure improvements, and equipment. In many respects, China is following on the heels of a similar massive infusion of web-based tools in Western education.

A large body of research already exists which evaluates the effectiveness of various web-based instructional approaches and explored the successes and mistakes that the “pioneers” of WBI experienced. This has allowed Chinese educators and administrators to take an informed approach to integrating WBI in the Chinese classroom. However, a growing number of researchers are raising important issues about the validity of such research in the Chinese cultural context.

Although there has been significant and illuminating research done in WBI, the large majority of it has been performed in the Western classroom. Additionally, much of the literature surrounding WBI has adopted a constructivist approach. Unfortunately, many researchers have pointed to significant incompatibilities between Chinese and constructivist pedagogy, including the relationship between teacher and student, typical Chinese learning strategies, culturally-acceptable classroom behavior, and the motivation for learning. To date, there has been little research done on web-based instructional tools which are designed specifically to accommodate for pedagogical differences in non-Western cultures.

One of the strongest cultural traits ascribed to China is its collectivism; that is, a strong group focus, and an emphasis on collective, rather than individual achievement. This is in strong contrast to most Western nations, which are highly individualistic. This project is an attempt to create a web-based instructional tool for secondary Chinese students which facilitates their collectivist tendencies, while still fitting in the structure of individual assessment as necessitated by the Chinese educational system. The tool will supplement a modified version of Bloom’s mastery learning approach, by having students make two attempts to master an instructional unit. In Bloom’s model, students who achieve mastery in the first attempt typically are introduced to more difficult material in the second stage, while the others make another attempt.

The main differences for this project are that students are organized into study groups, and assessed not only on their individual achievement, but also their study group’s achievement; and that the students who achieve mastery in the first stage become peer-instructors in the second stage for their study group. Consequently, the web-based tool for this project requires a system which assesses mastery, as well as a way of tracking student and study group progress by both students and teachers.

The hope is not to find the best approach for using WBI in Chinese pedagogy, but merely a possible approach, which at least attempts to account for Chinese cultural and pedagogical differences. More than anything, it is clear that more research is needed which looks specifically at the dynamics between web-based instructional tools and their surrounding cultural context.

 

Specs in the Form of User Stories

I used to be really big on writing detailed specifications upfront when designing and writing software. It makes logical sense — the more detailed and specific the requirements are, the easier it is to write code exactly with the big picture in mind.

The problem is that this rarely holds true in the real world for complex problems, especially those in a domain that isn’t well-defined and clean cut. The reality is that the vision and direction of a software project changes frequently in both major and minor ways throughout the course of a project. It doesn’t mean that specifications are worthless — just that there isn’t a huge payoff in thinking out every last detail and dreaming up every scenario that the user might face very early in the project, because things are likely to change.

Along those lines, I’ve seen a different sort of specifications in some projects whose purpose is to informally layout a “projected vision” of what the software will do. Some practitioners talk about the idea of “user stories” — telling the story of a user using the software in a real-life situation, and thus anticipating some of the major aspects of the necessary interactions the user has with the software, as authentically contextualized as possible. I’m going to take a similar approach in thinking out how the idea of mastery will be implemented in the system I’m proposing. This “user story specs” will serve as a guide for my eventual software design.

Note: I don’t yet have an official name for the software I’m writing, so for the purposes of these user stories, I’m just referring to it as CWBI.

Ronnie – Instructor

Ronnie is a high school physics teacher in China. He has 2 sections of about 50 students each. Ronnie has been using CWBI software with his students to supplement his instruction since the beginning of the year. Earlier, Ronnie had divided his class randomly into “study groups” of 10 students; so 5 study groups per section. He set up a course and added all of his students to the course, dividing them into their respective study groups.

Ronnie now wants teach a module on kinematics. This module consists of 3 main parts — motion in 1D, vectors, and motion in a plane — with each part taking approximately 3 weeks to complete. In CWBI, he creates a new module, and adds these 3 instructional units from the available supplements provided by CWBI. For each unit, he sets the start date and end date for the unit, as well as a midpoint cutoff date when the 1st assessment in Bloom’s mastery learning model will be taken. Ronnie usually sets the cutoff at 2 weeks, leaving 1 week between the 1st and 2nd assessments. Ronnie also sets his definition of “mastery” by specifying the percentages of problems in each difficulty level that students must answer correctly.

The kinematics module starts, and Ronnie conducts his teaching as usual in the classroom. Ronnie does not give homework to his students; rather, each student is expected to log hours on CWBI to run through practice problems. CWBI provides Ronnie with statistics on the number of practice problems his students perform. Although he does not factor this number into his grading, it enables him to intervene for some students who may be procrastinating, or may be experiencing difficulties. He is also able to view quickly how many of his students are approaching mastery.

As the cutoff date approaches, Ronnie starts to get a sense of how much of his class is understanding the material, and he is able to spend more time on concepts that students seem to be having difficulty on. The cutoff day has finally arrived, and Ronnie’s students take a proctored assessment in the computer lab through CWBI. Ronnie locks the practice problem area for the duration of the assessment. At the end of the assessment, Ronnie looks up the scores in CWBI. Ronnie teaches a challenging class, and as expected, only about 40% of his class achieves mastery at the first assessment. At this point, Ronnie switches gears, and spends most of class time in the computer lab. The 40% of the class who achieve mastery now are peer tutors, helping the students in their study group to understanding the material. The 1 week passes quickly, but by the end, 90% of the class have achieved mastery.

Freda – Student

Freda is a 2nd year high school student in China (US equivalent 11th grade), taking physics from Ronnie. This is Freda’s first course that is supplemented by a web-based instructional tool, but she is looking forward to the time on the computer, as she uses it for chatting with friends online frequently. Freda goes to class and begins a unit in 1D motion, the first part in a larger series of unit in the kinematics module.

Freda listens to Ronnie’s lecture, and reads in her textbook about 1D motion. When she starts getting a feel for what she needs to know, she logs onto CWBI and selects her course and current instructional unit. Noting what kind of performance she needs to achieve mastery as defined by her teacher, she goes to the practice problem area, and tries her hand at several problems. The first few problems are relatively easy, but she stumbles on a few. Looking at her progress, she knows that she is not yet at a mastery level. She has some difficulties with understanding the difference between acceleration and velocity, so she asks some of her fellow students online. Freda is able to understand a few things, but there are some areas which she needs more help in, so the next day, she goes to Ronnie’s office for some help.

After talking with Ronnie, Freda is much more confident on being able to solve the problems she earlier had difficulty on, so she goes back to the practice problem area. This time, she gets many more answers correctly, and is able to build her confidence by going through a variety of problems. She notices that according to CWBI’s projection, her latest session indicates that she should achieve mastery were she given an assessment. Freda is in luck, because the 1st assessment cutoff happens to be the very next day. Freda passes with flying colors, and as such, is now given the task of helping other students in her study group to achieve mastery themselves. The degree to which her study group is able to achieve mastery is part of her grade, so she has extra motivation to help, and she also has plenty of time to do so, now that classroom time has moved to the lab.

To Freda’s delight, 9 out of 10 members of her study group achieve mastery level by the end of the instructional unit. She is also looking forward to the next unit, as she has already read a little about it in her textbook, and has started doing practice problems ahead of time.

 

Bloom’s Mastery Learning Approach

I’ve done some more in-depth reading into mastery learning and some of the other mastery-related instructional approaches. It’s actually quite a big area, and as I’ve mentioned earlier, the point of the project is not to study the concept of mastery, but I do want to choose a model.

After a review of some of the literature, I’ve found some key points. Mastery learning, as a theoretical approach, goes back to the work of Benjamin Bloom in 1968, who came up with the “Learning for Mastery” (LFM) method. Bloom was interested in how he could improve traditional classroom instruction by examining what it was about individual tutoring that made it an effective instructional approach.

Bloom contended that most instructors were dividing their instructional material into smaller units of instruction, but that the way the students’ progress was assessed was not helpful for their learning. To be specific, instructors typically had the students take an assessment at the end of the unit of instruction, which served to give the students a grade for their performance, but regardless of how the student did, he or she continued on into the next unit of instruction without any benefit or lesson learned through the assessment.

Bloom’s proposal, instead, was to have 2 formative assessments per unit of instruction. The purpose of the first assessment is similar to the traditional instructional approach; however, this time, the results of this first assessment are not only used to give a grade to the student, but also function as a diagnostic to the instructor as to what particular areas of the instructional unit the student is having difficulty with.

Those students who failed to achieve mastery for the first formative assessment would now be given further instruction using different instructional approaches (Bloom believed that varying instructional methods would help a larger percentage of students achieve, since learners learned effectively in different ways). They would be assessed a second time to determine how far they had progressed.

Meanwhile, students who achieved mastery in the first formative assessment would go on to learn concepts which extended and built upon the unit of instruction, and their second assessment would primarily revolve around these extended concepts. The purpose was to raise the bar higher, while not leaving the rest of the class behind.

There are other similar or derived approaches; the literature also mentions the Personalized System of Instruction (PSI), the Problem Based Learning (PBL), and Outcome-Based Education (OBE) as related concepts. PSI allows students to control the pace of instruction themselves, and use primarily written materials for self-instruction. I don’t think this will fit well into the Chinese educational context. PBL is more concerned with the process of inquiry and authentic learning models, than centrally on the idea of mastery. Fhe focus of OBE seems to be more on the larger curriculum development process, and defining desired “outcomes” or “goals,” then working backwards to instructional strategies. Mastery learning is used frequently in conjunction with OBE, but is not a necessary component.

After brainstorming a little on mastery learning, I’ve come up with a slightly modified model based on Bloom’s. What I plan to do is to have 2 assessments, similar to Bloom’s model, but for students who pass the first assessment, the primary task of the 2nd stage is to tutor and assist other students who did not pass the first assessment. It’s often said that having to teach a subject forces one to learn it in depth. More importantly, I believe that this works relatively well with the collaborative bent of Chinese students, and as long as students’ scores are not comparatively dependent on each others’ success or failure, this could mean a relative win-win situation for all students involved.

There is a question of whether my proposed system will have the idea of group mastery. The idea was that groups would achieve mastery only when all the students in that group achieved individual mastery, and that this would somehow be tied to assessment of the group. While in principle, it could motivate students and provide an opportunity for collaboration, there are negative consequences to consider when a large majority of the group is pressuring an individual who is unmotivated or has social adjustment problems.

But now that I am adapting Bloom’s approach with a collaborative tutoring aspect, I feel more inclined to discard the idea of group mastery altogether, since I am delving into the cultural value of collaboration through different means. If I can think of a different way to approach group mastery that doesn’t have the obvious potential to reduce students’ intrinsic motivations to learn, then I’ll put this back into the site design; otherwise, for now, I’m shelving it.

Here is a list of articles I read, which helped me to understand Bloom’s idea of mastery learning. These are mostly secondary sources, so I’ll need to read up on some primary and peer-reviewed sources later for a proper literature review, but it’s a sufficient start. I’m not taking the time to look up exact references and do APA and all that jazz, since I’ll be getting better references later anyway.

Articles Read

• Anderson, S. A. (1994). Synthesis of research on mastery learning.
• Douglas, C. (2002). The effects of mastery and performance goals on college students’ motivation.
• ERIC Digest E530. (1984). Connecting performance assessment to instruction: A comparison of behavioral assessment, mastery learning, curriculum-based measurement, and performance assessment.
• Ford, B. & Klicka, M. A. (1998). The effectiveness of individualized Computer Assisted Instruction in basic algebra and fundamentals of mathematics courses.
• Guskey, T. R. (2005). Formative classroom assessment and Benjamin S. Bloom: Theory, research, and implications.
• Guskey, T. R. (1994). Outcome-Based Education and Mastery Learning: Clarifying the differences.
• Price, R. (2000). PSI revisited: Designing college courses using the Personalized System of Instruction (PSI) model.

 

Mastery in the Literature…

When I first brainstormed for this project, I didn’t think a whole lot on how I will define “mastery.” This is an important part of this project, though, because it is central to determining the progress of each student in their respective units of instruction.

I had assumed that mastery could simply be defined as getting a certain number of problems correct, whether a percentage of total problems, or a percentage of problems within a fixed time period. I remember taking a “unit mastery” course for Sociology, which basically required the student to get something like 85% or better on a multiple choice examination based on a book chapter.

As with most important concepts, though, there’s a lot more to mastery than meets the eye. For example, the system for determining mastery for my Sociology course is more complex than getting a certain percentage correct. There are other issues, such as:

• How is the course broken down into instructional units that can be tested? How granular is this?
• Can you take tests for the instructional units out of order?
• Can you take tests multiple times until you pass? Is there any consequence for failing a test?
• Is there a time limit for when a test must be passed by?
• Does the unit mastery test itself have a time limit?

Actually, these are just issues dealing mostly with practical details. There are other, more philosophical issues to consider as well, such as: what kind of test would appropriately demonstrate a student’s mastery? What kind of record is kept of the student’s progress, and who has access to this? What is the teacher’s role in mastery-based instruction?

After thinking a little more about this, I came to the conclusion that while it would be OK to propose a system that defined mastery as “getting a certain percentage of questions correct,” it would be important to fill in more details, as well as be able to give a basic rationale for the reasoning behind this.

To do this with any kind of credibility in the academic realm, though, I need to read up on the idea of mastery in the literature. A preliminary search on this topic has yielded 3 key approaches under the mastery umbrella that have come up often in my literature searches:

• The Mastery Learning Method. One of the fundamental ideas of this approach is that nearly all students can learn individually or in groups, given the appropriate resources, tools and environment. The equivalent “A” and “B” grades are only accepted, as these demonstrate “mastery.”
• Personalized System of Instruction (PSI). Proposed by Fred Keller in the 1970s, and is based on operant conditioning (behaviorism). The key principles involve self-pacing, the use of written verbal communication between instructor and students, and the use of proctors who score tests and individual tutor students. Also, classroom time is not used to introduce material, but as a time for motivation.
• Problem-Based Learning (PBL). This group-based approach is based on a mix of cognitive and constructivist theory. In this case, the instructor (or facilitator) gives a problem to a group of students. The students may do individual research on the problem, but will come together as a group to exchange ideas.

I will be looking more closely at what the literature shows for these 3 approaches in the coming weeks. Although the focus of this project is not on the concept of mastery itself, I would like to try to match a custom approach which seems like it would work well in Chinese educational culture, and for the practical affordances and limitations presented by a web-based educational tool.