Table of Contents
INTRODUCTION........................................................................................................3
1. Research Background............................................................................................3
2. Research Question..................................................................................................5
3. Research Objectives...............................................................................................8
4. Research Process...................................................................................................9
CHAPTER 1: LITERATURE REVIEW..................................................................11
1.1 History and Evolution of Curriculum Development for Primary Level............11
1.2 Curriculum Development Tools for Primary Level...........................................12
1.2.1 Bradley’s Effectiveness Model for curriculum Development Indicators..13
1.2.2 Tyler’s Objectives-Centered Model...........................................................15
1.2.3 Stufflebeam’s Context, Input, Process, Product Model.............................16
1.2.4 Scriven’s Goal-Free Model........................................................................18
1.2.5 Stake’s Responsive Model.........................................................................20
1.2.6 Eisner’s Connoisseurship Model................................................................21
1.2.7 Value-Added Assessment...........................................................................23
1.2 The Crossing between Curriculum Development and Management Tools.......24
1.2.1 The importance of management in education............................................24
1.2.2 The Logical Framework Approach and its practice to enhance managerial
activities at schools..............................................................................................26
1.3 Steps to establish evaluation system for curriculum.........................................31
1.4 Criteria in setting the evaluation system............................................................33
CHAPTER 2: METHODOLOGY............................................................................37
2.1 Research Methodology.......................................................................................37
2.2 Sample................................................................................................................38
2.3 Instrumentation..................................................................................................39
2.4 Data Collection...................................................................................................41
2.5 Method of Analysis.............................................................................................42
2.6 Summary............................................................................................................42
1
CHAPTER 3: DATA ANALYSIS..............................................................................43
3.1 Overview of Cambridge International Education program..............................43
3.2 Data Analysis.....................................................................................................45
3.2.1 The process of identifying problems..........................................................45
3.2.2 The series of surveys to the establishment of evaluation system..............49
CHAPTER 4: RECOMMENDATION.....................................................................60
4.1 The establishment of evaluation system............................................................60
4.1.1 The first-level evaluation system...............................................................60
4.1.2 The second-level evaluation system...........................................................68
4.2 Challenges to program evaluation.....................................................................69
4.3 Recommendations..............................................................................................71
CONCLUSION...........................................................................................................74
1. Limitations of the research..................................................................................74
2. Conclusion............................................................................................................74
2
INTRODUCTION
1. Research Background
There is no doubt in considering education as an essential part of our life.
One believes that educating and being educated is not only a great source to equip
oneself with sufficient knowledge and skills, but also a necessary tool to thrive on
later in life. However, for the past few decades, we no longer just encounter with the
question of whether a child is deprived of education or not. We are confronting with
how to best improve our existing educational system. Lately, a good deal of
attention has been paid to how to evaluate existing and upcoming curriculum and
educational projects so that they could benefit students decently. As a result, a
number of theories in the domain of education and teaching have become
increasingly refined to evaluate and assess a curriculum is beneficial for students or
should it be modified and improved.
Curriculum, as being defined in Oxford Dictionary (Oxford University Press,
2006), is first and foremost the courses that are taught by a school, college, et
cetera. In education, a curriculum is broadly defined as the totality of student
experiences that occur in the educational process. The term frequently alludes
particularly to an arranged grouping of guidelines, or to a perspective of the
understudy's experiences in accordance with the educator's or schools instructional
objectives. Curriculum may fuse the arranged association of students with
instructional substance, materials, assets, and courses of action for assessing the
accomplishment of instructive objectives. Additionally, curriculum split into several
categories, such as the implicit and explicit, as well as the official curriculum and
extra-curricular activities.
There have been several notable works by Western educators and researchers
in this particular field of education since 1940s and each one of them provide
different outlook on how to conduct the best curriculum for primary pupil students.
Interestingly, mentioning evaluation and assessment, it is immediate to draw a
connection to managing and controlling, which is an important part of management
knowledge and practices.
3
Managerial evaluation theories have proved to serve the utmost course- how
to best utilize existing resources to bring out the most effective and efficient results.
An issue emerging among the educators and managers is that whether one should
enable school systems to examine and critique their curriculum development
procedures with managerial tools. Those in charge for curriculum planning surely
need help to employ a good enough process for establishing goals and determining
the appropriateness for their particular situation they are facing with, to apply
managerial philosophies in creating what is best for their students, in other words,
their clients.
Among the tremendous variety of existing managerial theories in controlling
and evaluating for projects, one could easily note that Logical Framework Approach
(developed by U.S. Agency for International Development, 1969), which is a
management tool commonly used for designing, monitoring, and evaluating
development projects, has gained popularity in both private business sector as well
as educational institutes. The Logical Framework Approach continues to gain
adherents, though it is a management tool invented more than 40 years ago.
This thesis reports the findings of a thorough study to establish the factors
needed to form a scientifically appropriate and successful curriculum for a
collaborated project between Topscore Educational Consultancy Company and
Doan Thi Diem Primary School. Subsequently, these factors are in hope to be
applied in other educational institutions and establishments in Hanoi and in
Vietnam. Emphasis is placed on the establishment of an evaluation system created
by the combination of existing educational and managerial tools, as well as the
requirements of sufficient quality control during the development process. In that
sense, central to the research study is exploration of the sharing role and
responsibility of participating members and experts, which take up a crucially
important part in this educational and training project.
So often that the educational management, or management in general, has
been brought down to too simple factors such as lack of funding to facilitate
activities, however, it is not the case being examined in this research. It is
4
observable that the central factor in the failed progress of State-own or semi Stateown projects lies in the lack of dedicated research and cooperation among
participating members. In Vietnam, funding has never been enough due to
continuous demands from education reformation and to the fact that Vietnam still
stands in the rank of low-income countries. Despite that, one can notice the role of
good management here, that being said, management refers to maximize or proper
utilization and make best use of limited and a scarce resource. In that sense, good
research and cooperation among membership that enables the working consistently
and discretely towards the betterment of curriculum plays a great role in the success
of the project.
To assist in the possible improvement of this project, it is required the
understanding of the essence of the work, including responsibility, staffing,
collection, policies and procedures, funding and structure of academic curriculum.
My research, therefore, has incorporated interviews with Cambridge International
Education program staff and a questionnaire survey to all the participating students
and their parents.
2. Research Question
Great interest has been shown in the past decade in the curriculum field.
Most of the emphasis has appeared to centre around multi-ethnic approaches, new
academic programs, building autonomy, new school organizational patterns, and
media technology rather than around curriculum development procedures itself.
Consequently, school systems have found ample help in determining which new
programs to adopt, often these programs have been package proposals complete
from goals to evaluation. New patterns of school organization also have been
recommended, beginning with such changes as adding classes for prekindergarten
on through un-grading the elementary and secondary schools and the making of
suggestions for a middle school which would change the more traditional junior
high structure.
By the same token, nowadays, Vietnam’s society has shown its great concern
of educational system and curriculum reformation. It is observable that the number
5
of international programs conducted in metropolitan cities such as Hanoi or Ho Chi
Minh City has been increasing relentlessly. The involvement of English-taught
programs demonstrates a great deal of parents’ concern for advanced educational
path for their children, especially in order to aid them in the context of
globalization.
Among the leading schools for primary education, Doan Thi Diem primary
school stands as a prestigious education establishment. The school integrates several
international programs with A from England, B from USA and C from Singapore,
with hope to provide students with a wide range of choices that matches their
parent’s satisfaction and requirements. The international program in collaboration
with Top Score Vietnam, an educational consultancy company, using curriculum
adoption and teaching materials from Cambridge University and its teaching
consultancy department, is called Cambridge International Education program
(hereby is abbreviated as CIE at times).
During the last few years, the CIE program is incorporated into some
selective schools in Hanoi, including primary schools Doan Thi Diem. By 2010, the
program was put into some schools in the city on the basis of voluntary registration
of parents and students. To meet the high quality of training, CIE program focused
on recruiting teachers, in which the criteria of expertise and teaching experience are
highly competitive.
Initially, the program used to be a collaborative project between Doan Thi
Diem Primary School and EMG educational consultancy company. However, due to
the lack of due responsibilities provided from EMG, the company itself has been
excluded from the program and then the control was taken over by Top Score
educational consultancy. The CIE program, hence, is now expected to learn from
the experienced of the past failing incidence and hopes to advance the program’s
quality to the next level, especially in terms of curriculum.
After two weeks of adapting and observing, it occurred to me that most of
the problems this program has faced with lies in its internal process of assessing and
controlling the quality of its curriculum development progress. Curriculum, as being
6
understood in this research, is an organizational pattern for structuring leaning,
which is the pre-planned dimension of the instructional program.
The purposes of the study reported here, therefore, are to identify criteria for
curriculum development that are crucial, functional and observable so that it could
align with the educational goals of the school itself as well as the individual goals of
each students; and to test these criteria in existing school system. Through initial
analysis, it is uncovered that there are three basic problems that were seriously
hindering not only meaningful evaluation of this project, but also their
implementation.
Firstly, the planning was too vague where the objectives were multiple and
not clearly related to project activities. There has not existed yet the clear picture of
what the project would look like if it were successful. Thus, evaluators could not
compare – in an objective manner – what was planned with what actually happened.
Secondly, and perhaps most importantly, the evaluation was an adversary process:
with the absence of clear targets and frequent disagreements (even among project
team members) as to just what the project was about, evaluators ended up using
their own judgment as to what they thought were “good things” and “bad things”.
The subsequent evaluation results would then frequently become a basis for
further argument about what was good or bad, rather than resulting in constructive
actions for project improvement. Educational curriculum, on the other hand,
requires a group of experts to examine scientifically and thoroughly its quality. The
CIE program needs a council of experienced educational experts that could help it
to toss light on two questions: one, do planned courses, programs, activities, and
learning opportunities as developed and organized actually produce desired results;
and two, how can the curriculum offerings best be improved.
Through that understanding and observation being made at CIE project, this
research hopes to provide a tool for curriculum evaluation by examining and testing
a combined tool between education and contemporary management.
7
3. Research Objectives
Having a keen interest in education as a tool for sustainable development, the
researcher has come to realize having an academic background of economics and
management as well as a concern in primary education itself is not a conflict. On
the other hand, this crossing gives interesting perspectives and outlooks on problem
identification and problem solving.
The two main objectives of this thesis is firstly, to identify existing problems
related to CIE Program at Doan Thi Diem Primary School- a program to integrate
international standard curriculum developed by Cambridge University into
Vietnamese educational system. Secondly, through the examination of these
problems and pitfalls, the researcher hopes to give out solutions to two issues: to
what extent the current developed curriculum has fulfilled the educational goals of
Doan Thi Diem Primary School; and the initiatives to improve the current quality of
curriculum using neat tools from managerial science, with help of a comprehensive
evaluation system.
In the context of constantly changing competition, even an educational
institution now must strive to find out its prevailing points to satisfying its
customers, while staying true to the nature and significant of true and righteous
education. In that sense, this report is as well hoped to contribute as an insightful
paperwork to this important project of Doan Thi Diem Primary School.
The research’s mere ambition is not to provide a thorough inspection of all
aspects covered by these abovementioned theories; but to put some more
perspectives into the case of this CIE Program with hope to enhance its operational
activities. Even though the time of researching at Doan Thi Diem Primary School
was tentatively short and future research is indeed needed to consult the program in
much greater details, this research is still expected to stand as a suggestion from the
view of a business student to the program to perform in its most effective and
successful way.
8
4. Research Process
To fulfil the research objectives, the following session dwells on the
background that forms the academic side of this report. There are two schools of
theories, namely Curriculum Development tools and Logical Framework as a
management tool that are explained and described in an adequate brief.
After examining these theories, the author spends the rest and major
magnitude of this report to analyse the situation and context of CIE Project in
accordance with the preceding theories. The analysis is solely established in the
basis of the internship I have conducted at Doan Thi Diem Primary School from
December 2014 to April 2015. At the end of the report, solutions and suggestions
were generated in the form of a comprehensive evaluation system.
There stood three major steps in this research, including identifying problems
(determining research questions), interviewing and questioning (determining
research objectives and research process) and giving out solutions (determining
research conclusion).
Throughout these three major steps, a host of surveys, questionnaires and
interviews would take place with the participation of all different stakeholders of
Cambridge International Educational program, which shall be listed in the next
session of this report. The inquiry documents and the overall process aimed to be
short and concise in order to gain the information needed without having to
overanalyse pages of data, which could help to prevent an extended period of time.
9
Figure 1.1: Research process at Cambridge International Education program
Identify problems
Quick survey sent to
teachers
Examine the received
results
Interview and give
questionnaires to other
stakeholders
Interview School Board
Members and Project
managers to identify
Project's objectives
Interview parents and
students to identify
their expectations and
levels of satisfaction
Give solutions
Establish an evaluation
system combining
stakeholders' opinions
Give out some
initatives to obtain best
results
Source: author
All of the inquiries shall be found fully in the Appendix section.
10
CHAPTER 1: LITERATURE REVIEW
1.1 History and Evolution of Curriculum Development for Primary Level
Curriculum assessment in general has a long history. As Guba and Lincoln
(1981) pointed out, a Chinese head in 2200 B.C obliged that his open authorities
exhibit their capability in formal competency tests. In the United States, the
sympathy toward assessing schools can be followed at any rate as far back as the
proposals of the Committee of Ten, which toward the end of the nineteenth century
set maybe the first illustration of "evaluative gauges" for a country's auxiliary
schools (United States’ National Education Association, 1969).
As of late, then again, the enthusiasm for educational module assessment
specifically has appeared to increment especially. Society's emphasis on instructive
responsibility, the specialists' requests for instructive change, and the instructor's
attendant requirement for confirmation of results have all added to the current
enthusiasm for speculations and systems for educational module assessment.
Unfortunately, much of this interest seems to have resulted in an ill- conceived
obsession with test results and test results only. Therefore, a more extensive
viewpoint and more expanded methodologies appear essential.
Initially, primary school was synonymous with essential instruction, taking
youngsters from kindergarten through evaluation. As the populace developed and
got to be progressively urban and rural rather than country, the one-room school
building offered path to the multi-room school building, which got to be various
schools. The change of rudimentary training is clear. With a consistent, unfaltering
climb in differing qualities in a huge number of nationwide grade schools, the
instructive methodology of educators must adjust.
While school understudies of the twenty first century took the fundamental
classes in primary school (Social Studies, Science, Language Arts, Math, and so
on.), numerous schools today are changing their curricula and joining classes, for
example, Mandarin Chinese training. Despite the fact that the ordinary instructive
classes are still by and by, an alternate method is being met by directors and
11
instructors so as to successfully show all understudies, and stay aware of a changing
and advancing society.
Curriculum evaluation, therefore, should be understood as the process of
collecting data on a curriculum to determine its value or worth with the aim of
deciding whether to adopt, reject, or revise the programme, which is concerned by
various parties. For example, the public want to know whether the curriculum
implemented has achieved its aims and objectives; teachers want to know whether
what they are doing in the classroom is effective; and the developer or planner
wants to know how to improve the curriculum product.
1.2 Curriculum Development Tools for Primary Level
Educational curriculum development is a methodology of enhancing the
educational module. Different methodologies have been utilized as a part of
creating curricula. Ordinarily utilized methodologies comprise of analysis (i.e. need
examination, errand investigation), planning (i.e. target outline), selecting (i.e.
picking fitting learning/showing routines and proper evaluation strategy)
development (i.e. development of the educational module usage board/ educational
module assessment panel) and checking (i.e. educational module survey advisory
group).
Effectiveness Model for Curriculum Development Indicators, for instance, a
modern theory of curriculum development, holds that teachers as educational
experts actively construct subjective feedbacks of what they are teaching based on
their constant interaction on the daily basis with students (Bradley, 1985).
According to this paradigm, an adequately effective curriculum must meet the
expectation of being consistent, responsive and responsible to its clients. This
sophisticated set of indicators, however, was not the pioneer in the field of
educational evaluation.
Many other theories have long been standing as utile tools for curriculum
assessment, such as Objectives- Centred Model (Tyler, 1950) providing a relatively
easy steps for overall controlling of curriculum development; Context, Input,
Process and Product Model (Stufflebeam, 1971) with tremendous improvements
12
from the aforementioned theory; Goal- Free Model (Scriven, 1972) being the first to
question the assumption that goals or objectives are crucial in the evaluation
process; Responsive Model (Stake, 1975) making a major contribution by noting
that the concerns of the stakeholders should be paramount first and Connoisseurship
Model (Eisner, 1979) which emphasizes qualitative appreciation. As the movement
of constantly refined curriculum effectiveness and accuracy that has been there for
more than half of the century, all of these ideas are bombarding those responsible
for decision making in such matters.
In this research, there are six of the most significant works being presented
and scanned as the primary research background. Each of these mentioned tools
shall be presented through its (1) definition, (2) characteristics of each model; (3)
advantages and disadvantage of the model, with hope to form a big picture of
existing curriculum evaluation systems in the world.
1.2.1 Bradley’s Effectiveness Model for curriculum Development Indicators
The Bradley’s effective model for curriculum development indicators is a set
of measurable checkpoints, with simple responses (yes or no) to identify whether
the current curriculum plan is set properly or not. The full version of this model is
presented in the table below.
Table 1.1: Bradley’s Effectiveness Model for curriculum Development
Indicators
Indicator
Description
Yes or
No
Vertical
The course of study reflects a K-12 format that enables
curriculum
teachers to have quick and constant access to what is being
continuity
taught in the grade levels below and above them. Also,
upward spiraling prevents undue or useless curricular
repetition
Horizontal
The course of study developed provides content and
curriculum
objectives that are common to all classrooms of the same
continuity
grade level... also, daily lesson plans reflect a commonality
13
for the same grade level
Instruction
based
Lesson plans are derived from the course of study, and
on curriculum materials used are correlated with the content,
curriculum
objectives, and authentic tasks developed
Curriculum
Philosophical and financial commitments are evident.
priority
Clerical assistance is provided and reasonable stipends are
paid to teachers for work during the summer months. in
addition, curriculum topics appear on school board agendas,
administrative
meetings
agendas,
and
building=staff
meeting agendas
Broad
Buildings in the district have teacher representatives on the
involvement
curricular committees, elementary, middle level or junior
high, and high school principals (or designees) are
represented; and school board members are apprised of and
approve the course of study
Long-range
Each program in the district is included in the 5-tear
planning
sequence and review cycle. Also, a philosophy of education
and theory of curriculum permeate the entire school district
Decision
Controversies that occur during the development of a
making
program center on the nature of the decision, and not on
clarity
who makes the decision
Positive
Also, the initial thoughts about the curriculum come from
human
teachers, principals, and the curriculum leader. All
relations
participating members are willing to risk disagreeing with
anyone else,; however, communication lines are not allowed
to break down
Theory-into-
This district philosophy, vision, mission, exit (graduation)
practice
outcomes, program philosophy, rationale statement, program
approach
goals, program objectives, learning outcomes, and authentic
tasks are consistent and recognizable.
Planned
Tangible evidence shows that the internal and external
14
change
publics accept the developed program course of study for
the school district. The process of developing a course of
study for each program or discipline in a school district is no
longer one of determining how to do it, but one of
determining how to do it better
Source: Glatthorn, 2015
The indicators for effective curriculum development represent working
characteristics that any complex educational institution must have in order to be
responsive and responsible to its clients (Wiles. J, 2002). Further, the measurement
can also be oriented to meet the needs of school regardless of scales, which can
focus on a specific evaluation of a curriculum aspect, such as literature, language,
math, or any different content area. The obvious advantage of this model is its
simplicity and readiness to use. However, being as simple as it is, the model is
mostly appropriate for overall assessment and not detailed enough to serve as a
thorough tool of curriculum evaluation.
1.2.2 Tyler’s Objectives-Centered Model
One of the soonest curriculum evaluation models, which keeps on affecting
numerous evaluation activities, was that proposed by Ralph Tyler (1950) in his
work Basic Principles of Curriculum and Instruction. As elucidated in this work and
used as a piece of evaluation attempts, the Tyler approach moved sensibly and
purposely through a couple of related steps:
(1) Begin with the behavioral objectives that have been previously determined.
Those targets ought to indicate both the substance of learning and the understudy
conduct expected as “Demonstrate familiarity with dependable sources of
information on questions relating to nutrition.”
(2) Identify the situations that will give the student the opportunity to express the
behavior embodied in the objective and that evoke or encourage this behavior.
(3) Select, modify, or build suitable assessment instruments, and check the
instruments for objectivity, dependability, and legitimacy.
(4) Use the instruments to obtain summarized or appraised results.
15
(5) Compare the results obtained from several instruments before and after given
periods in order to estimate the amount of change taking place.
(6) Analyze the results in order to determine strengths and weaknesses of the
curriculum and to identify possible explanations about the reason for this particular
pattern of strengths and weaknesses.
(7) Use the results to make the necessary modifications in the curriculum
(Glatthorn, 1987)
The Tyler model has several advantages, firstly as it is relatively easy to
understand and apply. It focuses attention on curricular strengths and weaknesses,
rather than being concerned merely with the performance of each student. The most
striking matter here is it also emphasizes the importance of a continuing cycle of
appraisal, examination, and constant improvement.
However, according to Guba and Lincoln (1981), it still bears plenty of
pitfalls, namely: not suggesting how the objectives themselves should be evaluated
nor providing standards or suggest how standards should be developed. Else, its
accentuation on the earlier explanation of targets may confine creativity in
curriculum development, which appears to place undue accentuation on the preassessment
and
post-appraisal,
disregarding
totally
the
requirement
for
developmental evaluation. By the same token, Baron and Boschee (1995), in their
book Authentic Assessment: The Key to Unlocking Student Success, stress that “we
are encountering fundamental changes in the way we view and conduct assessment
in schools” (p.1).
1.2.3 Stufflebeam’s Context, Input, Process, Product Model
These obvious weaknesses in the Tyler model led several evaluation
experts in the late 1960s and early 1970s to criticize the Tyler model and to
offer their own alternatives. The alternative that had the greatest impact was that
developed by a Phi Delta Kappa committee chaired by Daniel Stufflebeam (1971).
This model seemed to appeal to educational leaders because it emphasized the
importance of producing evaluative data for decision making; in fact, decision
making was the sole justification for evaluation, in the view of the Phi Delta Kappa
committee.
16
To service the needs of decision makers, the Stufflebeam model provides a
means for generating data relating to four stages of program operation: context
evaluation, which continuously assesses needs and problems in the context to help
decision makers deter- mine goals and objectives;
input evaluation,
which
assesses alternative means for achieving those goals to help decision makers choose
optimal means; process evaluation, which monitors the processes both to ensure
that the means are actually
being implemented and to make the necessary
modifications; and product evaluation, which compares actual ends with intended
ends and leads to a series of recycling decisions.
During each of these four stages, specific steps are taken:
(1)
(2)
(3)
(4)
(5)
(6)
The kinds of decisions are identified.
The kinds of data needed to make those decisions are identified.
Those data are collected.
The criteria for determining quality are established.
The data are analyzed on the basis of those criteria.
The needed information is provided to decision makers. (Glatthorn, 1987, pp.
273–274)
The context, input, process, product (CIPP) model, as it has come to be
called, has several attractive features for those interested in curriculum evaluation.
Its emphasis on decision making seems appropriate for administrators concerned
with improving curricula. Its concern for the formative aspects of evaluation
remedies a serious deficiency in the Tyler model. Finally, the detailed guidelines
and forms created by the committee provide step- by-step guidance for users. The
CIPP model, however, has some serious drawbacks associated with it. Its main
weakness seems to be its failure to recognize the complexity of the decision-making
process in organizations. It assumes more rationality than exists in such situations
and ignores the political factors that play a large part in these decisions. Therefore,
as Guba and Lincoln (1981) noted, it seems difficult to implement and expensive to
maintain.
1.2.4 Scriven’s Goal-Free Model
Michael Scriven (1972) was the first to question the assumption that goals or
objectives are crucial in the educational evaluation process. After his involvement in
17
several evaluation projects where so-called side effects seemed more significant
than the original objectives, he began to question the seemingly arbitrary distinction
between intended and unintended effects. His goal-free evaluation model was the
outcome of this dissatisfaction. In conducting a goal-free evaluation, the evaluator
functions as an unbiased observer who begins by generating a profile of needs for
the group served by a given program. Only then, by using methods that are
primarily qualitative in nature, the evaluator assesses the actual effects of the
program. If a program has an effect that is responsive to one of the identified needs,
then the program is perceived as useful.
Goal-free evaluator, therefore, avoids learning the stated purpose, goals or
intended achievements, of the program prior to or during the evaluation. Instead, he
observes and measures actual processes and outcomes; and interviews program
consumers, which prevents only looking at the program as it pertains to the intended
goals at the risk of overlooking many positive and/or negative unintended sideeffects. The evaluator, subsequently, asks: “What does the program actually do?”
rather than “what does the program intend to do?” As being stated in his most
notable work, “Merit is determined by relating program effects to the relevant needs
of the impacted population, (Scriven, 1991. p. 180).” a comprehensive needs
assessment should be conducted simultaneously with data collection. “The
evaluator should provide experiential accounts of program activity so that readers of
the report can, through naturalistic generalization, arrive at their own judgments of
quality in addition to those the evaluator provides, (Stake, 2004, p. 215).”
With this new approach, there stand a host of critical advantages for the
utilization of this tool. Firstly, it may identify unintended positive and negative sideeffects and other context specific information. As a supplement to a traditional
evaluation, it serves as a form of triangulating both data collection methods and data
sources. Secondly, it circumvents the traditional outcome evaluation and the
difficulty of identifying true current goals and true original goals, and then defining
and weighing them.
18
By the same token, it is, therefore, less intrusive to the program and
potentially less costly to the client. By reducing interaction with program staff, it is
less susceptible to social, perceptual, and cognitive biases; is reversible; an
evaluation may begin goal- free and later become goal-based using the goal-free
data for preliminary investigative purposes. Moreover, it is less subject to bias
introduced by intentionally or unintentionally trying to satisfy the client because it
is not explicit in what the client is attempting to do; it offers fewer opportunities for
evaluator bias or corruption because the evaluator is unable to clearly determine
ways of cheating. For the evaluator, it requires increased effort, identifies
incompetence, and enhances the balance of power among the evaluator and client.
Lastly, and perhaps most importantly, its focuses on human experience and
what people actually do and feel, allows for understanding how program
implementer deal with its non-programmed decisions (Stake, 2004), which should
be understood as decisions regarding relatively novel problems, or problems that an
individual, group, organization, or entity has never encountered (George & Jones,
2000).
Scriven’s main contribution, obviously, was to redirect the attention of
evaluators and administrators to the importance of unintended effects—a redirection
that seems especially useful in education. If a mathematics program achieves its
objectives of improving computational skills but has the unintended effect of
diminishing interest in mathematics, then it cannot be judged completely successful.
Scriven’s emphasis on qualitative methods also seemed to come at an opportune
moment, when there was increasing dissatisfaction in the research community with
the dominance of quantitative methodologies.
As Scriven himself notes, however, goal-free evaluation should be used to
complement, not supplant, goal-based assessments. Used alone, it cannot provide
sufficient information for the decision maker. Some critics have faulted Scriven for
not providing more explicit directions for developing and implementing the goalfree model; as a consequence, it probably can be used only by experts who do not
require explicit guidance in assessing needs and detecting effects.
19
1.2.5 Stake’s Responsive Model
Robert Stake (1975) made a major contribution to curriculum evaluation in
his development of the responsive model, because the responsive model is based
explicitly on the assumption that the concerns of the stakeholders—those for
whom the evaluation is done—should be paramount in determining the evaluation
issues. He made the point this way: “To emphasize evaluation issues that are
important for each particular program, I recommend the responsive evaluation
approach”. It is an approach that trades off some measurement precision in order to
increase the usefulness of the findings to persons in and around the program. An
educational evaluation is a responsive evaluation “if it orients more directly to
program activities than to program intents; responds to audience requirements for
information; and if the different value perspectives present are referred to in
reporting the success and failure of the program” (p.14). Stake recommends an
interactive and recursive evaluation process that embodies these steps:
(1) The evaluator meets with clients, staff, and audiences to gain a sense of their
perspectives on and intentions regarding the evaluation.
(2) The evaluator draws on such discussions and the analysis of any documents
to determine the scope of the evaluation project.
(3) The evaluator observes the program closely to get a sense of its operation
and to note any unintended deviations from announced intents.
(4) The evaluator discovers the stated and real purposes of the project and the
concerns that various audiences have about it and the evaluation.
(5) The evaluator identifies the issues and problems with which the evaluation
should be concerned. For each issue and problem, the evaluator develops an
evaluation design, specifying the kinds of data needed.
(6) The evaluator selects the means needed to acquire the data desired. Most
often, the means will be human observers or judges.
(7) The evaluator implements the data-collection procedures.
(8) The evaluator organizes the information into themes and prepares
“portrayals” that communicate in natural ways the thematic reports. The portrayals
may involve videotapes, artifacts, case studies, or other “faithful representations.”
20
- Xem thêm -
.DOCX 
75 
76 
129 
