EXPERIMENTAL RESEARCH AND DESIGN IN EDUCATION
(Only for M.Ed. Students, Teacher Educators and Educational researchers.This article is not meant for B.Ed. students.)
Prepared by
SABARISH-P
M.Sc., M.Ed., JRF & NET
Assistant Professor in Physical Science, Arafa Institute for Teacher Education
Attur, Thrissur.
INTRODUCTION:
Experimental Research is an attempt by the researcher to maintain
control over all factors that may affect the result of an experiment. In doing
this, the researcher attempts to determine or predict what may
occur.experimental Design – is a blueprint of the procedure that enables the
researcher to test his hypothesis by reaching valid conclusions about
relationships between independent and dependent variables. It refers to the
conceptual framework within which the experiment is conducted.
Steps involved in conducting an
experimental study
·
Identify
and define the problem.
·
Formulate
hypotheses and deduce their consequences.
·
Construct
an experimental design that represents all the elements, conditions, and
relations of the consequences.
1 .Select sample of subject.
2. Group or pair subjects.
3. Identify and control non experimental factors.
4. Select or construct, and validate instruments to measure outcomes.
5. Conduct pilot study.
6. Determine place, time, and duration of the experiment.
§ Conduct the experiment.
§ Compile raw data and reduce to usable
form.
§ Apply an appropriate test of
significance.
Some terms and symbols
1. Experimenter(E):The person who
manipulates the experimental conditions.
2. Subject(S):The living organism that
is studied.
3. Independent /treatment/antecedent variable(IV,X):The
manipulated variable.It may be a teaching method,a teaching aid etc
4. Dependent/criterion/predicted variable(Y):Iit is the condition or
characteristic that changes as the experimenter changes the X.
5. Control group(C):The group that does
not receive any experimental treatment.
6. Experimental group(E):The group that
is given the independent variable treatment.
7. Pre-test(T1):Test before the
application of X.
8. Post-test(T2):Test after the
application of X.
The Three Important Criteria In
Selecting an experimental design
1.Appropriateness
The
researcher should select the design that will do the job it is supposed to do
and is able to arrange objectively the experimental conditions to meet the
requirements of his study.
2.Adequacy
of control
It must
provide adequate control so that the effects of the independent variable on the
dependent variable can be measured.
3.Validity
of Experimental Design
Internal
Validity :The extraneous variables that affect the control of a design.
External
Validity:Concerned with the generalizability or representativeness of
the experimental findings.
Threats to Internal Validity
1. History:Unplanned or extraneous events that
occur during the research and affect the results.
Example: Researcher collects gross sales data before and after a 5 day 50% off
sale. During the sale a hurricane occurs and results of the study may be
affected because of the hurricane, not the sale.
2. Maturation–Occurs when an effect is due to
maturational or other natural changes in the subjects.(being
older,wiser,stronger,tired)
Example: Subjects become tired after completing a training session, and their
responses on the Posttest are affected.
3. Pre-testing :Occurs when the act of taking a
test or responding to a questionnaireaffects the subjects.
Example: Subjects take a Pretest and think about some of the items. On the
Posttest they change to answers they feel are more acceptable. Experimental
group learns from the pretest.
4. Measuring Instruments - Changes in instruments, calibration
of instruments, observers, or scorers may cause changes in the measurements.
Example: Interviewers are very careful with their first two or three interviews
but on the 4th, 5th, 6th become fatigued and are less careful and make errors.
5. Statistical Regression :Aresult may be due to respondent’s
being identified on the basis of extreme high or low scores.
Example: Managers who are performing poorly are selected for training. Their
average Posttest scores will be higher than their Pretest scores because of statistical
regression, even if no training were given.
6. Differential Selection - Different individuals or groups
would have different previous knowledge or ability which would affect the final
measurement if not taken into account.
Example: A group of subjects who have viewed a TV program is compared with a
group which has not. There is no way of knowing that the groups would have been
equivalent since they were not randomly assigned to view the TV program.
7. Experimental Mortality - The loss of subjects from
comparison groups could greatly affect the comparisons because of unique
characteristics of those subjects. Groups to be compared need to be the same
after as before the experiment.
Example: Over a 6 month experiment aimed to change accounting practices, 12
accountants drop out of the experimental group and none drop out of the control
group. Not only is there differential loss in the two groups, but the 12
dropouts may be very different from those who remained in the experimental
group.
8. Interaction of Factors, such as Selection Maturation, etc. -
Combinations of these factors may interact especially in multiple group
comparisons to produce erroneous measurements.
9. Experimenter bias: Attributes or expectations of
researcher that influence results. Attributes: Age, sex, race, status, hostility,
authoritarianism, physical appearance
Expectations: Deliberate or
unintentional effects of bias on part of researcher that results in treating subjects differently.
10.
Diffusion of Treatment:
Treatment given to experimental group affects control group
(contamination).
Example:
Members of the two groups are aware of the research or share information about
treatment with one another. Best if groups do not know about one another1
11 .Subject EffectsChanges in subjects’ behavior in
response to the experimental situation. Alter behavior to
respond more favorably.
Example:Hawthorne
effect (subjects respond differently because aware being studied). John Henry effect (control group may
compete or overcompensate). Demoralization
effect (control group feels neglected, puts forth less effort or gives up
). Novelty effect (subjects more
motivated, participatory because new and different)
Threats
to External Validity or Generalizability
Ø POPULATION VALIDITY:
I.
Selection of subjects-Generalization is limited to the
subjects in the sample if subjects are not selected randomly from an identified
population.
II.
Characteristics of subjects-Generalization is limited to the
characteristics of the sample or population (socioeconomic status,age,location,ability,race)
III.
Subject-treatment interaction-Generalization may be limited
because of the interaction between the subjects and treatment.
Ø ECOLOGICAL VALIDITY:
I.
Description of variables-Generalization is limited to the
operational definitions of the independent and dependent variable.
II.
Multiple treatment interference-In experiments in which subjects
receive more than one treatment, generalizability is limited to similar
multiple treatment situations because of the effect of first treatment on
subsequent treatments.
III.
Setting-treatment interaction-Generalization is given to the
setting in which the study is conducted.(room,time of day,others presence,other
surroundings etc)
IV.
Time of measurement treatment
interaction-Results
may be limited to the time frame in which they were obtained.Treatments causing
immediate effects may not have lasting effects.
V.
Pretest-post test sensitization-They may interact with the treatment
so that similar results are obtained only when the testing conditions are
tested.
VI.
Novelty or disruption effect-Subjects may respond differently
because of a change in routine and generalization may be limited to situations
that involve similar novelty .
Types of experimental designs
A set of commonly used symbols are
X = The exposure of an independent
variable to a group of test subjects for which the effects are to be determined
O = The process of observation or
measurement of the dependent variable (effect outcome) on the test subjects
R = The random assignment of test
subjects to separate treatment groups
Classification of experimental designs
1.Pre-experimental
designs
a.The one
shot case study
b.The one
group pretest -post test study
c.The static
group comparison design
2.True
experimental designs
a. The post test only equivalent group
design
b. The pretest posttest equivalent group
design
c. The two group non random pre test
post test design
d. The Solomon four group design
3.Factorial
designs
a) Simple factorial design of 2 by 2
4Quasi-experimental
designs
a. The pretest posttest nonequivalent group design Counter balanced
design
b. Counter balanced design
5.Time-series designs
a.One –Group
time-Series Design.
b.Control –Group Time- Series Design
Pre experimental design
Pre-
experimental design represents the crudest form of experimentation and is
undertaken only when nothing stronger is possible. The designs are
characterized by and absence of randomization of test subjects. The
pre-experimental designs are weak in their scientific measurement power because
they fail to control adequately the various threats to the internal validity
1.The "One-Shot Case Study"(inadequate
control of extraneous variables)
A single
group of test subjects is exposed to the independent variable treatment X, and
then a single measurement on the dependent variable is taken .One-Shot case
study does not use pre test and control group. As a result this design is
inadequate for establishing causality. For eg. A study on the employee
education campaign about the automation of the office activities without a
prior measurement of employee knowledge. Result would reveal only how much the
employees know after the education campaign, but there is no way to judge the
effectiveness of the campaign
Treatment Post-test
X O
No
control group. This design has virtually no internal or external validity.
2.One group Pre-test, Post-test
design
Minimal
Control. There is somewhat more structure, there is a single selected group
under observation, with a careful measurement being done before applying the
experimental treatment and then measuring after. This design has minimal
internal validity, controlling only for selection of subject and experimental
mortality. It has no external validity.
Continuing the example given above it measures the awareness of the employees
before the campaign and after the campaign
Pre-test Treatment Post-test
O X O
3.The static group comparison design
Compares the
status of a group that has received anexperimental treatment with one that has
not.The limitation is that there is no provision for establishing the
equivalence of the experimental and control group.
Treatment Post-test
X O
C O
The main
advantage of this design is randomization. The post-test comparison with
randomized subjects controls for the main effects of history, maturation, and
pre-testing; because no pre-test is used there can be no interaction effect of
pre-test and X. Another advantage of this design is that it can be extended to
include more than two groups if necessary.
For eg in a field setting an experiment is designed to study the effect of
a natural disaster (experimental treatment) on the psychological trauma
(measured outcome). A pre-test before the natural disaster say tsunami is
possible but not on a large scale. Moreover the timing of the pretest would be
problematic. The control group, receiving the post-test would consist of
subject whose property is safe.
True experimental design/lab
experiment
The experiments performed in an
artificial or contrived environment is known as lab experiments. In the lab
experiments the researcher has complete control over all aspects. The
researcher has a control over the experiment, who, what, when, where and how.
1.Two
groups ,randomized subjects,post test only design
Test
subjects are randomly assigned to either the experimental or control group. The
experimental group is then exposed to the independent treatment, after which
both group receive a post treatment measure of the dependent variable. In this
design, the pretest measurements are omitted. The design is
R X
O1
R
CO2
The experiment effect is measured by the
difference between O1 and O2. The design is more simple and attractive.
Internal validity threats from the history, maturation, selection and
statistical regression are adequately controlled by random assignment. Since
the subjects are measured only once, the threats of testing and implementation
are handled. The different mortality rates between experimental and control
groups continue to be a problem. The design reduces the external validity
problem of testing interaction effect.
2.Two
groups, Random Selection, Pre-test, Post-test
Group Pre-test Treatment Post-test
Expl groupE(R) O X O
Control
gpC(R) O C O
The
advantage here is the randomization, so that any differences that appear in the
posttest should be the result of the experimental variable rather than possible
difference between the two groups to start with. This is the classical type of experimental
design and has good internal validity. The external validity or
generalizability of the study is limited by the possible effect of pre-testing.
3.Two groups,
Nonrandom Selection, Pre-test, Post-test
Group Pre-test Treatment Post-test
Expll gp
E(R) O XO
Control gpC(R) O CO
The main
weakness of this research design is the internal validity is questioned from
the interaction between such variables as selection and maturation or selection
and testing. In the absence of
randomization, the possibility always exists that some critical difference, not
reflected in the pretest, is operating to contaminate the posttest data. For example, if the experimental group
consists of volunteers, they may be more highly motivated, or if they happen to
have a different experience background that affects how they interact with the
experimental treatment - such factors rather than X by itself, may account for
the differences.
4.Solomon Four-Group Design
Group Pre-test Treatment Post-test
Pre-tested
Eptll Gp E (R) O X O
Pre-tested
Controlgp C (R) O O
Unpre-tested
Eptll Gp UE (R) X O
Unpre-tested
Control GpUC (R) O
This design
overcomes the external validity weakness in the above design caused when
pre-testing affect the subjects in such a way that they become sensitized to
the experimental variable and they respond differently than the unpre-tested
subjects.
Quasi
experimental design
In a
quasi-experimental design, the research substitutes statistical
"controls" for the absence of physical control of the experimental
situation.
1,Non
equivalent pretest posttest design:
The most
common quasi-experimental design is the non equivalent group Pre-test/Post-test
Design. This design is the same as the
classic controlled experimental design except that the subjects cannot be
randomly assigned to either the experimental or the control group, or the
researcher cannot control which group will get the treatment. In other words, participants do not all have
the same chance of being in the control or the experimental groups, or of
receiving or not receiving the treatment.
This design
can be diagrammed as follows:
Pretest treatment posttest
O1
X O2
O3CO4
Comparison
of the pre test result (O1 – O2) is one indicator of the degree of equivalence
between test and control groups. If the pre test results are significantly
different, there is a real question about comparability. On the other hand, if
pretest observations are similar between groups, there is more reason to
believe internal validity of the experiment is good.
The
counterbalanced design
It may be used when the random assignment of
subject to experimental group and control group is not possible. This design is also known as rotation group
design. In counterbalanced design each
group of subject is assigned to experimental treatment at different times
during the
experiement.This
design overcomes the weakness of non-equivalent design. When intact groups are used, rotation of
groups provides an opportunity to eliminate any differences that might exist
between the groups. Since all the groups
are exposed to all the treatments, the results obtained cannot be attributed to
the preexisting differences in the subjects.
The limitation of this design is that there is carry-over effect of the
groups from one treatment to the next.
Therefore, this design should be used only when the experimental
treatments are such that the administration of one treatment on a group will
have no effect on the next treatment.
There is possibility of boring students with repeated testing.
Replication o1X1
O2X2 O3X3 O4X4
1 Group A
B C D
2 GroupB D A C
3 GroupC A D B
4 GroupD C B A
factorial
design
More than
one independent variables can be manipulate simultaniously. Both indepentent and interaction effects of
two or more than two factors can be studied.
Experiments may be designed to study simultaneously the effects of two
or more variables. Such an experiment
are called factorial experiment.
Simple
Factorial Design:
A factor is
a major independent variable . A level is
a subdivision of a factor. A simple
factorial design is 2x2 factorial design.
In this design there are two independent variables and each os the
variables has two levels. One advantage
is that information is obtained about the interaction of factors. Both independent and interaction efects of
two or more than two factors can be studied with the help of this factorial
design. With two factors each taking two
levels, a factorial experiment would have four treatment combinations in total,
and is usually called a 2x2 factorial design.
The first independent variable, which is manipulated, has two values
called the experimental variable. The
second independent variable, which is divided into levels, may be called
control variable. For example, there are two experimental treatments, that is, teaching through co-operative
learning and teaching through lecture method.
It is observed that there may be different levels of intelligence of the
students. On the basis of the IQ score
the experimenter divides the students into two groups:one high intelligent
group and the other the low intellegent group.
There are four groups of students within each of the two levels of
intelligence.
High
Intelligence Group
|
Low
Intelligence Group
|
||
Teaching
Through Co-operative Learning Method
|
Gain
Score on the Dependent Variable
|
Gain
Score on the Dependent Variable
|
|
Teaching Through Lecture Method
|
Gain Score on the
Dependent Variable
|
Gain Score on the
Dependent Variable
|
|
Since one of the objectives is to compare various
combinations of these groups, the experimenter has to obtain the mean scores for
each row and each column. The
experimenter can first study the main effect of the two independent variables
and the interaction effect between the intelligence level and teaching method
TIME
SERIES DESIGN
A research
design in which measurements of the same variables are taken at different
points in time, often with a view to studying social trends. For this reason
such designs are sometimes also known as trend designs and are distinguishable
from ‘one case post test only designs in which measurements are taken only
once. Time series designs can be used in conjunction with official data, for
example by plotting crime rates for the same area but for different point in
time (monthly, quarterly, annually). This acts as a basis for making statements
about trends in levels of crime. It is possible to plot the trends for different
variables at one and the same time with a view to making inferences about their
relationship, for example, to map unemployment rates against crime ...
Time
–Series Designs
There are two types of time –series
designs.
1.One
–Group time-Series Design.
A series of
measurement on the dependent variable are taken before and after the group is
exposed to experimental treatment. The
experimenter takes a number of measurements on the independent variable Y,
exposes the group to the experimental treatment X, and then again takes additional measurements (T) on the independent variable Y.
Advantages:
This design is useful in the school settings
to study the effects of a major change in administrative policy upon various
issues concerning discipline. It is also useful in the study of attitude change
in the students as a result of the effect produced by the introduction of a
documentary film designed to change attitudes.The multi-testing of students in
this design provides a check on some sources of internal validity than in Design
1.
Limitations:
1.This
design fails to control the effects due to history. For example, the factors such as climatic
changes,examinations may contribute to the observed change in the dependent
variable.
2. Because
of the repeated tests, there may be a kind of interaction effect of testing
that would restrict the findings the those populations which have been
subjected to repeated testing.
3.The usual
statistical tests of significance may not be appropriate with a time design.
2 Control –Group Time- Series Design
Control-group Time-Series design is an
extension of One-group Time-Series Design.
To overcome the limitations of
One-group Time-series Design, this design utilizes a control group. The control group is also an intact class
group. This group like the experimental
group is tested on the dependent variable at the same intervals of time, but is
not exposed to the experimental treatment.
Advantages:
1.This
design overcomes the weakness of One-group Time-Series Design by controlling
the effects due to history.
2.The
inclusion of control group in this design is useful for necessary comparison.
Limitations:
There may be
interaction effect due to repeated tests and this would restrict the findings
to those populations which have been subjected to repeated testing. The usual
statistical techniques may not be applicable with such designs.
Conclusion:
Experimentation is a sophisticated technique
for problem solving and may not be an appropriate activity for the beginning
researcher . It has been suggested that
teachers may make their most effective contribution to educational research by
identifying important problems that they encounter in their classrooms and by
working cooperatively with research specialists in the conduct and
interpretation of classroom experiments.
Reference
1,Reserch in
education: James H. McMillan &
Sallly Schumacher.
2.Methodology
of Educatin Research: Lokesh Koul.
3.Research
in eduction: John W. Best and James V.
Khan.
4Understanding
Educational Research: An Introductionb:
Deobold B Vandalen.
5.Introduction to
educationl Research: C.M. Charles.
