Vol. 16 : No. 2< >
Editor's Note: This research paper was presented
live at the Teachers Develop Teachers' Research - TDTR5 - Conference, Middle
East Technical University, Ankara, Turkey, September 9, 2001. It addresses the
crucial problems of course design, levels of feedback, and teacher satisfaction.
The results have significant implications for teacher selection, training, and
Teachers Feel Contented with Online Courses
Abramov, Natalie Martkovich
Our aim was to examine
levels of satisfaction of English teachers (currently taking online courses or
having taken such courses in the past), and determine the factors influencing
often, online course designers do not get sufficient feedback on levels of
satisfaction of their courses’ participants – feedback that might otherwise
bring forth some improvement in the course design and delivery approaches, as
well as in proper placement testing procedures.
We were especially interested in observing
some connection between confidence levels of participants (i.e., to what extent
they felt computer-fluent) and their satisfaction levels, in order to find out
whether this connection indeed exists, and if so, what sort of relation it is
– positive or negative.
Professional experience is a variable which
might influence satisfaction levels: the longer this teacher has been in the
field, the more experience he/she might have accumulated to make learning easier
and more enjoyable.
On the other hand, those who have been
working for only a few years may have retained their learning skills; their
learnability may be higher, hence their satisfaction level may be expected to be
greater as well.
Such factors as age, country of birth, being
a native vs. non-native speaker of English, cultural background, educational
status, etc. may play a certain role in determining satisfaction levels and have
to be taken into consideration when predicting satisfaction levels.
However, we found the main factors
influencing satisfaction levels to be the delivery mode and the collaboration
aspect. It also appears that communication with instructors and peers plays a
decisive role in determining satisfaction levels of participants.
It all began last winter when my online
students started complaining, which happened just a few weeks after the course
had begun. Normally, one expects a few unhappy students in every group, but in
this case, the students were themselves teachers, some of them with lots of
experience. Just look at their complaints:
When I was accepted into the course, I was told that my computer
skills were sufficient, and that I'd learn along the way. I feel like a very
weak pupil who doesn't understand many of the things posted; the language isn't
familiar to me, and I'm not sure how to proceed. I now know how students feel
when something isn't clear. Very frustrating and demoralizing.
We have had severe computer problems. My computer has been infected with
a virus that was extremely difficult to get rid of, and Debby's computer
crashed. Rodika has been ill, and Orit seems to have disappeared! Quite frankly,
something to which I was really looking forward turned into a nightmare, with
due dates coming nearer and nearer, and more things to do pouring in, with no
way of doing anything about it!
Help!!! Why did you make me a group leader????!!!!! This is a
grave mistake. I am at minus-basic level. Please reply.
am writing to you because I have a problem with the group. Since we received our
assignment on Tuesday, I have written 3 e-mails off to Victoria and Nataly. I
have received no reply, letter, or any word from them. I am unable to take
responsibility for others who don't want to be involved.
I cannot understand why this has developed into such an issue! I see now
that being in one group with Jenny after this will be pretty unpleasant. I don't
think she should have run to you complaining before she even knows how I usually
work and what has happened.
was definitely going wrong in the logistics department. We decided to research
the topic. The initial idea was to collect the data from the existing pool of
students by e-mail questionnaire; but as we realized that the problem was fairly
universal, we came to a conclusion that it would be better to extend our survey,
making it available to a wider audience.
survey questionnaire was created and posted on the Web as an interactive (clickable)
form. The survey consisted of three parts:
One: Description of the course
· How was the communication with the instructor conducted?
· When were you informed about the course requirements?
· What was the nature of peer collaboration in group assignments?
· When working on group assignment, what was your role in decision-making?
· What level of computer proficiency did the course require?
· What was the level of your computer proficiency at the beginning of the course?
· Was there an improvement in the level of your computer proficiency when you finished the course?
· Was the course material studied easy or hard?
· What was the most important aspect learned during the course?
· If an improvement in your computer proficiency occurred, what caused it?
· When you submitted an assignment for assessment, did you get a feedback on time?
· When you would send a question to your instructor, did you get an answer on time?
· Was the grading system clear and objective?
· To what extent was the assessment of your assignments fair?
Two: Satisfaction with the course
general, how can you define your level of satisfaction with the course taken?
on your experience, will you take any online courses in the future?
questions will you ask before making a decision to take an online course in the
Three: What was the influence of these factors on your satisfaction with the
Your insufficient computer
proficiency level at the beginning of the course
The improvement in your computer
proficiency caused by the course
Your previous experience with
New skills learned, or old skills
The content of the courses
Timeliness of instructor’s
answers to your questions and feedback on your assignments
Fairness of assessment
The emotional aspect of
communication with the instructor
Effectiveness of individual
tutoring over e-mail
The necessity to participate in
Your role in groupwork
· Workload distribution
had a fairly short period of time to collect the data; fortunately, the mailing
lists where we posted the request to fill in the questionnaire, and the Web as a
medium turned out to be very efficient. The results thus obtained are presented
were 70 respondents; those who identified their origin were from 8 countries:
Satisfaction levels were distributed the following way:
to medium 2%
to high 48%
The following methods were used to communicate with the
Participants were informed about prerequisites
before registering for the course - 73%
after registering for the course – 27%
The course material studied was perceived by the participants as
easy - 3%
The participants' questions were answered on time:
Assignments were graded on time:
Were the answers helpful and comprehensive?
helpful and comprehensive 33%
and comprehensive 41%
helpful and comprehensive 22%
very helpful and comprehensive 5%
helpful and comprehensive at all 0%
The answers were encouraging:
Was the grading system clear and objective?
To what extent was the assessment of your assignments fair?
most cases 42 %
part of the cases 6 %
never 0 %
The most important aspect learned during the course was:
The main cause of improvement in computer-related skills was:
Influence Satisfaction with the Course
the influence of the following factors on your satisfaction with the course?
The necessity to participate in group assignments
little influence 16%
Your role in groupwork decision-making
little influence 16%
little influence 14%
Insufficient computer proficiency level at the beginning of the
not apply 74%
little influence 6%
The improvement in computer proficiency caused by the course
not apply 30%
little influence 5%
Previous experience with online courses
not apply 54%
little influence 7%
New skills learned, or old skills improved
not apply 9%
little influence 3%
The content of the course
not apply 5%
little influence 2%
Timeliness of instructor’s answers to questions and feedback on
little influence 13%
Fairness of assessment
little influence 9%
The emotional aspect of communication with the instructor
little influence 13%
Efficiency of individual tutoring over e-mail
little influence 11%
to take DL courses in the future:
The majority of the respondents who indicated that their initial computer
skills were higher (intermediate+ or advanced) than required for the course they
took (intermediate), and that – quite naturally – there was no improvement
in their computer proficiency, belonged to low satisfaction levels group. On the
contrary, most of the respondents reporting consistent initial skills deficiency
(mostly moderate – one step below the required) or adequacy of skills noted
that their computer-related skills improved towards the end of the course. Not
surprisingly, they belonged to medium-to-high and high satisfaction level
Thus we hypothesize
that a moderate gap between the initial skill level and the requirements can be
viewed as a positive motivational challenge; at the same time, overqualification
(having much higher computer skills than required for the course) can serve as a
negative outcome predictor.
placement testing, one should include some elements that would expose
overqualification. Such applicants should be screened and filtered: they are a
potential low satisfaction risk group!
It turned out that 27% of the respondents (~1/4) learned what the
prerequisites were only after actually having started the course. However, this
1:3 “informed/uninformed” ratio was relatively constant throughout all the
satisfaction level groups.
Timeliness of instructor response and assignment
For all respondents:
No or very little
influence - 20%
considerable influence or powerful influence – 80%
For respondents with
medium-to-high and high satisfaction levels:
¾ got their papers
graded on time;
8/9 received prompt
response from the instructor.
The character of the answers:
comprehensive: Reported by 4/5 of high- and medium-to-high- satisfaction
Reported by 9/10 of high- and medium-to-high- satisfaction respondents
Fair distribution of
team workload was consistently reported by 7/8 of high- and medium-to-high-
satisfaction respondents; 1/8 indicated that some worked more than the others
– these respondents probably were those who worked less, so they were still
quite happy with the courses…
low-satisfaction group, only about
half of the respondents reported fair workload distribution; it looks like for
this group, fairness of workload distribution was not among the decisive
In all satisfaction
level groups, most of the respondents noted their opinion was taken into
consideration, so it looks that, although generally being quite important, this
factor does not define satisfaction level directly.
While in lower
satisfaction groups only about 60% of the respondents noted that the grading
system was sufficiently clear and objective, in higher satisfaction groups
practically all respondents reported it; this factor appears to be consistently
connected with higher satisfaction levels.
earlier said, proficiency improvement does not directly correlate with
satisfaction levels. However, while in low-satisfaction groups this claim
remains valid, in higher-satisfaction groups the proficiency increase figures
are much higher.
question whether a respondent was actively using the new skills and knowledge
was included as a hidden verification tool against the emotional
self-assessment: persons who are really dissatisfied with a course will not use the stuff
learnt in it for their teaching. This turns out to be a correct assumption: only
about half of the low-satisfaction group reported using what they had learned,
while in the higher-satisfaction group, the figure rises to 4/5.
As you remember, we asked whether the respondent was planning to take
similar courses in the future.
For the whole population, 31% answered “maybe”,
and 69 gave an outright “yes”.
When analyzed separately for different satisfaction
level groups, this looks different:
In low-satisfaction groups, only 40% said “yes”,
with 60% “maybe”;
In higher-satisfaction groups, however, “yes”
doubled to 78%. Nothing surprising in this, of course, but it’s nice to know.
A colleague who has both taken and authored quite a
few online courses has noted recently: “Online courses are addictive – both
to take, and to develop and teach. “
The online form we used also contained a fill-in field:
what questions will you ask before entering another online course?
The answers to this included
lots of questions showing that people just do not read course prerequisites.
However, some of the questions were quite relevant and up to the point. We
collected and grouped them:
What types of online
interaction will be used?
Is communication by message
board or email?
the timetable be adhered to?
many assignments will be required and what is the schedule for submission?
many articles do we have to read?
many hours of work is expected? (The course I took was for 56 hours and the work
was at least double)
I spread the workload as suits me?
Will there be group work? Is
group work compulsory?
How much groupwork is required,
and how will it be evaluated?
Will I be put in a group
without being asked first or will I be able to choose my partners?
How will the instructor assign
groups and group work - will I have a say so in whom I want to work with?
will roles be allocated and equity in workload determined?
What is the background of the
other participants (computer and teaching experience)?
Is the instructor more a
teacher or a technologist?
How soon should I expect
feedback from the instructor?
Does the instructor know more
than I do? (I was almost at the same level as the instructor in the course I
To what extent is the
instructor willing to answer questions, explain and encourage?
How does the instructor see his
role and what is his general attitude to queries? Will he be flexible?
Availability of instructor,
methods of feedback.
What is the instructor's
experience of online learning and teaching?
Standards and expectations of
instructors; whether the presence of non-native English speakers will influence
those standards and expectations.
Has the course been trial-run
Is the course accredited?
What advantages will having
such a qualification provide?
What bodies will recognize
courses completed through this organization?
What are the professional
credentials of the online program providers, instructors, and administrators?
minimize uncertainty, which later results in dissatisfaction, course
descriptions should include:
prerequisites – skills, software and hardware; see example:
structure, schedule, procedures, workload, group work, communication protocols,
information concerning the faculty (course authors, instructors and
What graduates say about the course;
A link to a sample unit/module/lesson.
Ways to Achieve
Higher Satisfaction Levels
To ensure higher satisfaction levels,
special attention should be paid to the following aspects of the course:
Communication media: e-mail is by far the
most popular means of communication; learners seem to feel most comfortable with
it. In addition, e-mail is “push technology”, as opposed to “pull
technology” like message boards. If you want ALL your messages delivered and
received on time, PUSH!
About ¼ of online learners manage not to
have found out what the prerequisites are until after they have started the
course. It makes sense to create a kind of a “license agreement” listing all
the major points a learner has to know, and develop a mechanism that involves
some sort of confirmation sent by every learner before actually starting
To help the learner make sure the course
"fits" as regards qualifications vs. requirements, it is advisable
To develop an elaborate screening test that
enables the course administrator to screen applicants who are either severely
underqualified, or apparently overqualified; this applies not only to
computer-related skills, but to any other aspect as well;
To provide enough tools (online tutorials,
mini-training sessions, self-assessment tests, quizzes, etc.);
To pay special attention to performance of
learners during the first 2-3 weeks and offer additional help if you see some of
them might need it.
Encouragement should be used sparingly:
otherwise learners develop a feeling of being attended on by a babysitter, which
eventually affects their confidence and ability to learn independently. By no
means may an instructor show irritation – even if the learner keeps asking the
same question for the fifth time within one week.
Learners have different learning styles: some
are “social learners” and work best in teams; others prefer doing it alone.
Hence, unless the nature of an assignment dictates otherwise, the option of
working alone or joining a team should be left open. The first two or three
assignments done in instructor-appointed and reshuffled teams are enough for the
group to get to know each other; after that, whenever teamwork is assigned, it
is better to give the learners a chance to team up with someone they prefer.
A Frequently Asked Questions page will
drastically cut the number of questions the instructor has to answer.
should be informed that their papers will not be graded instantly – usually it
takes 4-5 days for the instructor to grade 25 papers. If this point is not
stressed at the beginning of the courses, the learners tend to develop
unrealistic expectations. A protocol should be developed to enable instructors
have sufficient time for answering questions. Seeing their projects published on
the course web site gives learners a feeling of accomplishment.
LA eLearning Center Mission Statement
site was created with two main assumptions in mind:
Everything that can be digitized will eventually be digitized.
Computers will never replace teachers; however, teachers who do use
computers will replace those who don't.
want to teach better, but not to overwork;
believe that feeling good about their work does not necessarily go
together with feeling dog-tired;
are looking for a nice and easy way to learn some new skills and methods
that will make their work more challenging and enjoyable;
realize that studying from home makes more sense than commuting to the
can organize their time and study without being prodded;
are not computer-phobic.
day thousands of people buy a new computer. More and more children grow in homes
where computer is as common and habitual as a refrigerator or a TV set. For
these kids, using a PC is as natural as writing or riding a bike. Tomorrow they
will come to school; some of them are already school kids and will become
students in a year or two.
and especially universities are ready for this quiet revolution in education.
More and more equipment finds its way to classrooms; more and more courses are
offered in distance mode. Students who use computers with ease will certainly
prefer to study without having to travel every day. Universities will benefit
from it, too: they will accept more students without having to build more
classrooms, light them, clean them, air-condition them, etc.
of the teaching/learning interaction will occur online: materials will be
uploaded to websites and accessed from home; questions will be asked and
answered via e-mail and chat; papers will be FTP-ed to instructors' folders for
grading; message boards will replace university corridors.
e-Learning will soon be competing with conventional
face-to-face instruction in many fields and countries; in some areas it will
even squeeze the latter out to the periphery of the action field.
Today's pupils who will become e-learners tomorrow are
quickly grasping the meaning of this and acquiring the skills needed for
efficient integration into this new and exciting mode of knowledge acquisition.
The new generation of learners will speak Computerese at a native-speaker level.
Teachers, on the other hand, seem to be lagging more
and more behind. Imagine a teacher of writing who still uses his inkpot while
all his pupils have already switched to ball-point pens. But a computer is just
a smart pen, after all!
teachers who have noticed the advent of the new millennium.
© 2001 LA e-Learning Center: email@example.com,
13/7 Megiddo St., Ashkelon, Israel 78718 +972 (08) 678 2795 (Phone/Fax) +972
(050) 714705 (Mobile)
Lev Abramov, M.Sc., and Natalie Martkovich, M.A., come from professional EFL
background and were among early adopters of CALL. They have shared their
experience in the field by teaching in-service courses in several Israeli
universities and colleges; today they co-own and manage their own company for
CALL-related online in-service courses for EFL teachers. Their teaching
initiative and research are a grassroots venture derived from their practical
involvement in the field. Lev is currently combining his work with postgraduate
studies at the University of Southern Queensland.
Abramov may be reached at firstname.lastname@example.org.