> 1. Assessment
> Course Notes
Search this site
4. A collaborative consultant with skills in:
"For we dare not make ourselves of the number, or compare ourselves with some that commend themselves: but measuring themselves by themselves, and comparing themselves by themselves, are not wise." 2Cor 10:12 KJV
"How would we know who the smart people are?" Response made by the boy who was scoring highest on tests when an instrument was trialed that assisted with memorizing facts for regurgitation.
When we measure, what should we measure against? A standard relative to others, or an absolute standard.
"Be ye therefore perfect, even as your Father which is in heaven is perfect." Matt 5:48
Many of the values driving assessment are for the purposes of self-gratification, for making someone feel valued for what they do, rather than who they are.
Read the "Poisoned Apple". Using bell curve assessment is wrong.
It is wrong to think that the "norm" is normal.
Standardized tests may be true for a population, but not for an individual.
What is elitism? Who are the elite? The privileged class, not those with more wisdom, knowledge, intelligence.
Read Gerald Bracey. He debunks teaching/learning assessments published in the popular press, based on statistics.
e-valuate : fundamental assumption is that value is there.
How am I as a professional in my job evaluated?
In church work, largely by self assessment, knowing that I am achieving the goals I set for myself. Subtle indirect.
Validity as a measure of my performance, on a scale of 0-5: 4
This is my perception, only subjective.
Gathering data to reach a conclusion
Gathering the data - from highly subjective to intensive, detailed study
Making a value judgment (in terms of criteria) on the basis of the data
Drawing out a value
Criteria - established standards, a reflection of the agreed values; agreement is incredibly hard to get because values are hard to define.
Setting the standards and the criteria controls the evaluation.
The leader has an important role in getting consensus on the values, standards, criteria
To set a value
Question critically your own values, and the way you make evaluations and appraisals.
The POINT: the jury, or the judge, makes the final decision
The judge uses the criteria of the law as interpreted by history, precedent, case law.
The jury does not decide on the basis of law, they decide on the basis of presentations made.
This is an evaluation process, by the judge or jury, as the judgment. The presentation is the assessment.
But the assessments are not unbiased, they are slanted. It brings with it its own values. In learning assessment, tests are selected according to the results we know they will give us, the ones we are looking for.
Never make a decision until you have heard the other side of the story.
2 kinds of evaluation:
Formative - Summative
- or -
Process - Product
These are the 4 most important words in the evaluation literature, short of the word evaluation.
Evaluation will include 2 aspects.
The process (formative) and the resulting product (summative).
This applies best in situations when you reach a finish point and the product is completed.
In a continuous process, how do you do this? You take a snapshot at a point in time, and evaluate what you have then as the product.
There is a risk with these words that we use them as labels, and use them in appropriately, and hide behind them.
Assignment for tonight
"Program Evaluation" handout
Many dissertations don't propose an hypothesis, they only ask a question. This is OK.
Evaluation = answering an important question.
You define important, but there must be importance associated with it.
The question has to be one that finding an answer to will bring about changes.
the genders are equal
gender split in management sb the same as in workforce
eliminate causes of imbalance
|Question:||What are the reasons for the gender balance found in management being different from the workforce as a whole?|
|Jim's||Is gender balance an important quality to have in this agency?|
Is it fair to bring a bias as an evaluator? Yes, but it is not hidden, there is no attempt to ask a trick question, the bias is known up front. It is OK to ask loaded questions.
Evaluation is mostly value.
There is no bullet-proof question you can ask to do an evaluation.
A document was produced in the class summarizing 14 of the ideas written up by the different groups who worked on this in the course.
What is an ideal learning environment?
One that is flexible, loving, accepting, adapts to the material being taught and the individuals participating.
Stimulating, affirming, non-threatening.
Asking this question is a waste of time if we don't plan to get there. To get there means change. You need to start with assessment of where we are. Then make incremental changes to move to where we want to be. Further assessment to collect data on whether we are moving toward where we want to be.
AERA Conference on assessment, San Diego, April 1998
Mehrens, Michigan State - ask for copy of his paper on the consequences of assessment.
Conclusion: The consequences of assessment are meaningless. This does not mean assessment is bad, it is just not driving us where we want to go.
Robert Linn, University of Colorado, Boulder. "Assessment and accountability". Gave the same conclusion.
Also Douglas Carnine
"What gets measured gets done." Waterman quoting Haire.
But we measure in education and nothing changes?
Temptation to ask questions for which answers can be found, but which have only limited value. We prefer to do this rather than ask the really important questions that it is hard to find answers to.
There is a lack of a feeling of accountability. It doesn't really make a difference that can be measured quickly or easily. Feedback loop is too long. This contrasts with neurosurgeons and airline pilots.
Complex change occurs only if 5 things all happen. (Ambrose, 1987)
Add Communication between Vision and Skills
If one or more of these missing, how do you go about solving that?
Can be used as a process control mechanism, ie, all pieces must be present for the project to fly.
More Assessment Terms
"I would not have seen it if I had not believed it". You see what you believe. Eisendeich?
What is a formal assessment test?
Medicine EKG, urine analysis
Business Financial statements, ratios ROI
Most formal tests are standardized
Surveys, open-ended questionnaires
Anything in which you are not bound by a particular mode of response
Standardized (multiple intelligences, emotional IQ)
Norm referenced (most educational assessment tests)
Norm referenced tests assume that there is a norm that you can compare against.
These tests work only for groups. They are invalid for an individual. It may have marginal value for a classroom, but more valid for whole school or school district, to compare this year with last year. And then, only valid if the assumption about the norm being normal is valid.
So what are the legitimate uses of the tests? When used for accountability, but not for individuals.
School reports showing position on a graph is inappropriate, "child abuse".
Most tests have an appropriate use. Avoid the inappropriate use of tests.
Standardized assessment will never help achieve learning objectives. It is only for making comparisons with the norm.
We need "appropriate learning".
What does the student know?
What is known?
What can the student do?
What can be done?
How does the student think?
What are the thought processes?
Rate of acquisition: how much new stuff can a student learn in a given period
How do you know you are at the optimum rate of acquisition? By the amount of retention revealed in tests. The tests have to be tailored according to starting point.
New things can be learned at around 7 (up to 7 ± 2) new things maximum, to allow it to be integrated by the brain. Also applies to learning new physical skills. Spaced practice. Learn basketball, then break for 6 hours, can perform better than if had continued to practice all the way.
Teach to the specific knowledge base and learning styles of the students.
How variation can be allowed, and what does it have to do with assessment.
What is the purpose of education? It used to be to prepare people for adult life.
Spache found that limit of tolerance in schools is ± 6 months, ie students referred to special help if a student is more than 6 mths behind say. But take the conservative view that "normal" IQ is 80-120.
For a 6-year old
IQ 80 = 4.8 year equivalent
IQ 120 = 7.2 year
This is ± 14.45 mths, more than twice the range of tolerance.
We are teaching to the middle of normal, and eliminating the extremes, creating behavior problems.
We concentrate too much on the process and too little on the outcome?
Curriculum assessment, curriculum-based evaluation.
Measures for Learning
Note: These are assessments (collections of data) only, there is no value judgment here
1. time on task - are they concentrating?
2. task completion - do they finish the work?
3. task comprehension - do they understand it? This is the ultimate goal
Assessment = to sit beside
An instructional strategy which presents information in small increments and allows for adequate rehearsal (repetition) to ensure automaticity.
This is how you learn to play a video game, or use a word processor
See 2nd article in current issue of JRCE: Would Jesus give an F? by James A
Jesus does not give an F, but we may choose an F.
Find 3+ questions that we could ask in our own workplace, and develop the methodologies we would use to research these questions.
Keep a journal of the time spent developing the questions and the methodology. The time should add up to about 35 hours.
For another 3 credits, actually do this. Ask questions that I could study and answer.
I wrote the assignment on three questions from my workplace, but did not do the work for other three credits.
Created: Tuesday, January 04, 2000 05:15 PM
Last Modified: Saturday, May 10, 2003 9:30 PM