In the age of Accountability: Performance and Results

Evaluating Performance

There are a number of factors involved in evaluating performance and results in the educational field. The perspective of a district superintendent is different than a principal’s view which in turn is totally different than the view of a teacher with regards to evaluation. The important factor here is the systems of performance and evaluation work together to get clear and better results, easier said than done. But that is the objective or goal; and as long as many as possible are aiming and working towards that goal then the closer everyone is to hit the mark.

The book presents different evaluation designs that appeared to be quite intimidating for a classroom teacher. I don’t mean it in a negative way. I think that these kind of detailed evaluations are not used by most teachers. But I’m sure my principal and those in charge of macro managing use these program evaluations. They need to know which programs are working or not, they need to know if they need to move a principal or a teacher to another position. And it’s also important for classroom teachers and professors to know and use them in their own class context. 

I searched the internet and found out that the Kirkpatrick evaluation model was the most widely “talked” about, especially in the education field. And I can see why. I think the four levels are quite comprehensive.

I found two other models that I want to write about and describe how I could use them in my classroom.

Objectives Based Model

This model, also called by some the Goals Orientated Model, is credited to R.W. Tyler. It begins with the objective statement. The objectives may be directed by the client, formulated by the evaluator or specified by the service providers. The goal of this model is to determine whether the program’s goals have been achieved. Results inform how to handle new instructional theory (revise, adopt, reject).
In designing and evaluating a program one should a) set goals, b) derive specific behavioral objectives from the goal, c) establish measures and instruction to the objectives, d) reconcile the instruction to the objectives (strategies), and finally e) evaluate the program against attainment of these objectives.

 The flow chart below shows the Objectives Based sequence. 



The table below shows a model using an objective from the math curriculum in third grade. Most times the goal would have more than one objective and consequently more than one measure, strategy and evaluation.

 

Goal	The student uses place value to communicate about increasingly large whole numbers in verbal and written form. (3.1)
Objectives	A) The student will read, write (in symbols and words), and describe the value of whole numbers through 999, 999
Measures	TSW : read a set of  four, five and six digit numbers- write a set of four, five, six digit numbers
Strategy	TTW use promethean flip chart and introduce thousand family. Students practice reading larger than three digit numbers using number cards.
Evaluation	Depending on the class and individual outcome

The way I would use this model would be according to the weekly objectives put forth in our district online curriculum. The plan would be to have a spread sheet similar to the table above with the objectives, strategies and evaluation columns for the weekly instruction. Then at the end of the unit conduct a summative evaluation and see what percentage of the class met the criterion.  

Qualitative Evaluation Model

The second evaluation is credited to Michael Patton. Patton’s model emphasizes the qualitative methods, such as observations, case studies, interviews, and document analysis. The importance is placed on the quality of the methods used in the evaluation. Associated with this model is the Dick and Carey model of instruction.

The components for the Dick and Carey model consist of nine procedural steps. The sequential steps for the design are as follows: (a) identify instructional goals, (needs assessment) (b) conduct instructional analysis and  identify entry behaviors (learners and contexts), (c) write performance objectives, (d)  develop criterion referenced tests, (e) develop instructional strategies, (f) develop and select instructional material, (g) design and conduct formative evaluations, (h) revise instruction based on the formative evaluations. Some have added another step that is the summative evaluation.

What I can observe from this model is the emphasis on good quality strategies, material and presentations. I could use this template to make sure I cover the grounds as I plan my lessons. Our district has an online curriculum support page where have a weekly lesson plan. This lesson plan is quite general and not meant to take the place of the individual teacher lesson plan. So we use this lesson plan as a guide and produce our own. I think this instructional design would be quite useful.

Questions Regarding Instructional Design

In considering instructional design evaluations, I think that like in any other industry or business one should pay extra attention at the outcome. And questions regarding the performance or process should initiate from the outcome. What is the goal of the program?

From an educational perspective one should look at the student outcome with regards to learning satisfaction and amounts of learning. This could be done in small measurements (year to year) or large cumulative measurements (at the end of high school). One would delve into the world of psychology when considering certain groups of learners. Some age groups have a distinct approach to learning. Learner satisfaction is viewed differently by a young learner in comparison to a high school or a graduate student. Learner satisfaction in some cases is relative to the context. So evaluating programs according to satisfaction is not a one size fits all.

When evaluating a program according to the amounts of learning, one needs to consider the difficulty of getting true measurements. In the last post we had to immerse ourselves into the postitivistic, relativistic and contextualistic views in education. In measuring the learner outcome we have a lot of relativism and contextualism that affects the outcome. I like to use the analogy of taking measurements in the concrete world. Consider measuring water. All the interested stakeholders need to agree when, where, how, and which standard of measurement to use. After all that agreement, it could be considered a true and objective measurement. But humans are not like water. And on top our standards of measurements change. In the old days leaner satisfaction was not an issue, whereas today it plays an important role. This too has to do with context. A boy climbing a tree in some other part of the world could be having as much fun as a boy playing a video game. The question is how to mesh learner satisfaction with great amounts of learning in the present.

The cost vs. the benefits does need to be considered in instructional programs. In chapter 14 the book talks about human performance and how programs try to improve by eliminating a service or decreasing costs. In today’s economics, companies that avoid the need for handling have greater economic outcomes. But with regards to instructional programs sometimes trying to cut costs could produce other difficulties. For example, if schools decide to cut the counselors position you might have a number of social difficulties in the classrooms and eventually affecting student performance. But at the same time, funding and educating young learners (pre-school) might have a more positive impact on the adult population. So, cost vs. benefits need to be carefully considered and studied.  

Regarding Performance

One of the hats a teacher wears is the performance consultant (PC) hat. Large companies have specific PC positions for their performance needs. As educators we need to be well trained and equipped with resources to solve the performance difficulties. Although teachers are incapable of solving all of the students needs in the class, teachers ought to be able to notice special needs or atypical needs in the learning process of his or her students. Even though experience is a great teacher, educators still need to be trained to be better equipped to detect performance deficiencies.

As I was reading about informal reading, I couldn’t help but associate it to the early Greek philosophers and their followers. In teaching, I try to make the learning experience as authentic as possible in the context provided. I think here is a great problem in the shifting educational sphere. Authentic learning happens in the classroom when the student can assimilate and claim understanding in some form or fashion.

Informal Learning Through Problem and Solution.

As I read the book, I cannot help but think about the context in which I work. I know that a lot of the content of the book relates to the commercial or business realm, but I constantly visualize and try to connect content from the book to my classroom, especially the class and situations from this past year since they are fresh in my mind.  

I think that if we look at someone in their everyday life we can see a plethora of daily problems that need to be solved; from deciding what to wear to how to answer to a crying child. Usually we don’t see them as problems but as routines.

Some performance problems that occur at my school are getting from our classroom to the gym or cafeteria or using the restroom time appropriately. This might not be an academic performance problem but it is an issue that needs to be addressed at the beginning of the year. I try to teach the practicality of the rules and routines by using authentic examples. If I want to teach the class the reason why and how we should walk in the hallways I use the following lesson. I would have half of the class go to the end of the hallway and instruct them to come walking while the other half would do the same coming down the hall in the opposite direction. Then as the two groups meet in the middle, some in line, some not, I have them talk about what they observed. Eventually students have a better understanding of procedures; instead of just “rules to be followed because I told you so.”

Performance Support

The context in a business is quite different with regards to the outcome. At the end of the day the business wants to have a profit and not necessarily be engaged in learning. So a business will have lots of support systems that work towards the desired end. In education depending on the objective, the support systems could be used to make things more practical but sometimes the process is the objective. The student having to figure out how to solve a problem would be his own support system. In education, educators want their students to be creative thinkers and authentic problems solvers. If I think a support system is another student who understands better a certain assignment, the more knowledgeable student would not be helping if she does the work for the other student. But if a teacher or student helps the weaker student understand the task, or provide tools to solve the task, then it is a support system. So the role of support systems has to be looked at differently according to the goal or objective.

Another area to consider with regards to the educational context is oversaturation. I have noticed that some students get pulled out from the classroom for various in school programs. These children get lots of support through the different. Some of them are well designed to measure progress or outcome. This allows teachers to have a clear understanding (explicit knowledge) of the students performance. But from my own experience, I have seen that too much “pull outs” could be detrimental. This is not a documented conclusion but instead from a personal insight. I guess this would fall into the tacit knowledge category regarding some of my students. Some students need more time than others to process information, to make connections and therefore fixate learning. So as educators, we need to carefully consider the frequency of performance support.