Dr. Thomas Guskey: Gathering Evidence on the Effects of Professional Development

June 17, 2008

I am live blogging this session of Thomas Guskey.

Evaluating professional development:  there are many ideas that we look at when we think through the evaluation process.  But it is something that we naturally do.  

1.  Participant’s reaction to the experience.Executive Director of the AZ K 12 Center

  • Is the temperature right, are the materials appropriate, is the food good, are the conditions for learning appropriate…….

2.  Participant’s learning from the experience.

  • Have the participants learned something and are walking away from the experience with more than they came?

3.  Organization Support & Change

  • Is the organization ready to make this shift?
  • Building principal is the Critical Element.
  • The principal needs to actively participate or the message is that the experience is not important
  • When the practice at the school is in conflict to the new learning that has just taken place…. it sends a poor message, and is detrimental to the relationships and structure of the whole PD system.
4.  Participants’ use of New Knowledge and Skill
  • Did you influence practic?  Are they using new knowledge and skill?
5.  Results:  Students Learning Outcomes
  • Impact on the students?
outcomes for today:
1. Understand these five levels.
2. Each level builds on one another.  If one breaks down then the next one will not happen.
3.  When you plan professional development, the order is in REVERSE.
  • First issue that comes up is what you want to accomplish and the evidence of that accomplishment.  
  • Then thinking what new knowledge of skills needed to accomplish these skills. What are the practices we need to gain to accomplish this work?  
  • Is the organization ready to think about this endeavor?
  • What do we want the students to learn?
  • What are the conditions I need to put in place to make it happen?
Two Premises that will guide our work:
1.  Ideas are more important that vocabulary
  • We seem to be changing the vocabulary but the ideas are the same–
  • inservice>staff development>professional development>professional growth> professional renew>human resource development>professional learning
If you want to concentrate on the quality of what you do, don’t worry about what you call it…
2.  We are in desperate need of good information on Effective Professional Development.
  • “What Works” Clearinghouse Evidence Standards-(check it out)
  • He showed data from this study
  • Any change in student improvement resulted in a 40 hour instruction series
  • Content–was all focused on curriculum and pedagogy 
  • Then teachers could adapt at their discretion based on their students
But these studies are limited–we really need more information on effective professional development.
We don’t have best practices we have better practices.
Part II of the morning:
What makes professional development effective?
We need to think differently about planning professional development.
1.  Program topic?
  • The topics don’t matter–it is important, but some times the very important content can be presented to a selection:  needs survey–old way–these are problems, concerns, dilemmas and wants.
  • Selection needs to be aware of the symptoms of needs.  teachers are in demanding roles, so we need to interpret the symptoms of their needs, and then translate that to proper PD.  
  • Shrink the ratio of the involvement in learning between the time they are there, and the time they spend on learning.
  • If you manage learning well, you don’t have to manage the learners. (Bloom)
  • See the symptoms, and interpret the needs.
2.  Program Planning?
  • There must be more to your planning and PD than raising the test scores.
  • Should be planned collaboratively between the district and site level.
  • Across buildings.
  • Contextually relevant
  • Find ways of collaborating and engage people with you on the topics.
3.  Program Participants?
  • Mandatory or voluntary?  research says it depends.  On what youwant to accomplish
  • If your focus is to share info-mandatory large scale session serves well.  1.  gives all a shared knowledge base; 2.  fives you shared vocabulary; 3.  dispels rumors.
  • But when you move to implementation it must be voluntary.  Then who is going to volunteer?  the ones who are the most talented and effective.  The resistant ones will be the ones who need change the most.  So you go with the ones who will choose it first, then be visible with the results, so the slow-to-engage will become interested.  Focus on the results.  (not the people)  It is what they do.
  • When the results become well known the interest will spread.
  • Plan 3-4 years out–sustained efforts
4. Program leadership?
  • It is not who shares the info it is what they do.  Inside people or outside people–same
  • Person presenting has to appear credible.  (inside or out)
  • Ideas have to be seen as valuable and useful; relevant to their world.  Have to be able to relate to the setting
  • If you bring in anybody who talks down and is demeaning to them this will be bad.  Get references and talk to others.
  • It is not who but how they present the material to them.
Two things parents want to know at open house
competence (not years of teaching)
if the person cares about their student
in parent/teacher conference–if the first thing you communicate is grades, it shows you don’t know the kids.  
5.  Group size?
  • Size depends on the topic and objective
6.  Program timing?
  • According to research, best time to do pd is summer
  • Removes teachers from ongoing responsibilities of the classroom
  • Sub plans are a pain
  • Worse time-after school
7.  Length of training?
  • Larger magnitude of training, the longer it will be
  • Structural change offers opportunity if you change positively what you do with the structure
  • If you invest in the learning about the format, then results were seen
8.  Types of Activities?
  • Are adults different from kids?  no.  
  • What makes for excellent pd is the same for kids
  • Learning stiles, intelligences, engagement, interests, sharing, motivating
  • Good PD looks alot like good teaching.
9.  Extent & Complexity of change
  • The more extensive the change you are asking, the more extensive the PD must be.
  • Teaching is an intervention
10.  Follow-up & improvements?
  • The most vital dimension of effective PD
  • The most neglected
  • Principal can have a vital role
  • Structured opportunities will be imperative to the success of any professional learning experience
So what WILL make PD Effective?
1. Always begin with a clear focus on learners and learning.
  • If you plan well, evaluation takes care of itself.
  • If I start by saying my outcomes, I have a basic framework of my evaluation.
There must be clear goals along with assessment procedures to document progress.
  • Ralph Tyler-basic principals of learning
  • Decide what you want them to learn, decide what evidence there will be to tell you verify that they learned.
  • The most valuable criteria of education is student learning
  • Teaching and learning is absolutely linked.
  • 3 step planning:  1.  How does the activity relate to the school mission; 2.  What are the intended student learning outcomes?; 3.  What evidence best reflects those outcomes?
Clear goals help moilize everyone and keep efforts on task.
goals prevent distraction by peripheral issues that wast crucial time and divert energy.
2.  Engage in rigorous self analysis.
  • Requires the courage to ask tough questions
  • The skill to find honest answers
  • Whether it works is defined by what learning takes place, not what we do…or our experience in teaching it.
  • Look at what went well, and what didn’t go well and then change your practice or method.
  • Need structured opportunities for collaboration–you get quality support and guidance on how to improve
Researching your goals requires a clear sense of where you are.
  • And where you are is determined by where your students are as determined weekly
  • All the time….
  • Ask who is not learning, why, and what can we do about it?
3.  Recognize change is an individual and organizational process.
Remember the stages of concern
  • 1.  personal
  • 2. management
  • 3. impact
motivation–kids are motivated by success. research shows.  
  • Think of the video games….what is the purpose of pacman?  gobble up dots…..what is interesting about this or relevant?  you get to improve your score
how do you get the personal concerns–needs survey:  Here Dr. Guskey was referring to the fact that all a needs survey is is a way to identify personal concerns.  But we can tell lots from those personal concerns.  And we need to consider those when we are considering planning for PD.  Sort of the Question behind the Question .  What is it that REALLY will help the issue.  What is the REAL need?
Consider the order of change
  • Teacher attitudes and beliefs
  • Teaching practices
  • student learning
which comes first?  practices first, student learning then attitudes. (Huge aha for me)
Experiences shape the attitudes and beliefs.
Don’t try to change the attitudes and beliefs.  Try to change the practices and then the student learning will come.  Then possibly their attitudes will come.  As student learning happens:  small change, small change in attitudes.  large change in student learning, large change in attitude.

Most vital element in the professional learning is the follow up.  We need to look for evidence and come back to it during the year.  
How soon can you see results?  2 weeks.  If you don’t see results in this time, teachers will give up on the change.  They will not want to waste time.   
This same order of change holds for students as well.  If students’ experiences change, their attitudes will follow.  (Remember the Titans movie:  Denzil Washington’s character insisted on a certain type and level of behavior.  The team saw results and started to win.   The attitudes and respect for one another came later).  Change the experience, the attitudes and beliefs will follow.  From the district level, this is another huge aha for me.  I believe that as I move into next year, and shape and plan my professional development activities and experiences for teachers, I will REALLY focus on quality and modeling of this type of pedagogy.   The better quality of instruction and experiences, the more likely I will be able to move teachers in the direction of technology/curriculum-infused classrooms.  
4.  Think BIG, but start SMALL
  • Do a measured process.  not in the first year is everything going to be accomplished.  
  • Start small in one subject.  build up.  
  • Annual pd plans do not work.  you must have a vision.  3-5 years out.  year one-trial.  year 2, practice and year 3 implementation……
  • Use a systematic set of steps.  support must be systematic
  • Change is dynamic and large scale but always will be implemented through a series of smaller steps.
  • Have a grand idea but with small steps.
  • We must balance the urgency for improvement with the need to validate effectiveness. FORMATIVE.
5.  Ensure the procedures are ongoing and professionally embedded.  
  • Change and embed with the procedures.
  • Look at what is the current structure and then what will fit within that structure
  • Allow kids to get better and do better.  Allow them to learn from their mistakes and let them do better.  
  • Why do we not let students fix their mistakes?  They must engage in it.
Change is a natural and recurring process that is essential to all learning environments.  Change is essential to the teaching process.  
6.  Work in teams to maintain suppport.
  • Teams need to be mixed up.  not just grade levels or groups.  be sure to include resource teachers and gifted and talented students.  Midwest publications are great.  games magazine.  
  • Apt initials.  take the famous person, and describe them using adjectives with the first letters of the person’s name. egg bearer= easter bunny
  • Take your initials and come up a phrase that describes you.
  • Find about these and more by using collaborative teams.
7. Use available resources
  • Use your curriculum resources and guides that have been developed.  don’t let hem sit on the shelf.  keep them living documents.
  • Teachers typically plan in terms of materials and activities.
  • Show how the activities can blend with the plan
Major models of pd
  1. training
  2. observation  / assessment
  3. involvement in a development / improvement process
  4. studygroups
  5. inquiry/action research
  6. individually guided activities
  7. mentoring
8.  Include procedures for feedback on results
  • Most effetive feedback is the aha moment on the kids.  you must build feedback from students.  immediate, short, directly tied to where you can see iprovements.  
  • As coach or mentor, you must make them see that what they are doing key to success
9. Provide continued followup support and pressure
  • Implementation dip-sometimes the effectiveness suffers and there should be a perfect marriage of support and pressure.
  • Support with pressure.  you can do this, but you must do this. (serve and volley tennis game vs. the base line game.)

At each level we must consider 
a.  What questions are addresed?
B.  How will information be gathered?
C.  What is measured or assessed?
D.  How will the information be used?

1.  Participat’s reactions
  • did they like it?
  • was their time well spent
  • did the material make sense
  • wil it be useful
  • was the leader knowaledgable & helpful
  • were the refreshments fresh and tasty
  • was the room for the right temperature
  • were the chairs comfortable?
B.  How will information be gathered?
  • questionnnaires and sureveys adminisstered at the end of the sessions
C.  What is measured?  
  • Initial satisfaction with the experience
D.  How will the information be used?
  • To improve program planning, design and delivery
2.  Participant’s learning
A.   What questions are addressed?
  • did participants acquire the intended knowledge and skills?
B.  How will information be gathered?  
  • paper and pencil instruments
  • performance assesments/tasks
  • simulations-pretend and role play–adapted to the goals you set up
  • demonstrations
  • participant reflections
  • participant portfolios
C.  What is measured or assessed?
  • New knowledge and skills of participants
D. How will the information be used?
  • to improve the content, format and organization of the program
3.  Organization support and change
A. Questions:
  • was implementation advocated, supported and facilitated
  • was the support public and overt
  • were problems addressed quickly and efficiently
  • were sufficient resources made available
  • were successes recognized and shared–think “stickers”
  • what was the impact on the organization
  • did it affect organizational climate and procedures?
B.  How will information be gathered?
  • district and school records
  • minutes from follow-up meetings
  • questionnaires / surveys—once per month–structured, directed, short
  • interviews with participants or administrators–even little visits-impromptu
  • participant reflections/portfolios
C. What is measured or assessed?
  • The organization’s advocacy, support, accommodation, facilitation & recognition.
D.  How will the information be used?
  • to document and improve organizational support
  • to inform future change effort
  • you must plan backward.
4.  Participants use of New knowledge and skills–rubrics–this is what effective implementations looks like and this is what I’ll see when I know it is implemented.
A.  Questions
  • Did participants effectively apply or implement the new knowledge & skills
B.  How will information be gathered?
Observe only 4-6 pieces of evidence–any more are too many
  • questionnaires/surveys
  • interviews with participants and their supervisors
  • participant reflections
  • direct observations
  • video or audio tapes
C.  What is measured or assessed?
  • Degree and quality of implementation
  • you must look at both of those dimensions.  mutual adaptation…….teachers must adapt any innovation in their context.  
D.  How will the information be used?
  • To document and improve the implementation of program content
5.  RESULTS:  Student Learning Outcomes
A.  What questions are addressed
  • what was the impact on students
  • did it affect student performance or achievements
  • did it influence students’ attitudes dispositions or behaviors?
  • are students more confidant as learners
  • is student attendance improving–first unit taught needs to be the easiest and most achievable unit so that all students achieve.
  • are dropouts decreasing?
B.  How will information be gatherd
  • school records
  • school records
  • questionnaires/surveys
  • interviews with students, parents, teachers, and administrators
  • participant portfolios
C.  What is measured or assessed?
  • Cognitive (performance & achievement
  • Affective (attitudes & Dispositions)
  • psychomotor (skills & behavior)
D.  How will the information be used?
  • to focus and improve all aspects of program design, implementation, and follow-up
  • to demonstrate the ultimate impact of professional development.
There is a difference between evidence and proof.  Managers know how to do things right.  Leaders know the right things to do.
We need to know the right things to do.  We need to take a stand.  Be courageous and take a stand.  Take a stand based on what you believe in—-you know the right things to do.  If it doesn’t make a different for the kids don’t do it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: