Evaluating Leadership/Professional Development Programs

Background

At the forefront of various sweeping reforms in education that have emerged nationally in the last decade or so, professional development—in leadership, instruction, and technology, to name a few—has become an essential element of nearly every reform package adopted by any school or district around the country. The desirable goal of having highly trained and intelligent teachers led by wise and clever principals presiding over inquisitive and eager student bodies is so alluring in that, oftentimes, when a problem is identified, educators have a knee-jerk instinct to demand more staff development, more training. The idea that educators need only “study harder” and learn more is incredibly seductive.

Unfortunately, staff-development, professional training, and leadership instruction programs and initiatives often prove ineffective because they are created out of thin air, in the abstract. In particular, when professional development programs are ad-hoc or one time only, i.e. one seminar, they have great difficulty sustaining results in the long run. This is because knowledge itself does not necessarily generate skill and most definitely does not equal practice. Moreover, so many day to day challenges arise in education that a few scattered classes and seminars are not capable of equipping educators with the skills necessary to be effective on a routine, long-term basis. Nonetheless, many educators, leaders, and schools continue to implement program after program and hold seminar after seminar despite the fact that real results are not being achieved.

For professional development programs to work, there has to be constant, ongoing, and relentless evaluation, research, and inquiry. Does this work? Why? Why not? When does it work? All of these questions and more must be applied to any professional development program if it is to stay relevant, focused, and achieve results.

Benefits

While it may seem difficult at first to maintain such a high level of vigilance, the rewards of committing to the evaluation process are endless. First and foremost, thoughtful evaluation of programs and initiatives will achieve results for students, teachers, and principals. Having the right answer to a student’s problem or a difficulty the school faces is of course immeasurably valuable. Moreover, success tends to have a snowballing effect—the better the students perform, the better the school becomes which in turn affects educator job performance, contentment etc.

Wasted time, funds, and emotional effort can also be avoided by maintaining a critical eye. Why make teachers waste personal or class time attending seminars or conferences from which they will ultimately take nothing back to the classroom?

Finally, by committing to ongoing evaluation, educators can create a professional climate of enthusiasm, passion, and interest. Many principals and teachers, after committing to evaluating themselves and their professional development programs become intensely engrossed in the search for what does and does not work in the classroom. Such enthusiasm can only be attained by emotionally and intellectually involving educators in a cooperative, collegiate on-going quest for self-improvement.

Examples

SCOUT (Students Constructing their Own Understandings with Technology) is an intensive technology training education “camp” for educators seeking to find new ways to use technology in the classroom to enhance student learning. Technical experts, or “counselors” guide the teachers through the camp, providing assistance and instructing them on potential ways to help their students improve through technology. Importantly, SCOUT offers ongoing assistance after the seminar is over to educators who participated. Also, SCOUT evaluates every seminar by the following four levels of achievement:

Level 1: Determining participants’ reactions to the experience.
Level 2: Measuring the knowledge and skill which participants acquire as a result of professional development.
Level 3: Measuring participants’ actual use of knowledge and skills they have gained.
Level 4: Measuring the impact of participant changes in knowledge and skills on student learning.

SCOUT gathers data on each of these four topics after every seminar and is currently attempting to develop a “dense network of peer relationships” within the educational community. By doing both these things, SCOUT is ensuring that, at the least, it will stay current and informative in what it has to offers to educators and, at best, possibly even make a change for the better in the lives of the students it ultimately serves.

Related Links

Click a category to toggle display of related links.

What Is It?

Evaluating Science and Mathematics Professional Development Programs

This is a good report that explains why evaluating professional development programs is important, how to go about doing it, what it looks like, and who should do it. It is very summative. (From Weiss, Iris R. Horizon Research Inc.)

NCREL on Evaluating Professional Growth and Development

This web page asserts an important relationship between school reform and professional development. The page has links to NCREL information on evaluation--from evaluation teams, to evaluation purpose and design. It includes audio files and useful contact information.

Policies That Support Professional Development in an Era of Reform

This is an incredibly valuable article. It deals with what is and what is not a successful professional development program, paying particular attention to difficulties associated with evaluating development initiatives. Weak surveys, lack of oversight, the introduction of too many changes, and a lack of coherence are all cited as obstacles to effective professional development programs. (From Darling-Hammond, Linda. Other Middle Grades Resources)

Skill Shop: Got a Minute? A Stairwell Talk Can Turn Evaluation Into Everyday Business

This succinct article describes what evaluation looks like and why it is needed in professional development programs. Some simple, everyday suggestion and strategies are provided for educators looking to keep tabs on their own learning. (From Champion, Robby. Journal of Staff Development, Summer, 2000)

We're All Here to Learn

Joan Richardson's article delineates the characteristics of several schools and districts that have been rewarded for their use of leadership/staff development programs. An important of success is constant evaluation of staff development initiatives. Teachers undergoing staff development in model programs often have "coaches" who continually monitor their progress and suggest improvements and corrections. Weekly meetings, discussions, and exchanges with other teachers allow every teacher to continually evaluate what he or she is doing. (From Richardson, Joan. Journal of Staff Development, 1998)

WestEd offerings on evaluation

This Web page offers a listing, with links, of WestEd programs, practices, and resources on evaluation. See especially information on evaluation of “Professional and Organizational Learning.”

What Works in the Elementary School: Results Based Staff Development

The National Staff Development Council provides three online books that describe what staff development strategies work at the elementary, middle, and high school levels. Each book is easily downloaded in pdf format. A number of chapters are devoted to the evaluation of staff development programs all of which advocate a need to understand the direct correlation between programs and results. Professional development programs alone are not enough if they do not achieve anything. (From Killion, Joellen. National Staff Development Council, 2002)
Tools & Resources

Achieving Your Vision of Professional Development: How to Assess Your Needs and Get What You Want

This 177-page guide is designed to 1. develop a clear picture of effective professional development 2. demonstrate how to develop school-based professional development activities 3. explain how to evaluate the quality, content, and results of a professional development experience. (From SERVE 2000)

Assessing a School Staff as a Community of Professional Learners

SEDL describes here “the instrument” for measuring the level at which a school functions as a “community of professional learners.” Noting that public education has become swamped over the years with new ideas and promises about reform, SEDL sought to scientifically quantify the level at which some of the schools in their region were actually functioning when it cam to engaging is staff development and getting results. This example of a scientific approach to assessing and evaluating development programs can serve as an excellent resource to remind principals and educators of the need to not only reform schools but to know how and why they are doing so. (From Southwest Educational Laboratory. Issues…About Change, vol. 7, no. 1)

Measuring Impact of Educational Programs

Though not directed specifically at professional development programs, this report and its recommendations are nonetheless excellent for any principal looking to evaluate his or her professional development program. This article takes you step by step through the evaluation process and is an excellent starting point to understanding how to evaluate programs and assess their results. (From Diem, Keith G. Rutgers Cooperative Extension: Fact Sheet)

Minnesota Evaluation Studies Institute

The annual Minnesota Evaluation Studies Institute (MESI) is designed for professional evaluators, program directors in non-profit and for-profit organizations, and others interested in conducting or using evaluations. Sponsored by the Department of Educational Policy and Administration in the College of Education and Human Development at the University of Minnesota, MESI provides a unique opportunity in a small setting to learn and reflect on contemporary issues in evaluation with national experts and practitioners. This conference provides valuable insights into professional development practices. (From College of Education and Human Development, University of Minnesota)

NYU Professional Development Laboratory

Among the Professional Development Laboratory's (PDL) goals are: to "Develop leadership capacity in teachers and principals for the purpose of improving student learning" and to "Evaluate the impact of professional development on student learning." PDL offers programs for teachers and principals.

Professional Development Planning & Evaluation Workshop

McREL offers a workshop intended to work with school and/or district staff to design comprehensive professional development plans to achieve school goals and to evaluate professional development programs.

Professional Development: Learning from the Best

This is a toolkit for schools and districts based on the National Awards Program for Model Professional Development. You can download the Adobe Acrobat Reader version of the toolkit as one big file or you can download the individual sections of the toolkit. The toolkit has four sections/steps: Step 1: Designing Professional Development; Step 2: Implementing Professional Development; Step 3: Evaluating and Improving Professional Development; and Step 4: Sharing Professional Development Learning. A variety of "Action Planner Tools" are also available. (From Hassel, Emily. North Central Regional Educational Laboratory)

School Leadership Development Program Self-Assessment

What's the best way to assess a school-leadership-development program? With that question in mind, e-Lead (in close cooperation with our advisory board) developed this assessment tool. (From e-Lead)

Tapped In

The web page states TAPPED IN™ is the online workplace of an international community of education professionals. K-12 teachers and librarians, professional development staff, teacher education faculty and students, and researchers engage in professional development programs and informal collaborative activities with colleagues.” In other words, this website is a place where principals and all educators can come to discuss what works and what doesn’t when it comes to professional development programs.

The 2002 User-Friendly Handbook for Project Evaluation

This guide is "aimed at people who need to learn more about both what evaluation can do and how to do an evaluation..." It covers both qualitative and quantitative evaluation methods and how they relate to an evaluation strategy. (From National Science Foundation)

The Inquiry Learning Forum

This web site allows educators to develop their own personal professional development plan and share information with one another. Lesson plans and examples on video are available. Lessons from the Forum can help principals evaluate their own development programs.

W.K. Kellogg Foundation Evaluation Handbook

Though not targeted toward education, this guide is useful in considering evaluation of any kind. It is designed to provide a framework for program staff for moving "from a shared vision for effective evaluation to a blueprint for designing and conducting evaluation, to actual practice."
Model Programs

Educators Go To SCOUT Camp for Technology-Enhanced Learning

SCOUT camp, a program designed to enhance student-teacher learning about technology use, is a recognized success story because “During SCOUT Camp’s first three years, its focus has been on gathering evaluation feedback that could be used to improve and refine the professional development model.” This article breaks the research and data down and shows how, why, and where the program has been successful. (From DeWert, Marjorie Helsel and Cory, Sheila Levine. Journal of Staff Development, Winter 1998)

Islands of Hope in a Sea of Dreams

This link leads to a research report written that analyzes how staff development succeeded in a variety of schools and districts and why. The report focuses on eight schools that received the National Award for Model Professional Development. (From Joellen Killion (NSDC) for the U.S. Department of Education and WestEd)

Principals as Head-Learners in Schools

This interview shows how real-life principals credit success in their schools to ongoing evaluation and adaptation of development programs. One principal, for example, states that he constantly evaluates every teacher in the school and in turn eagerly listens when he is evaluated by analysts who come to his school. (From Richardson, Joan. Results, February 1999)

Professional Development, Classroom

This article places emphasis on the need for assessment and evaluation when it comes to staff development. An example is given wherein development program results are identified and understood through the use of teacher/student surveys as well as cognitive tests. This article is an excellent example of the mind frame a principal might need when evaluating his or her own development programs. (From McKenzie, Barbra and Turbill, John. University of Wollongong, February 1999)

Teachers Get Help from the Guide on the Side

This link leads to an article describing the successes of Long Beach’s coaching program. Though “no one thinks it's the solution to teacher development in the Long Beach schools…the impact of the program's first year is palpable, and the district seems likely to stick with the coaching approach for the foreseeable future.” Examples of coaching “journals” are provided and the role of principals in coaching is discussed as well. (From Norton, John. Changing Schools in Long Beach, Fall 1999)

U.S. Department of Education Model Professional Development Awards

Through its National Awards Program for Model Professional Development, the U.S. Department of Education recognizes schools and school districts with exemplary professional development programs. Winning schools and districts have professional development programs that: 1) Produce professional growth as an integral part of school culture; 2) Address the needs of all students, and 3) Promote professional development practices that ensure equity by being free of bias and accessible to all educators. Successful applicants are able to demonstrate how well their professional development programs result in increased student outcomes. NSDC has published several articles about the winners, including profiles of each winning school or district for the past two years. (From National Staff Development Council)
Selected Research & Articles

Designing Powerful Professional Development for Teachers and Principals

NSDC executive director for 16 years, Sparks’ book is invaluable, in depth reading for principals and teachers alike. Among other things, the importance of and recommendations for successful evaluation and assessment of development programs is discussed. (From Sparks, Dennis. National Staff Development Council)

Evaluating the Success of Professional Development

Though the focus of this article is on using staff development to better utilize technology, it nonetheless clearly defines the need for evaluation, describes some kinds of evaluation, provides some small examples, and includes a few resources for evaluating development programs. It is a succinct but useful link. (From Technology Team Leadership Institute. Pathways to School Improvement: To the Point, NCREL)

Principals Evaluating Peers

This is an excellent article that deals with some of the common difficulties any principal might face when attempting to evaluate his or her teachers. Though the article does not focus specifically on professional development, the problems and issues with evaluation discussed by the author are absolutely applicable. (From Gil, Libia S. The School Administrator Web Edition, October 1998)

School Leadership Study

Principals play a vital role in setting the direction for successful schools, but existing knowledge on the best ways to prepare and develop highly qualified principals is sparse. To fill in the gaps, a major research effort is underway to determine what is known about principal preparation and development, what works, and how to create scalable models that can bring effective leadership to all America's schools. The research project, "School Leadership Study: Developing Successful Principals," was commissioned by The Wallace Foundation and undertaken by Stanford University in conjunction with The Finance Project. This research began in Fall of 2004 and will continue over the next three years. (From Stanford Educational Leadership Institute)

Scooping Up Meaningful Evidence

This brief article does a great job of describing the new demands placed on program evaluation by the No Child Left Behind Act. In it, Guskey also describes a backward planning process, combined with collecting evidence each step of the way, which helps staff developers meet goals of No Child Left Behind. (From Thomas R. Guskey. Journal of Staff Development, Fall 2003 (Vol. 24, No. 4).)