Using Cases to Evaluate Educators

Evaluators expend huge amounts of energy observing teachers in typical self-contained classrooms, searching for signs of effective teaching. In doing so they may gain a sense of what teaching influences student learning in these situations.  At the same time they risk losing opportunities to judge teaching and professional behavior under other conditions. Case methods offer a possible remedy to this problem.

“We are inclined to the view that the case method—long used in medicine and law, and more recently in public administration and business—will in the coming decades be relied upon increasingly in the field of education, both in the pre-service and the in-service training of teachers and administrators.”  (R.N.Bush, Stanford, 1954).

Early adopter Bob Bush correctly predicted the power of cases to educate educators. We have stretched his  prediction to use cases—or multimedia web-based slices of educational life—to judge educators’ abilities to make professional decisions in any and all aspects of their work.

How?  We record situations that educators face in their everyday lives and enrich the accounts with data and artifacts.  Teaching Notes pose questions that prompt others—teachers, administrators, and parents—to make explicit what they know and what they think they know that is relevant to the case.  We push case analysts to describe what they might do, why, and  what might happen if they pursued a particular course of action.  Answers to these questions provide the grist for judging people’s abilities to forecast behavior that is more and less defensible.  Critical perspectives—analyses offered by people recognized as knowledgeable in the facts  of the case—serve as benchmarks against which analyses can be judged.  

Like other empirical research, good case analyses are high in external validity.  Smart educators craft solutions that plausibly “fit” the people involved and the ecology—classrooms, schools, and communities—of the case itself.

Bob McNergney is co-founder of and partner in CaseNEX-DataCation, LLC.

DataCation, PupilPath, Skedula and the SLC?

Maybe I am showing my age, but I don’t think so; the SLC has always stood for Small Learning Community and still does.   Many large schools like New Dorp High School, however, use the Small Learning Community model to create smaller learning environments within big schools.  We have witnessed this strategy improve academic performance at many of our bigger schools. 

But now, you will want to be on the lookout for the new version of SLC - short for “Shared Learning Collaborative.”  The new SLC offers unique technologies and services to store data by schools, districts, and states.  This means educators with information about academic learning can connect students, content, and tools to personalize instruction for all students. 

After attending the SLC Camp in NYC I saw hundreds of educators, parents, and developers get excited about what “collaboration” in the schools might come to mean.  Renewed efforts to tag content are facilitating the connection of data points to form instructional maps that can guide individuals toward common ends.

If you do a Google Search on “Slope of a Line,” for example, it will yield 12,900,000 URLs.  The last time I checked, nobody had time to evaluate all that content. The SLC, however, will allow you to tap into many content resources that are immediately relevant to your students’ performances.

Whether you are referring to “Small Learning Community” or now “Shared Learning Collaborative,” the SLC makes sense in today’s schools.

Peter Bencivenga

President 

CaseNEX - DataCation

Have you seen Skedula in NYC?  If not watch it here: http://vimeo.com/39936168

Is Your School Maximizing Its Progress Report Score? Part 2 of 3

The NYC Progress Report shapes how your school is perceived by prospective parents.  Of the three NTC accountability reports, it is the only one that transforms schools ratings into letter grades.  This school is an A, that school is a C.  Validity of the metrics aside, parents understand that an A is better than a C.  

I have become obsessed with dissecting the Progress Report metrics.  My hobby is reviewing the new metric announcements in early spring and predicting how they will affect school scores.  The work begins when I plan reports we can add to DataCation that allow for verification and improvement of scores.

This article explores the DOE defined metrics of excellence and how to maximize their point value.  My intent is to help you improve your progress report score with your current resources.  By using data reports in the DataCation portals or other systems, your staff can satisfy the DOE standardized metrics and still have time to promote local programs that give your school the personality and vigor that reflect excellence.  

Student Performance

The Student Performance Section metrics are all about graduation rates.  Its metrics include the school’s simple graduation rate (the number of students who earned diplomas) as well as the school’s weighted graduation rate (the number of students who earned advanced diplomas).

Graduation –

There is one simple rule - the more graduates, the better.  Be proactive about your graduation rate.  The hard work for graduation happens in the classroom between the teacher and student, but if the student is not scheduled for the required classes, she or he will never graduate.  Use the NCLB/GRAD Graduation Requirements report to ensure proper scheduling of classes needed for graduation.  The report not only displays which students have not met the course requirements, but it also displays if the student is programmed for a class in that subject area.  This report can help guidance staff correct student schedules early in the term.  This report also displays the number of exams the students must pass to meet graduation requirements.  If you know which students need certain exams, you can make informed decisions about teaching and class resources which will help both the students and the school progress report score.

Weighted Graduation (for this metric and the Closing the Achievement Gap metric) -

How many students in your school would earn an Advanced Regents diploma if they pass one more exam?  Have you targeted these students to provide support for the missing exam?  The NCLB/GRAD Graduation Requirements and Graduation Tracker reports can show you who these students are and which exam is outstanding.  Getting these students to pass this last exam will increase their score on this metric by 25%.

The Graduation Tracker Report displays credits earned and exam grades and calculates diploma types.  Flags are included so you can track by ethnicity, by ELL status and IEP status to enable you to track students in these groups for the Closing the Achievement Gap metrics.  This report calculates diploma types including Local, Regents, Regents with Honors, Advanced Regents and Advanced Regents with Honors.  The higher the diploma earned, the more points earned on your progress report.  Where the averaging needed to determine the honors designation is not difficult, it is time consuming.  Your guidance staff can use their time in more productive ways than calculating exam averages.  Let the counselors counsel and the computers compute while the school earns the maximum and correct progress report score. 

Student Progress –

The largest piece of the Progress Report, Student Progress, is worth 60 points.  Progress is defined through credits earned and exams passed.

Keeping students on track to earn 10 or more credits – Half of the points in the section of the progress report are earned in this metric.  This metric measures students in the 1^st, 2^nd and 3^rd year cohorts to earn “10 or more credits” in a school year.  These 10 credits must include 6 credits in the core subject areas of English, Math, History and Science with at least some credit earned in each of these areas.

The NCLB/GRAD Progress Credit Accumulation report will allow you to be proactive in your programming and follow up of these students.  The report has four options to provide full information on this complex metric.  At the end of each marking period, run the Total Credits option.  The report includes the number of credits earned last year, the number of credits earned this year and the number of classes passed and failed for the marking period.  Using a two-term school as an example, you can find which students earned more than 10 credits last year but are NOT passing 5 or more classes this year.  This information will allow you to target these students to get them back on track.  If you turn this data around, you can confirm interventions are working – Who earned fewer than 10 credits last year, but is passing more than 5 classes this year?  This type of verification on the success of school based intervention programs will allow you to adjust to meet the needs of your struggling students, as well as earn points on the progress report.

Summer School counts!  After grades are uploaded in June, run the Total Credits option to guide summer school programming to ensure maximum points earned on this progress report metric.  To begin, define the maximum number of credits that can be earned in summer school and identify all students who could earn 10 credits with summer school courses.  Then run the Core Credits options to see if there are specific subject areas classes these students must take in summer school because they earned either fewer than 6 total in the core subject areas or 0 credits in any core subject area.

Schools need to minimize the number of students who have earned more than of 10 credits for the school year, but do not qualify for credit on this metric because they have not met the core credit requirements.  The Core Credits option displays the total number of core subject credits the student has earned along with any core areas where 0 credits were earned. Use this information to guide the summer school classes programmed for these students.

Weighted Regents Pass Rates  

The goal for this metric is to have all students in the school score a 65 or higher on the five core Regents exams (English, Global History, US History, any Math and any Science).  For most high school students, this metric matches the NYS graduation requirements.  However, even though NYS allows our Students with Disabilities to graduate from high school with safety net scores between 55 and 64, or by passing the RCTs, these scores cannot be used to satisfy this metric.   Because of the added difficulty involved in having this population pass the exams, this metric awards additional weight when they earn a passing score of 65 or higher.  It is crucial for both the student’s graduation and the school progress report that students who have not passed the exams get the support needed to be successful on these exams.

For guidance staff looking at individual students across all exams, the Data Tools Regents List displays the highest grades earned by the student in each of the five core exams.  RCTs are also listed in this report.  The report can be used to create groups of students who need interventions, support classes and tutoring and parent contact.

For Subject Area Assistant Principals or Department Chair people, the NCLB/GRAD Regents-RCT report provides information on students who have not met the exam requirement by subject area.  The report can be downloaded into excel and sent to your guidance or programming staff and used to schedule the students for both the exam and the support classes for the exams.

The New College Metrics –

These metrics first appeared on a last year’s progress report as an unscored measure.  2012-2013 will be scored, and schools must prepare to track these metrics to validate the DOE information. 

College Readiness Index (for this metric and the Closing the Achievement Gap metric)

One requirement to be college ready is to earn a 75 or higher on the English Regents and an 80 or higher and a 65 or higher on two Math Regents.  This metric has two very different sets of students that can be targeted to increase the score on this metric– the students who already took the exam and scored lower than the target scores of 75 and 80, and the students who have not yet taken the exam.  Skedula can help you track and target both metrics to increase your score on these metrics.

The NCLB/GRAD CUNY report displays the students who have taken the exams but have not reached the college-ready thresholds.   The table displays the highest score the student earned on the exam, so your school staff can make informed decisions regarding who should be retested. 

Students who have never taken the exam need a different measure to predict their success on the Regents.  The Skedula gradebook can provide you with this predictor.  Administrators have access to the Class Averages page for all classes.  Use the report to look for students with averages in the exam category that is above a certain grade so they can be targeted to receive additional support.  For example, using a teacher’s Algebra gradebook, we can target all students with an exam average between 70 and 85 for additional support to increase their math and exam skills to enable them to earn an 80 on the Algebra Regents.

College Enrollment Rate -

The first step in tracking this metric verifying is the post-secondary information input into ATS when a student graduates.  The information entered must be as accurate as possible.  Creating protocols for collecting this data is key to getting the score you, your staff and your students have earned on this metric.  There is a great deal of discussion in schools about using the cap and gown distribution to gather this data.  The majority of your students have finalized post secondary plans by the end of June and all of them want to attend graduation.  Design an information card that includes their after high school plans as well as an email address that can be used to contact the student in the fall if verification questions arise.  This information can be collected from the August graduates when they come to pick up their diplomas.  The ATS input should also be checked and printed in July and August for verification in the fall.

College Preparatory Course Index –

Students can only get credit for this metric once, so you want to schedule students so you catch as many students as possible in the college class net.  If a student has passed the Advanced Algebra Regents, that student will already be included in this.  If a student is still in Geometry, he or she can still contribute. The NCLB/GRAD Full Regents Report provides a list of students who have already been credited in this metric by passing the Algebra II, Math B, Chemistry or Physics Regents.  If your school offers college preparedness classes such as AP Psychology or College Now Sociology, a difficult decision may have to be made as to which student can be programmed for the class – do you choose the student who cannot add to this metric or the student who still can?  Only through thorough tracking and planning, can both students be given these opportunities to allow for their most fulfilling school experience and the school’s maximum points on the progress report. 

Closing the Achievement Gap – Additional Credits can be earned by your school by getting struggling populations earning advanced diplomas and passing the English and Math exams with college readiness scores.  The techniques and reports described earlier in this article (the NCLB/GRAD Graduation Requirements, Graduation Tracker and CUNY reports) can be used to increase your scores in this section.

I look forward to pursuing my hobby by continuing to review changes in the progress report, to adapt current and create new reports, and to help you maximize the points earned while allowing you to keep your focus on who really matters – the students.

Part 3 to follow "Deciphering the NCLB Waiver, what has really changed?

Stephanie Ring

Director of Accountability and Assessments 

CaseNEX-DataCation

Is Your School Maximizing Its Progress Report Score? Part 1 of 3

The NYC Progress Report shapes how your school is perceived by prospective parents.  Of the three NTC accountability reports, it is the only one that transforms schools ratings into letter grades.  This school is an A, that school is a C.  Validity of the metrics aside, parents understand that an A is better than a C.  

I have become obsessed with dissecting the Progress Report metrics.  My hobby is reviewing the new metric announcements in early spring and predicting how they will affect school scores.  The work begins when I plan reports we can add to Datacation that allow for verification and improvement of scores.

This article explores the DOE defined metrics of excellence and how to maximize their point value.  My intent is to help you improve your progress report score with your current resources.  By using data reports in the Datacation portals or other systems, your staff can satisfy the DOE standardized metrics and still have time to promote local programs that give your school the personality and vigor that reflect excellence.  

School Environment

This is the section of the Progress Report worth the fewest points and the only section of the progress report that includes qualitative data.  All of the qualitative data comes from the school survey results. 

From the School Survey

Communication – Skedula and Pupil Path can ease the communication burden. Communication is no longer defined solely as the spoken word.  Skedula includes many options:

Calendars – The School calendar appears on the Pupil Path home screen.  It informs readers of deadlines, sporting events, trips and the like.    Each course also has a calendar that apprises students and parents of assignment due dates.

Messaging – Parents can send and receive messages via their Pupil Path and personal email accounts.  Students can also send and receive messages using their account.  Messaging permits regular contact with parents and students.  Newsletters and curricular information can also be sent to parents and students through the messaging system. 

Curriculum posting – Teachers can create a non-graded assignment that includes information on upcoming curriculum.  This information will appear on the course calendar that appears on Pupil Path.

Anecdotals – Share with parents and staff the positive (and sometimes) negative happenings from classrooms.  Anecdotals can be posted simultaneously to multiple students thus reducing the time spent entering the information.  Progress reports are also available in anecdotal form.  All anecdotals can be customized to ensure that communications with parents are personalized .

Academic Expectation – Staff, parents and students must understand that what is expected at your school is the child’s highest performance level possible.  Using the Data Tool tracker reports, High School Readiness and College Readiness, compare the student’s current course grades, exam grades and attendance to the measures required to be successful when the student moves on to the next school level.  These reports are designed to facilitate discussions with your students and their parents of the academic performance needed to succeed at the next level of study.   The conversation about their child’s success in the future will further the perception that your school expects the most from the student in your school and in the future.

Attendance –

Daily attendance comprises 20% of the School Environment score and is the only quantitative measure in this section.  The Skedula Attendance Compliance Report allows schools to see students who have been marked absent for the day, but attended classes. This will help your school recognize students who qualify to have their daily attendance reversed.  Schools define the number of attended classes that triggers the attendance reversal, so the report can reflect school and DOE policies. 

The Skedula Daily Attendance report displays the attendance records for all students in the school or in any Skedula group.  Create a group of low attending students and use this report to check the progress of the interventions taken to improve attendance.

Stayed Tuned for Part 2 Next week.

Stephanie Ring

Director of Accountability and Assessments 

CaseNEX-DataCation