Is Your School Maximizing Its Progress Report Score? Accountability After the ESEA Waiver Part 3 of 3

On May 29^th, New York State announced that the state was granted an ESEA waiver for the federal No Child Left Behind rules.  Those who just read the headline cheered that the NCLB rules were waived for New York.  After all, before the waiver, schools only had one year before students in the 2014 cohort would have to reach a perfect performance index of 200 to keep the school’s good standing.  Data Specialists and Principals let out a collective sigh of relief.   Did this break come without cost?  What did New York State schools actually get in this waiver and what was the cost for receiving it?

      First, a brief description of the changes in rules and calculations under the waiver:

  

      1) The AMO target of a perfect 200 Performance Index for the 2014 cohort is extended.  The new AMO target is designed to have schools make up half of the gap between their 2010-2011 performance index and the perfect score of 200 over the next 6 years.  Effective AMO and Safe Harbor targets were unchanged by the waiver.

       2) The levels for performance on high school exams have changed.  Previously, English and Math shared a definition of performance levels.  Under the waiver, college readiness now helps to define these levels.

Level	Old Rules	New Rules
Level 4	Highest Regents score between 85-100	Highest Regents score between 90-100
Level 3	Highest Regents score between 65 – 84	English - Highest Regents Score between 75-89 Math – Highest Regents Score between 80-89
Level 2	Highest Regents score between 55-64 or RCT Pass for IEP students	English - Highest Regents Score between 65-74 Math – Highest Regents Score between 65-79
Level 1	Highest Regents score between 0-54 or RCT Fail	Highest Regents score 0-64, RCT Pass or Fail

The immediate impact of these new levels will be that performance index for (almost) every subgroup in (almost) every school will drop substantially this year

      3) Before the waiver, schools that did not make their target for an exam after 2 years would be classified as a School in Need of Improvement.  This reliance on performance index as the only metric used to define a School in Need Improvement, Corrective Action and Restructuring has ended under the waiver.  Priority and Focus schools will now be identified based on an expanded definition using performance index combined with low graduation rates (below 60% for multiple years).  Priority Schools will be determined and have until 2014-2015 to implement whole school reform.  Focus Schools are located in Focus Districts (districts containing priority schools) where the lowest performing students (performance index or graduation) are part of the focus identified subgroup.  A new category has been added for top performing schools.  Reward schools, the highest performing schools in the district, could be eligible for grant money to help their students.

As I read the waiver and the approval, I couldn’t help but categorize it into three parts - the good, the bad and the ugly.

The Good – There are some very good elements to the waiver.  Schools have received relief from the impact of having to receive a perfect performance index, where every student must pass the English and Math Regents with a grade of 65 or more.  Another positive change is the removal of the graduation requirement being tied to the use of the Safe Harbor target.  The Safe Harbor target was designed for subgroups performing so far from the average performance index that the AMO and Effective AMO targets were not sufficient to measure their annual progress.  Safe Harbor is the lowest target for struggling subgroups (it could actually be the highest for successful subgroups), and  holding the same struggling subgroup to the graduation requirement often meant it that the Safe Harbor target could not be used to met AYP.  Also good news for schools is that the school now has 2 years to improve performance in any subgroup before it becomes a local assistance school.  The former rules only gave schools one year to improve before they went SINI.   Schools will receive more points for students who pass Regents at a college ready level of 75 for English and 80 for Math.  This matches the standards being asked for by many colleges and rewarding schools for their students meet more rigorous standards.  On a personal note, the good news is that the waiver provides a fresh look at the accountability rules under NCLB.  It also stirs my imagination on what type of reports we can design for you in Skedula and the NCLB portals.

The Bad – Schools will now only receive partial credit towards AYP when students pass the Regents but do not achieve a college ready score.  I do believe that college readiness should be rewarded, but students who meet the graduation requirement of a mark of 65 to 74 on English (65 to 79 on Math) will only earn half credit for the school toward Performance Index.  This means high school teachers and staff must convince freshmen that they must begin preparing for college at the age of 14.  Even though students will have the opportunity to take two more Regents exams in math, if the student does not earn the 80 on the Algebra exam, it is more than likely that they will not earn the 80 on the Geometry or Advanced Algebra exams.    Although I do believe that students should discuss and plan for their futures, the practicality of having every 14 year old student work at this level will be a struggle for schools, teachers and students.   The English exam will not prove to be the same struggle as the exam is taken by high school juniors, who are already thinking about college plans.   An additional piece of bad news for schools is even though the goal of reaching a Performance Index of 200 has been put off until 2017, the benchmark AMO target that must be met is calculated at the pre-waiver, higher performance indices, while the schools performance index is calculated on the new lower point values.  Students in the 2013 cohort who scored between a 65 and 79 in math then moved forward into higher math courses may now need to take a non-credit bearing course so the school can earn full credit toward their performance index.  Will the score of 80 prepare them better for college?  Definitely for CUNY, but only maybe for other schools who do their own placement testing.  This is a sudden shift for a cohort in their final year of high school.

The Ugly - I was a teacher in NYC for 10 years and became the school data specialist when the position was created by the DOE.  For years I taught a first period preparation class for the Math A and Algebra Regents.  Many of the students in my class were special needs students with an exam graduation requirement of marks of 55 or higher.  One year I taught a senior special needs student who had made the decision to start life anew – he was leaving drugs and gangs behind.  Most days for him were a struggle, some of his steps were backwards, but his counselors along with me and his other teachers worked with him every day to help pull him towards his new life.  At the end of the term, he scored a 57 on the Math Regents.  He was as proud as could be to meet his graduation requirement after all of his hard work.  Under the new NCLB waiver, my school would not earn points towards NCLB Performance Index for his success, even though it met his graduation requirement.  I am a data geek and understand the need for the state to set high goals to facilitate school improvement, but the human side of me knows that this isn’t the only way to measure and reward success.  He took my class in the fall of his senior year, and we could have forced him to take the class and exam again to earn the 65, but we let him go.  He earned his way out of the course by completing his graduation requirement - especially since college was not his path.  This is only one of many stories educators can tell where the good of the student is at odds with the good of the school’s data.  Here’s another - The school needed a student to pass the Living Environment Regents to guarantee an 80% graduation rate.  However, if she passed and earned the local diploma, she would be disqualified from her vocational training program for students with disabilities.  As I said, there are many stories like these.  The good of the school should never be in direct conflict with the good of the student.  Unyielding metrics such as the new NCLB college metrics ensure these conflicts will continue.  

The one given in education accountability is that it does remain static.   I remain hopeful that the designers of the metrics can find a balance to serve all of our students while making our schools stronger.

 

Stephanie Ring

Director of Accountability and Assessments 

CaseNEX-DataCation

Using Cases to Evaluate Educators

Evaluators expend huge amounts of energy observing teachers in typical self-contained classrooms, searching for signs of effective teaching. In doing so they may gain a sense of what teaching influences student learning in these situations.  At the same time they risk losing opportunities to judge teaching and professional behavior under other conditions. Case methods offer a possible remedy to this problem.

“We are inclined to the view that the case method—long used in medicine and law, and more recently in public administration and business—will in the coming decades be relied upon increasingly in the field of education, both in the pre-service and the in-service training of teachers and administrators.”  (R.N.Bush, Stanford, 1954).

Early adopter Bob Bush correctly predicted the power of cases to educate educators. We have stretched his  prediction to use cases—or multimedia web-based slices of educational life—to judge educators’ abilities to make professional decisions in any and all aspects of their work.

How?  We record situations that educators face in their everyday lives and enrich the accounts with data and artifacts.  Teaching Notes pose questions that prompt others—teachers, administrators, and parents—to make explicit what they know and what they think they know that is relevant to the case.  We push case analysts to describe what they might do, why, and  what might happen if they pursued a particular course of action.  Answers to these questions provide the grist for judging people’s abilities to forecast behavior that is more and less defensible.  Critical perspectives—analyses offered by people recognized as knowledgeable in the facts  of the case—serve as benchmarks against which analyses can be judged.  

Like other empirical research, good case analyses are high in external validity.  Smart educators craft solutions that plausibly “fit” the people involved and the ecology—classrooms, schools, and communities—of the case itself.

Bob McNergney is co-founder of and partner in CaseNEX-DataCation, LLC.

DataCation, PupilPath, Skedula and the SLC?

Maybe I am showing my age, but I don’t think so; the SLC has always stood for Small Learning Community and still does.   Many large schools like New Dorp High School, however, use the Small Learning Community model to create smaller learning environments within big schools.  We have witnessed this strategy improve academic performance at many of our bigger schools. 

But now, you will want to be on the lookout for the new version of SLC - short for “Shared Learning Collaborative.”  The new SLC offers unique technologies and services to store data by schools, districts, and states.  This means educators with information about academic learning can connect students, content, and tools to personalize instruction for all students. 

After attending the SLC Camp in NYC I saw hundreds of educators, parents, and developers get excited about what “collaboration” in the schools might come to mean.  Renewed efforts to tag content are facilitating the connection of data points to form instructional maps that can guide individuals toward common ends.

If you do a Google Search on “Slope of a Line,” for example, it will yield 12,900,000 URLs.  The last time I checked, nobody had time to evaluate all that content. The SLC, however, will allow you to tap into many content resources that are immediately relevant to your students’ performances.

Whether you are referring to “Small Learning Community” or now “Shared Learning Collaborative,” the SLC makes sense in today’s schools.

Peter Bencivenga

President 

CaseNEX - DataCation

Have you seen Skedula in NYC?  If not watch it here: http://vimeo.com/39936168

Is Your School Maximizing Its Progress Report Score? Part 2 of 3

The NYC Progress Report shapes how your school is perceived by prospective parents.  Of the three NTC accountability reports, it is the only one that transforms schools ratings into letter grades.  This school is an A, that school is a C.  Validity of the metrics aside, parents understand that an A is better than a C.  

I have become obsessed with dissecting the Progress Report metrics.  My hobby is reviewing the new metric announcements in early spring and predicting how they will affect school scores.  The work begins when I plan reports we can add to DataCation that allow for verification and improvement of scores.

This article explores the DOE defined metrics of excellence and how to maximize their point value.  My intent is to help you improve your progress report score with your current resources.  By using data reports in the DataCation portals or other systems, your staff can satisfy the DOE standardized metrics and still have time to promote local programs that give your school the personality and vigor that reflect excellence.  

Student Performance

The Student Performance Section metrics are all about graduation rates.  Its metrics include the school’s simple graduation rate (the number of students who earned diplomas) as well as the school’s weighted graduation rate (the number of students who earned advanced diplomas).

Graduation –

There is one simple rule - the more graduates, the better.  Be proactive about your graduation rate.  The hard work for graduation happens in the classroom between the teacher and student, but if the student is not scheduled for the required classes, she or he will never graduate.  Use the NCLB/GRAD Graduation Requirements report to ensure proper scheduling of classes needed for graduation.  The report not only displays which students have not met the course requirements, but it also displays if the student is programmed for a class in that subject area.  This report can help guidance staff correct student schedules early in the term.  This report also displays the number of exams the students must pass to meet graduation requirements.  If you know which students need certain exams, you can make informed decisions about teaching and class resources which will help both the students and the school progress report score.

Weighted Graduation (for this metric and the Closing the Achievement Gap metric) -

How many students in your school would earn an Advanced Regents diploma if they pass one more exam?  Have you targeted these students to provide support for the missing exam?  The NCLB/GRAD Graduation Requirements and Graduation Tracker reports can show you who these students are and which exam is outstanding.  Getting these students to pass this last exam will increase their score on this metric by 25%.

The Graduation Tracker Report displays credits earned and exam grades and calculates diploma types.  Flags are included so you can track by ethnicity, by ELL status and IEP status to enable you to track students in these groups for the Closing the Achievement Gap metrics.  This report calculates diploma types including Local, Regents, Regents with Honors, Advanced Regents and Advanced Regents with Honors.  The higher the diploma earned, the more points earned on your progress report.  Where the averaging needed to determine the honors designation is not difficult, it is time consuming.  Your guidance staff can use their time in more productive ways than calculating exam averages.  Let the counselors counsel and the computers compute while the school earns the maximum and correct progress report score. 

Student Progress –

The largest piece of the Progress Report, Student Progress, is worth 60 points.  Progress is defined through credits earned and exams passed.

Keeping students on track to earn 10 or more credits – Half of the points in the section of the progress report are earned in this metric.  This metric measures students in the 1^st, 2^nd and 3^rd year cohorts to earn “10 or more credits” in a school year.  These 10 credits must include 6 credits in the core subject areas of English, Math, History and Science with at least some credit earned in each of these areas.

The NCLB/GRAD Progress Credit Accumulation report will allow you to be proactive in your programming and follow up of these students.  The report has four options to provide full information on this complex metric.  At the end of each marking period, run the Total Credits option.  The report includes the number of credits earned last year, the number of credits earned this year and the number of classes passed and failed for the marking period.  Using a two-term school as an example, you can find which students earned more than 10 credits last year but are NOT passing 5 or more classes this year.  This information will allow you to target these students to get them back on track.  If you turn this data around, you can confirm interventions are working – Who earned fewer than 10 credits last year, but is passing more than 5 classes this year?  This type of verification on the success of school based intervention programs will allow you to adjust to meet the needs of your struggling students, as well as earn points on the progress report.

Summer School counts!  After grades are uploaded in June, run the Total Credits option to guide summer school programming to ensure maximum points earned on this progress report metric.  To begin, define the maximum number of credits that can be earned in summer school and identify all students who could earn 10 credits with summer school courses.  Then run the Core Credits options to see if there are specific subject areas classes these students must take in summer school because they earned either fewer than 6 total in the core subject areas or 0 credits in any core subject area.

Schools need to minimize the number of students who have earned more than of 10 credits for the school year, but do not qualify for credit on this metric because they have not met the core credit requirements.  The Core Credits option displays the total number of core subject credits the student has earned along with any core areas where 0 credits were earned. Use this information to guide the summer school classes programmed for these students.

Weighted Regents Pass Rates  

The goal for this metric is to have all students in the school score a 65 or higher on the five core Regents exams (English, Global History, US History, any Math and any Science).  For most high school students, this metric matches the NYS graduation requirements.  However, even though NYS allows our Students with Disabilities to graduate from high school with safety net scores between 55 and 64, or by passing the RCTs, these scores cannot be used to satisfy this metric.   Because of the added difficulty involved in having this population pass the exams, this metric awards additional weight when they earn a passing score of 65 or higher.  It is crucial for both the student’s graduation and the school progress report that students who have not passed the exams get the support needed to be successful on these exams.

For guidance staff looking at individual students across all exams, the Data Tools Regents List displays the highest grades earned by the student in each of the five core exams.  RCTs are also listed in this report.  The report can be used to create groups of students who need interventions, support classes and tutoring and parent contact.

For Subject Area Assistant Principals or Department Chair people, the NCLB/GRAD Regents-RCT report provides information on students who have not met the exam requirement by subject area.  The report can be downloaded into excel and sent to your guidance or programming staff and used to schedule the students for both the exam and the support classes for the exams.

The New College Metrics –

These metrics first appeared on a last year’s progress report as an unscored measure.  2012-2013 will be scored, and schools must prepare to track these metrics to validate the DOE information. 

College Readiness Index (for this metric and the Closing the Achievement Gap metric)

One requirement to be college ready is to earn a 75 or higher on the English Regents and an 80 or higher and a 65 or higher on two Math Regents.  This metric has two very different sets of students that can be targeted to increase the score on this metric– the students who already took the exam and scored lower than the target scores of 75 and 80, and the students who have not yet taken the exam.  Skedula can help you track and target both metrics to increase your score on these metrics.

The NCLB/GRAD CUNY report displays the students who have taken the exams but have not reached the college-ready thresholds.   The table displays the highest score the student earned on the exam, so your school staff can make informed decisions regarding who should be retested. 

Students who have never taken the exam need a different measure to predict their success on the Regents.  The Skedula gradebook can provide you with this predictor.  Administrators have access to the Class Averages page for all classes.  Use the report to look for students with averages in the exam category that is above a certain grade so they can be targeted to receive additional support.  For example, using a teacher’s Algebra gradebook, we can target all students with an exam average between 70 and 85 for additional support to increase their math and exam skills to enable them to earn an 80 on the Algebra Regents.

College Enrollment Rate -

The first step in tracking this metric verifying is the post-secondary information input into ATS when a student graduates.  The information entered must be as accurate as possible.  Creating protocols for collecting this data is key to getting the score you, your staff and your students have earned on this metric.  There is a great deal of discussion in schools about using the cap and gown distribution to gather this data.  The majority of your students have finalized post secondary plans by the end of June and all of them want to attend graduation.  Design an information card that includes their after high school plans as well as an email address that can be used to contact the student in the fall if verification questions arise.  This information can be collected from the August graduates when they come to pick up their diplomas.  The ATS input should also be checked and printed in July and August for verification in the fall.

College Preparatory Course Index –

Students can only get credit for this metric once, so you want to schedule students so you catch as many students as possible in the college class net.  If a student has passed the Advanced Algebra Regents, that student will already be included in this.  If a student is still in Geometry, he or she can still contribute. The NCLB/GRAD Full Regents Report provides a list of students who have already been credited in this metric by passing the Algebra II, Math B, Chemistry or Physics Regents.  If your school offers college preparedness classes such as AP Psychology or College Now Sociology, a difficult decision may have to be made as to which student can be programmed for the class – do you choose the student who cannot add to this metric or the student who still can?  Only through thorough tracking and planning, can both students be given these opportunities to allow for their most fulfilling school experience and the school’s maximum points on the progress report. 

Closing the Achievement Gap – Additional Credits can be earned by your school by getting struggling populations earning advanced diplomas and passing the English and Math exams with college readiness scores.  The techniques and reports described earlier in this article (the NCLB/GRAD Graduation Requirements, Graduation Tracker and CUNY reports) can be used to increase your scores in this section.

I look forward to pursuing my hobby by continuing to review changes in the progress report, to adapt current and create new reports, and to help you maximize the points earned while allowing you to keep your focus on who really matters – the students.

Part 3 to follow "Deciphering the NCLB Waiver, what has really changed?

Stephanie Ring

Director of Accountability and Assessments 

CaseNEX-DataCation

Is Your School Maximizing Its Progress Report Score? Part 1 of 3

The NYC Progress Report shapes how your school is perceived by prospective parents.  Of the three NTC accountability reports, it is the only one that transforms schools ratings into letter grades.  This school is an A, that school is a C.  Validity of the metrics aside, parents understand that an A is better than a C.  

I have become obsessed with dissecting the Progress Report metrics.  My hobby is reviewing the new metric announcements in early spring and predicting how they will affect school scores.  The work begins when I plan reports we can add to Datacation that allow for verification and improvement of scores.

This article explores the DOE defined metrics of excellence and how to maximize their point value.  My intent is to help you improve your progress report score with your current resources.  By using data reports in the Datacation portals or other systems, your staff can satisfy the DOE standardized metrics and still have time to promote local programs that give your school the personality and vigor that reflect excellence.  

School Environment

This is the section of the Progress Report worth the fewest points and the only section of the progress report that includes qualitative data.  All of the qualitative data comes from the school survey results. 

From the School Survey

Communication – Skedula and Pupil Path can ease the communication burden. Communication is no longer defined solely as the spoken word.  Skedula includes many options:

Calendars – The School calendar appears on the Pupil Path home screen.  It informs readers of deadlines, sporting events, trips and the like.    Each course also has a calendar that apprises students and parents of assignment due dates.

Messaging – Parents can send and receive messages via their Pupil Path and personal email accounts.  Students can also send and receive messages using their account.  Messaging permits regular contact with parents and students.  Newsletters and curricular information can also be sent to parents and students through the messaging system. 

Curriculum posting – Teachers can create a non-graded assignment that includes information on upcoming curriculum.  This information will appear on the course calendar that appears on Pupil Path.

Anecdotals – Share with parents and staff the positive (and sometimes) negative happenings from classrooms.  Anecdotals can be posted simultaneously to multiple students thus reducing the time spent entering the information.  Progress reports are also available in anecdotal form.  All anecdotals can be customized to ensure that communications with parents are personalized .

Academic Expectation – Staff, parents and students must understand that what is expected at your school is the child’s highest performance level possible.  Using the Data Tool tracker reports, High School Readiness and College Readiness, compare the student’s current course grades, exam grades and attendance to the measures required to be successful when the student moves on to the next school level.  These reports are designed to facilitate discussions with your students and their parents of the academic performance needed to succeed at the next level of study.   The conversation about their child’s success in the future will further the perception that your school expects the most from the student in your school and in the future.

Attendance –

Daily attendance comprises 20% of the School Environment score and is the only quantitative measure in this section.  The Skedula Attendance Compliance Report allows schools to see students who have been marked absent for the day, but attended classes. This will help your school recognize students who qualify to have their daily attendance reversed.  Schools define the number of attended classes that triggers the attendance reversal, so the report can reflect school and DOE policies. 

The Skedula Daily Attendance report displays the attendance records for all students in the school or in any Skedula group.  Create a group of low attending students and use this report to check the progress of the interventions taken to improve attendance.

Stayed Tuned for Part 2 Next week.

Stephanie Ring

Director of Accountability and Assessments 

CaseNEX-DataCation