This is a charter middle school that serves predominately low-income, minority students (grades 6–8) in Harlem who are usually 2 or 3 years behind grade level. The program is rated Effective. Students who were admitted to the school showed statistically significant improvements in English Language Arts scores, math scores, and attendance rates, compared with students who were not admitted. However, there were no significant differences between groups on matriculation.
This program’s rating is based on evidence that includes at least one high-quality randomized controlled trial.
Program Goals/Target Population
The Harlem Children’s Zone (HCZ) is a 97-block area in the Harlem neighborhood of New York, NY, that combines "no excuses" charter schools with communitywide initiatives to address some of the main problems that underprivileged children face every day, such as inadequate schools, high-crime neighborhoods, and health issues. One important component of the HCZ Project is the Promise Academy Charter Middle School, which aims to provide students in grades 6–8 with a well-rounded, high-quality education.
The Promise Academy middle school serves predominately low-income, minority students in Harlem who are usually 2 or 3 years behind grade level. While the community programs offered by HCZ are available to anyone living near the area, the charter middle school is not open to all students. Because of the limited amount of space available, admission to the Promise Academy is only offered to students whose numbers are randomly selected during the middle school lottery. The school began in 2004 with 100 sixth graders, and by 2009, there were 700 middle school students in the system.
The Promise Academy combines structural reforms with wraparound services to provide students with a comprehensive college preparatory program. The Promise Academy provides an extended school day and year, coordinated afterschool tutoring, and additional classes on Saturdays for anyone who needs extra help with homework. In the classroom, strong focus is placed on mastering basic skills, especially literacy and mathematics skills. Students have more than 2 hours of literacy instruction and more than 90 minutes of mathematics instruction every day. Students also explore coursework in the arts and sciences.
In addition, many students spend 10 or more hours a day in school. An average school day runs from 8 a.m. to 4 p.m., which is a 20 percent increase over a typical school day. Students are also given the opportunity to participate in afterschool programs, such as photography or music activities, that run from 4 p.m. to 6 p.m. The academic year is 210 days of school, which includes a mandatory 25-day summer program. Only 180 days of school are required by law.
The Promise Academy also runs a health clinic in the middle school building that provides students with free medical, dental, and mental-health services. Students are screened upon entering the school, and they receive regular check-ups through the Children’s Health Fund. Students also receive incentives for achievements (such as money or trips), nutritious and freshly made meals, bus fare, and other intangible benefits, such as support from committed staff members.
The Promise Academy Charter Middle School is part of the HCZ Project that provides a continuum of services focusing on the developmental needs of children in Harlem. In addition to the middle school programs, the Project includes elementary school programs that are available to students in grades K–5. However, the evaluation research examining the effectiveness of the elementary school programs has not yet reached the rigor of the research looking at the middle school programs.
Dobbie and Fryer (2010) found that students who won the lottery and attended the Promise Academy middle school showed statistically significant improvements in math scores, compared with the control group students (i.e., youth who did not win the lottery) who did not attend.
English Language Arts (ELA) Scores
Students who attended the Promise Academy middle school showed statistically significant improvements in ELA scores, compared with control group students.
Students who attended the Promise Academy middle school showed a statistically significant reduction in absences in the first 180 days of school, compared with control group students.
There were no statistically significant differences between students in matriculation (which measured whether a student was “on-time” given their expected grade).
Dobbie and Fryer (2010) examined the effects of being admitted to or attending the Promise Academy charter schools in the Harlem Children’s Zone (HCZ) on academic outcomes. In New York State, oversubscribed charter schools are required by law to allocate enrollment offers through a random lottery system. The study took advantage of this system by comparing the average outcomes of students who “won” the lottery and were offered admission to the Promise Academy to the average outcomes of students who “lost” the lottery and were not offered admission.
The sample included students from the middle school lotteries from 2005 through 2006 (data was not available from the 2004 middle school lottery). The treatment group included 169 lottery winners in grades 6–8. The lottery winners were 51 percent male, and 84 percent black and 14.5 percent Hispanic. Seventy-two percent qualified for free lunch. The control group included 289 grade 6–8 youth who did not win the lottery. The group was 43.3 percent male, and 82.6 percent black and 15.4 percent Hispanic. Sixty-five percent qualified for free lunch. There were no statistically significant differences between the groups.
The study used an intent-to-treat (ITT) analysis, which meant the treatment group included all students who were offered admission to Promise Academy, even if they did not attend. The ITT analysis therefore captured the effect of being offered admission, not of actually attending. The study also included treatment-on-treated (TOT) analysis, which looked at the average effect of enrollment in the Promise Academy on those students who won the lottery and chose to attend. The results for the TOT analysis are provided in the Other Information section of the profile.
Data was collected from files at HCZ as well as administrative data on student demographics and academic outcomes from the New York City Department of Education (NYCDOE). Academic outcomes in math and English Language Arts (ELA) were measured by the New York State tests that are conducted in the winters of third through eighth grade. The math portion of the exam includes questions on number sense and operations, algebra, geometry, measurement, and statistics. The section on ELA assesses students on information and understanding, literary response and expression, and critical analysis and evaluation. The study authors constructed measures of absenteeism and matriculation using NYCDOE data. Absenteeism was measured as the total number of absences a student accumulated during the first 180 days of the school year (after the first 180 days, NYCDOE no longer collects absence data from schools). Matriculation was used as an indicator for whether a student was “on-time” given their expected grade. Subgroup analysis was conducted by grade level.
In the 2008–09 school year, the Harlem Children’s Zone in the Harlem neighborhood of New York, NY, spent approximately $19,272 per pupil at the Promise Academy. This included in-school costs and afterschool programs. This can be compared to the median school district in New York State in 2006 that spent approximately $16,171 per pupil (Dobbie and Fryer 2010).
For more information about Harlem Children’s Zone, please see the program’s website: https://hcz.org/
Other Information (Including Subgroup Findings)
Dobbie and Fryer (2010) looked at the effects of being admitted (lottery winners) to Promise Academy by grade level. They also examined the effects of enrolling in the charter school. Lottery winners were 11.9 percent more likely to be performing on grade level in math in the sixth grade, 16.3 percent more likely in the seventh grade, and 27 percent more likely in eighth grade, compared with lottery losers across the three grades. These differences were statistically significant. The treatment-on-treated (TOT) analysis, which looked at the effects of actually attending the charter middle school, showed a similar pattern. There was a convergence in test scores between students who attended the Promise Academy middle school and the average white student in NYC. In other words, Promise Academy students had nearly caught up to the average white student in New York City public schools in math test scores after 3 years of schooling. The TOT analysis found that lottery winners who enrolled in the charter middle school scored 0.338 standard deviations higher in sixth-grade math, 0.371 higher in seventh-grade math, and 0.745 higher in eighth-grade math, compared with the control group. The results were statistically significant for all 3 years.
As with math scores, lottery winners, losers, and the average African American student in NYC had similar ELA scores, which were roughly 0.65 standard deviations behind the average white student in the city. ITT analysis showed there were no significant differences in ELA scores between the lottery winners and losers until eighth grade. In the eighth grade, however, Promise Academy students had significantly higher scores than the control group, although the treatment effect was not as strong on ELA scores as on math scores (i.e., lottery winners scored 0.196 standard deviations higher in eighth grade). The TOT analysis found that scores for students enrolled in Promise Academy were 0.279 standard deviations higher in eighth
Evidence-Base (Studies Reviewed)
These sources were used in the development of the program profile:Study 1
Dobbie, Will, and Roland G. Fryer Jr. 2010. Are High-Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children’s Zone
. Cambridge, Mass.: The Kennedy School, Harvard University.http://scholar.harvard.edu/files/fryer/files/hcz_nov_2010.pdf
These sources were used in the development of the program profile:
Hoxby, Caroline M., and Sonali Murarka. 2009. Charter Schools in New York City: Who Enrolls and How They Affect Their Students’ Achievement
. Cambridge, Mass.: National Bureau of Economic Research Working Paper.http://www.nber.org/papers/w14852
Hoxby, Caroline M., Jenny Lee Kang, and Sonali Murarka. 2009. Technical Report: How New York City Charter Schools Affect Achievement
. Cambridge, Mass.: National Bureau of Economic Research.
Harlem Children’s Zone. N/d. Whatever It Takes: A White Paper on the Harlem Children’s Zone