SCHOOL RESTRUCTURING AND ACCOUNTABILITY PROGRAM

Michigan Office of the
Auditor General
Thomas H. McTavish, C.P.A.
Auditor General

EXECUTIVE DIGEST #3120100
SCHOOL RESTRUCTURING AND ACCOUNTABILITY PROGRAM

INTRODUCTION
This report, issued in November 2001, contains the results of our performance audit of the School Restructuring and Accountability Program, Department of Education and Center for Educational Performance and Information.

AUDIT PURPOSE
This performance audit was conducted as part of the constitutional responsibility of the Office of the Auditor General. Performance audits are conducted on a priority basis related to the potential for improving effectiveness and efficiency.

BACKGROUND
Article VIII, Section 3 of the State Constitution vests in the State Board of Education the leadership and general supervision over all public instruction.

Effective September 28, 2000, Executive Order No. 2000-9 established as a temporary agency the Center for Educational Performance and Information and transferred to it certain functions and responsibilities previously performed by the Department, including preparing educational reports, such as the Michigan School Report and District Student Retention Report, and contracting with third parties to measure program effectiveness of K-12 systems. As a result, the Center will be responsible for reviewing the exceptions noted in Findings 1 and 2.

In March 1990, a framework for reforming Michigan schools was enacted into law (Act 25, P.A. 1990). This framework, commonly known as "Public Act 25," contained four components pertaining directly to school districts and buildings. Included were provisions for a school building-level and school district-level three- to five-year school improvement process, the development and implementation of a school district core curriculum, a building-level accreditation process, and annual education reporting to each community and to the intermediate school district of which each school district is a member. The components, while addressed separately in the law, were linked together, creating a system within which schools address the needs of their students and develop programs and strategies, in conjunction with parents and community members, to meet those needs. The goal of Public Act 25 was to produce high quality programs and services leading to improved educational performance by all students.

In 1993, legislation added a student performance requirement for school accreditation. The accreditation status of schools was announced for the first time in 1995.

The School Restructuring and Accountability Program is administered by the Department's Office of School Excellence. The mission of the Office is to provide leadership and activities to help Michigan schools become high quality schools and educational systems. The Program supports local school improvement through the development and promulgation of challenging performance and process standards, systems of support and accountability, and expectations for continuous progress to elevate the achievement of all Michigan students. The Program is responsible for policy development and Statewide leadership to implement school improvement, annual education reporting, and accreditation, consistent with the Revised School Code of 1995, within all public schools, intermediate school districts, and public school academies. The Program cooperates with other areas of the Department regarding State reform initiatives and the implementation of quality assurance for federal programs such as Title I, Special Education, and Goals 2000.

The Department holds school buildings and school districts responsible for adopting and implementing a three- to five-year school improvement plan by September 1 of each year. The Department requires attributes, such as goals centered on student academic learning, strategies to accomplish the goals, and evaluation of the plan, to be included in the three- to five-year school improvement plan.

For the fiscal year ended September 30, 2000, the Department expended approximately $1.6 million in administering its School Restructuring and Accountability Program responsibilities. As of September 30, 2000, the Department had 1 full-time equated employee assigned to the Program.

AUDIT OBJECTIVES AND CONCLUSIONS
Audit Objective: To assess the propriety of the Program's methodology for obtaining and reporting accurate school district performance data in the Michigan School Report.

Conclusion: We concluded that the Program's methodology for obtaining and reporting accurate school district performance data in the Michigan School Report needed improvement. Our assessment disclosed two material conditions:

The Department did not have an effective process for gathering school district accountability data (Finding 1).

The Center informed us that it will consider the recommendation made relative to this finding to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan.
The Department's methodology for calculating school district retention and dropout rates was not in accordance with statute and may have resulted in the Department's and school districts' reporting of unreliable information (Finding 2).
The Center informed us that it will consider the recommendation made relative to this finding to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan.
Audit Objective: To assess the reasonableness of the Program's methodology for accrediting school buildings throughout the State.

Conclusion: Based on our assessment, which disclosed two material conditions, we question the reasonableness of the Program's methodology for accrediting school buildings:

The Department did not assess the reasonableness of placing a high level of reliance on Michigan Educational Assessment Program (MEAP) test results for determining school building accreditation levels (Finding 3).
The Department agreed with the corresponding recommendation and informed us that it will review the reasonableness of MEAP as a part of the accreditation system.

The Department did not accredit school buildings and report such accreditation in compliance with statute and the Department's established process (Finding 4).
The Department agreed with the corresponding recommendation and informed us that it will accredit school buildings and report such accreditation in compliance with statute and the Department's established system. However, the Department disagreed with part of Finding 4.
Audit Objective: To assess the Program's effectiveness in assisting in developing and maintaining high quality schools and educational systems.

Conclusion: We concluded that the Program was generally effective in assisting in developing and maintaining high quality schools and educational systems. However, our assessment disclosed two reportable conditions related to technical assistance evaluation and administration and school improvement plan review (Findings 5 and 6).

AUDIT SCOPE AND METHODOLOGY
Our audit scope was to examine the program and other records of the School Restructuring and Accountability Program. Our audit was conducted in accordance with Government Auditing Standards issued by the Comptroller General of the United States and, accordingly, included such tests of the records and such other auditing procedures as we considered necessary in the circumstances.

Our ability to achieve our first audit objective in accordance with Government Auditing Standards was impeded by the Department's refusal to provide us unlimited access to employees associated with, and the withholding of requested documentation concerning adjustments to, the calculation of certain school year 1997-98 graduation/completion and dropout rates. The Department provided us with general information concerning its calculation of school district graduation/completion and dropout rates, documentation supporting the initial computation of school year 1997-98 school district graduation/completion and dropout rates, and documentation concerning revisions made to school year 1997-98 graduation/completion and dropout rates for the 8 school districts that we visited. Our review disclosed concerns with the Department's methodology for computing the rates (Finding 2). Not having unlimited access to employees to discuss, or having seen the supporting documentation for, revisions made to school year 1997-98 graduation/completion and dropout rates for districts other than those visited, we could not assess whether such discussions or documentation would have further affected our conclusion regarding our first objective and may have resulted in a more negative conclusion.

Our audit work included an examination of program and selected local school district records primarily for the period October 1996 through June 2000. Our methodology included a preliminary review of Program operations. This involved interviewing various Program staff and reviewing applicable statutes, policies and procedures, reports, and other reference materials. We reviewed audit reports on similar programs in other states.

We visited 25 school buildings in 8 local school districts, interviewed staff responsible for Program activities and using Program results, and obtained data pertaining to student academic performance. Also, we interviewed 3 technical assistance contract providers concerning their efforts to assist school districts in improving student performance.

We reviewed methods used by the Program to measure and evaluate Program effectiveness. We analyzed Program records to determine compliance with statutes.

We evaluated guidance provided to school districts for reporting data to the Department for inclusion in the Michigan School Report. We reviewed the Program's monitoring of data submitted by school districts that is used to determine school building accreditation status. We assessed the accuracy of data submitted by school districts and information contained in the Michigan School Report.

We analyzed the Department's accreditation methodology for reasonableness. We interviewed school district administrators regarding the appropriateness of the Department's accreditation methodology as an indicator of instructional quality.

We interviewed Program staff and school district administrators regarding outcomes demonstrating that the Program is developing high quality schools and educational systems. We analyzed school building student MEAP test results to determine whether school buildings provided with technical assistance demonstrated improved MEAP test results.

AGENCY RESPONSES
Our audit report contains 6 findings and 7 corresponding recommendations. The Center's preliminary response to Findings 1 and 2 indicated that it would consider the corresponding recommendations to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan. The Department's preliminary response to Findings 3 through 6 indicated that it agreed with the 5 corresponding recommendations but disagreed with part of Finding 4.

Full Audit Report - #3120100 - SCHOOL RESTRUCTURING AND ACCOUNTABILITY PROGRAM

Documents prefaced by require the Adobe Acrobat Reader®, a free application available on the Adobe homepage.

This page was created on October 26, 2001.


INTRODUCTION This report, issued in November 2001, contains the results of our performance audit of the School Restructuring and Accountability Program, Department of Education and Center for Educational Performance and Information.

AUDIT PURPOSE This performance audit was conducted as part of the constitutional responsibility of the Office of the Auditor General. Performance audits are conducted on a priority basis related to the potential for improving effectiveness and efficiency.

BACKGROUND Article VIII, Section 3 of the State Constitution vests in the State Board of Education the leadership and general supervision over all public instruction. Effective September 28, 2000, Executive Order No. 2000-9 established as a temporary agency the Center for Educational Performance and Information and transferred to it certain functions and responsibilities previously performed by the Department, including preparing educational reports, such as the Michigan School Report and District Student Retention Report, and contracting with third parties to measure program effectiveness of K-12 systems. As a result, the Center will be responsible for reviewing the exceptions noted in Findings 1 and 2. In March 1990, a framework for reforming Michigan schools was enacted into law (Act 25, P.A. 1990). This framework, commonly known as "Public Act 25," contained four components pertaining directly to school districts and buildings. Included were provisions for a school building-level and school district-level three- to five-year school improvement process, the development and implementation of a school district core curriculum, a building-level accreditation process, and annual education reporting to each community and to the intermediate school district of which each school district is a member. The components, while addressed separately in the law, were linked together, creating a system within which schools address the needs of their students and develop programs and strategies, in conjunction with parents and community members, to meet those needs. The goal of Public Act 25 was to produce high quality programs and services leading to improved educational performance by all students. In 1993, legislation added a student performance requirement for school accreditation. The accreditation status of schools was announced for the first time in 1995. The School Restructuring and Accountability Program is administered by the Department's Office of School Excellence. The mission of the Office is to provide leadership and activities to help Michigan schools become high quality schools and educational systems. The Program supports local school improvement through the development and promulgation of challenging performance and process standards, systems of support and accountability, and expectations for continuous progress to elevate the achievement of all Michigan students. The Program is responsible for policy development and Statewide leadership to implement school improvement, annual education reporting, and accreditation, consistent with the Revised School Code of 1995, within all public schools, intermediate school districts, and public school academies. The Program cooperates with other areas of the Department regarding State reform initiatives and the implementation of quality assurance for federal programs such as Title I, Special Education, and Goals 2000. The Department holds school buildings and school districts responsible for adopting and implementing a three- to five-year school improvement plan by September 1 of each year. The Department requires attributes, such as goals centered on student academic learning, strategies to accomplish the goals, and evaluation of the plan, to be included in the three- to five-year school improvement plan. For the fiscal year ended September 30, 2000, the Department expended approximately $1.6 million in administering its School Restructuring and Accountability Program responsibilities. As of September 30, 2000, the Department had 1 full-time equated employee assigned to the Program.

AUDIT OBJECTIVES AND CONCLUSIONS Audit Objective: To assess the propriety of the Program's methodology for obtaining and reporting accurate school district performance data in the Michigan School Report. Conclusion: We concluded that the Program's methodology for obtaining and reporting accurate school district performance data in the Michigan School Report needed improvement. Our assessment disclosed two material conditions: The Department did not have an effective process for gathering school district accountability data (Finding 1). The Center informed us that it will consider the recommendation made relative to this finding to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan. The Department's methodology for calculating school district retention and dropout rates was not in accordance with statute and may have resulted in the Department's and school districts' reporting of unreliable information (Finding 2). The Center informed us that it will consider the recommendation made relative to this finding to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan. Audit Objective: To assess the reasonableness of the Program's methodology for accrediting school buildings throughout the State. Conclusion: Based on our assessment, which disclosed two material conditions, we question the reasonableness of the Program's methodology for accrediting school buildings: The Department did not assess the reasonableness of placing a high level of reliance on Michigan Educational Assessment Program (MEAP) test results for determining school building accreditation levels (Finding 3). The Department agreed with the corresponding recommendation and informed us that it will review the reasonableness of MEAP as a part of the accreditation system. The Department did not accredit school buildings and report such accreditation in compliance with statute and the Department's established process (Finding 4). The Department agreed with the corresponding recommendation and informed us that it will accredit school buildings and report such accreditation in compliance with statute and the Department's established system. However, the Department disagreed with part of Finding 4. Audit Objective: To assess the Program's effectiveness in assisting in developing and maintaining high quality schools and educational systems. Conclusion: We concluded that the Program was generally effective in assisting in developing and maintaining high quality schools and educational systems. However, our assessment disclosed two reportable conditions related to technical assistance evaluation and administration and school improvement plan review (Findings 5 and 6).

AUDIT SCOPE AND METHODOLOGY Our audit scope was to examine the program and other records of the School Restructuring and Accountability Program. Our audit was conducted in accordance with Government Auditing Standards issued by the Comptroller General of the United States and, accordingly, included such tests of the records and such other auditing procedures as we considered necessary in the circumstances. Our ability to achieve our first audit objective in accordance with Government Auditing Standards was impeded by the Department's refusal to provide us unlimited access to employees associated with, and the withholding of requested documentation concerning adjustments to, the calculation of certain school year 1997-98 graduation/completion and dropout rates. The Department provided us with general information concerning its calculation of school district graduation/completion and dropout rates, documentation supporting the initial computation of school year 1997-98 school district graduation/completion and dropout rates, and documentation concerning revisions made to school year 1997-98 graduation/completion and dropout rates for the 8 school districts that we visited. Our review disclosed concerns with the Department's methodology for computing the rates (Finding 2). Not having unlimited access to employees to discuss, or having seen the supporting documentation for, revisions made to school year 1997-98 graduation/completion and dropout rates for districts other than those visited, we could not assess whether such discussions or documentation would have further affected our conclusion regarding our first objective and may have resulted in a more negative conclusion. Our audit work included an examination of program and selected local school district records primarily for the period October 1996 through June 2000. Our methodology included a preliminary review of Program operations. This involved interviewing various Program staff and reviewing applicable statutes, policies and procedures, reports, and other reference materials. We reviewed audit reports on similar programs in other states. We visited 25 school buildings in 8 local school districts, interviewed staff responsible for Program activities and using Program results, and obtained data pertaining to student academic performance. Also, we interviewed 3 technical assistance contract providers concerning their efforts to assist school districts in improving student performance. We reviewed methods used by the Program to measure and evaluate Program effectiveness. We analyzed Program records to determine compliance with statutes. We evaluated guidance provided to school districts for reporting data to the Department for inclusion in the Michigan School Report. We reviewed the Program's monitoring of data submitted by school districts that is used to determine school building accreditation status. We assessed the accuracy of data submitted by school districts and information contained in the Michigan School Report. We analyzed the Department's accreditation methodology for reasonableness. We interviewed school district administrators regarding the appropriateness of the Department's accreditation methodology as an indicator of instructional quality. We interviewed Program staff and school district administrators regarding outcomes demonstrating that the Program is developing high quality schools and educational systems. We analyzed school building student MEAP test results to determine whether school buildings provided with technical assistance demonstrated improved MEAP test results.

AGENCY RESPONSES Our audit report contains 6 findings and 7 corresponding recommendations. The Center's preliminary response to Findings 1 and 2 indicated that it would consider the corresponding recommendations to support its review of improvements needed in the data collection and reporting processes for all educational related data in Michigan. The Department's preliminary response to Findings 3 through 6 indicated that it agreed with the 5 corresponding recommendations but disagreed with part of Finding 4.