Ultimate World Financial Map: Massachusetts_Audits

Before we start.

These are copies of a few audits, none of AI were able to access the state of Massachusetts site, so I started copying and pasting the text that was readable to help my investigation.

A link to PERAC audits for anyone interested

PERAC audits

https://www.mass.gov/lists/retirement-board-audit-reports

Link to Emails between PERAC and myself, plus emails between PERAC and vendors. that's how my AI noticed the issues with this system.

PERAC emails

https://ultimateworldfinancialmap.blogspot.com/2025/12/publicrecordsrequestperacinvestmentreco.html

PERAC emails 2

https://ultimateworldfinancialmap.blogspot.com/2025/12/publicrecordsrequestperacinvestmentreco_22.html

Public_Records_Request–PERAC_Investment_Records_2024–2025 part 3

https://ultimateworldfinancialmap.blogspot.com/2025/12/perac-records-request-part-3.html

Links to all available Massachusetts state audits.

MA state auditor

https://www.mass.gov/lists/all-audit-reports-2011-to-today#2025-

PERAC Annual Reports

https://www.mass.gov/doc/2020-annual-report-full/download

https://www.mass.gov/doc/2021-annual-report-full/download

https://www.mass.gov/doc/2022-annual-report-full/download

https://www.mass.gov/doc/2023-annual-report-full-0/download

https://www.mass.gov/doc/2024-annual-report-full-0/download

PRIM reports, summary's and minutes

Links to stuff I copied from the PRIM site for AI

https://ultimateworldfinancialmap.blogspot.com/2025/12/prim-info-for-ai.html

Link to all PRIM reports, summary"s and minutes

PRIM

https://www.mapension.com/records-of-interest/

Beneficiaries of the Massachusetts State Employees' Retirement System (MSERS) and the Massachusetts Teachers' Retirement System (MTRS). Updated monthly. Retirees of municipal, county or regional entities as well as educators who retired from the Boston Public Schools are not listed here. For that data, please contact the applicable local or regional retirement system directly.

https://cthrupensions.mass.gov/#!/year/2025/

Some other random memo's and complaince paperwork I was sent

PERAC memo's, compliance and more

https://ultimateworldfinancialmap.blogspot.com/2025/12/peracmemosandmore.html

Site intructions:

Open Payroll is part of our commitment to improving transparency by providing a guided view through complex financial information.

This site provides a guided view through our budget and provides a transparent look at how we allocate public funds. The charts, graphs, and tables below are all highly interactive and we invite you to explore.

Massachusetts Audits

Official Audit Report – Issued November 8, 2024
Audit of Cybersecurity Awareness Training
Compliance Across Multiple State Agencies
For the period July 1, 2021 through April 30, 2023
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
November 8, 2024
Jason Snyder, Secretary and Commonwealth Chief Information Officer
Executive Office of Technology Services and Security
1 Ashburton Place, 8th Floor
Boston, MA 02108
Dear Mr. Snyder:
I am pleased to provide to you the results of the enclosed performance audit. Pursuant to our governing
statute, Section 12 of Chapter 11 of the Massachusetts General Laws, our audit covers multiple entities’
compliance with the Executive Office of Technology Services and Security’s cybersecurity training
standards. Specifically, the following entities were included as part of this comprehensive audit:
Executive Branch Agencies State Colleges and Universities Regional Transit Authorities
Executive Office of Technology
Services and Security
Framingham State University Cape Ann Transportation Authority
Bureau of the State House Holyoke Community College Cape Cod Regional Transit
Authority
Civil Service Commission Massachusetts Bay Community
College
Martha’s Vineyard Regional Transit
Authority
Department of Labor Standards Massasoit Community College Nantucket Regional Transit
Authority
Department of Mental Health North Shore Community College
Department of Public Health Northern Essex Community College
Department of Revenue Westfield State University (WSU)
Massachusetts Department of
Transportation
Group Insurance Commission
Massachusetts Parole Board
Registry of Motor Vehicles
State 911 Department
As is typically the case, this report details the audit objectives, scope, methodology, findings, and
recommendations for the audit period, July 1, 2021 through April 30, 2023. As you know, my audit team
discussed the contents of this report with agency managers. This report reflects those comments.
I appreciate you and all your efforts at the Executive Office of Technology Services and Security. The
cooperation and assistance provided to my staff during the audit went a long way toward a smooth
process. Thank you for encouraging and making available your team. I am available to discuss this audit if
you or your team have any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ............................................................................................... 10
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 16
1. EOTSS did not ensure that all of its employees completed cybersecurity awareness training................... 16
2. CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure that all of their employees
completed cybersecurity awareness training............................................................................................ 18
3. Seven state colleges and universities did not ensure that all of their employees completed cybersecurity
awareness training. .................................................................................................................................. 26
4. CATA, CCRTA, and VTA did not ensure that all of their employees completed cybersecurity awareness
training. .................................................................................................................................................... 32
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
List of Abbreviations
ii
LIST OF ABBREVIATIONS
911 State 911 Department
BSH Bureau of the State House
CATA Cape Ann Transportation Authority
CCRTA Cape Cod Regional Transit Authority
CSC Civil Service Commission
DLS Department of Labor Standards
DMH Department of Mental Health
DOR Department of Revenue
DPH Department of Public Health
EOTSS Executive Office of Technology Services and Security
FSU Framingham State University
GIC Group Insurance Commission
HCC Holyoke Community College
HRD Human Resources Division
MassDOT Massachusetts Department of Transportation
MBCC Massachusetts Bay Community College
MCC Massasoit Community College
MPB Massachusetts Parole Board
NECC Northern Essex Community College
NRTA Nantucket Regional Transit Authority
NSCC North Shore Community College
RMV Registry of Motor Vehicles
VTA Martha’s Vineyard Regional Transit Authority
WSU Westfield State University
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Executive Office of Technology Services and Security
(EOTSS), as well as 22 other executive branch agencies, state colleges and universities, and regional transit
authorities. This audit covers the period July 1, 2021 through April 30, 2023 and includes the following
agencies:
Executive Branch Agencies State Colleges and Universities Regional Transit Authorities
Executive Office of Technology
Services and Security (EOTSS)
Framingham State University (FSU) Cape Ann Transportation Authority
(CATA)
Bureau of the State House (BSH) Holyoke Community College (HCC) Cape Cod Regional Transit
Authority (CCRTA)
Civil Service Commission (CSC) Massachusetts Bay Community
College (MBCC)
Martha’s Vineyard Regional Transit
Authority (VTA)
Department of Labor Standards
(DLS)
Massasoit Community College
(MCC)
Nantucket Regional Transit
Authority (NRTA)
Department of Mental Health
(DMH)
North Shore Community College
(NSCC)
Department of Public Health (DPH) Northern Essex Community College
(NECC)
Department of Revenue (DOR) Westfield State University (WSU)
Massachusetts Department of
Transportation (MassDOT)
Group Insurance Commission (GIC)
Massachusetts Parole Board (MPB)
Registry of Motor Vehicles (RMV)
State 911 Department (911)
The purpose of our audit was to determine whether EOTSS and the above executive branch agencies,
state colleges and universities, and regional transit authorities ensured that their employees completed
cybersecurity awareness training in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information
Security Risk Management Standard IS.010.
Below is a summary of our findings, the effects of those finds, and our recommendations, with links to
each page listed.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
2
Finding 1
Page 16
EOTSS did not ensure that all of its employees completed cybersecurity awareness training.
Effect If EOTSS does not ensure that all of its employees complete cybersecurity awareness
training, then EOTSS may expose itself to an increased risk of cybersecurity attacks and
financial and/or reputational losses.
Recommendations
Page 17
1. EOTSS should strengthen their policy to improve oversight of executive branch state
agencies, including their timely completion of cybersecurity awareness trainings.
2. EOTSS should ensure that all employee training transcripts for all employees are
maintained and include records regarding cybersecurity awareness training
completion.
3. EOTSS should ensure that all of its employees complete cybersecurity awareness
training within 30 days of orientation and annually thereafter.
4. EOTSS should establish procedures to monitor employee cybersecurity awareness
training completion rates throughout the training cycle and use historical data retained
by HRD to ensure that employees meet training deadlines.
Finding 2
Page 18
CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure that all of their
employees completed cybersecurity awareness training.
Effect If executive branch agencies do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendation
Page 22
The aforementioned nine executive branch agencies should do the following:
1. provide cybersecurity awareness training (both an initial training within 30 days of
orientation and an annual refresher training thereafter) to all full-time employees,
contractors, and interns;
2. establish procedures to monitor employee cybersecurity awareness training
completion rates throughout the training cycle and use historical data retained by HRD
to ensure employees meet training deadlines; and
3. implement additional controls to ensure that the new hire onboarding process includes
all relevant coursework regarding cybersecurity awareness training.
Finding 3
Page 26
Seven state colleges and universities did not ensure that all of their employees completed
cybersecurity awareness training.
Effect If state colleges and universities do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendations
Page 29
1. The aforementioned seven state colleges and universities should update their
cybersecurity awareness training policies to require this training for all employees.
2. The aforementioned seven state colleges and universities should update their
cybersecurity awareness training policies to include consequences for non-completion
(e.g., restriction of access until they complete the training).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
3
Finding 4
Page 32
CATA, CCRTA, and VTA did not ensure that all of their employees completed cybersecurity
awareness training.
Effect If regional transit authorities do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendations
Page 34
The aforementioned three regional transit authorities should do the following:
1. update their cybersecurity awareness training policies to require this training for all
employees and
2. update their cybersecurity training policies to include consequences for noncompletion (e.g., restriction of access until training is completed).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
4
OVERVIEW OF AUDITED ENTITY
The Executive Office of Technology Services and Security (EOTSS), located at 1 Ashburton Place in Boston,
was established in 2017 in accordance with Section 2 of Chapter 7D of the Massachusetts General Laws.
According to its website, EOTSS was created to “improve data security, safeguard privacy, and promote
better service delivery across the Commonwealth.” EOTSS operates under the direction of the
Commonwealth’s chief information officer, who is appointed by the Governor.
According to its website,
The Executive Office of Technology Services and Security (EOTSS) seeks to provide secure and
quality digital information, services, and tools to customers and constituents when and where they
need them. . . . EOTSS provides responsive digital and security services that enable taxpayers,
motorists, businesses, visitors, families, and other citizens to do business with the
Commonwealth. . . . EOTSS also oversees and manages the enterprise technology and digital
infrastructure and services for over 125 state agencies and over 43,000 state employees. . . . Since
its creation, EOTSS has made critical investments in infrastructure resiliency, unifying cybersecurity
operations, and deploying a Standard Operating Environment (SOE) and technology architecture
across all agencies. The organization has also collaborated with agencies to improve the centralized
delivery of digital services for constituents, schools, businesses, government agencies, and
municipalities.
According to its website, EOTSS employed 452 full-time employees as of May 24, 2023.
Multi-Agency Approach
This report covers 22 additional agencies’ compliance with EOTSS’s cybersecurity awareness training
standard. We separated them out into three categories (other executive branch agencies in addition to
EOTSS, state colleges and universities, and regional transit authorities) for the purposes of this report.
The organization chart below shows the applicability of EOTSS guidance for the agencies in this report.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
5
Applicability of Information Security Risk Management Standard IS.0101
EOTSS and Other Executive Branch Agencies
EOTSS is responsible for the development and maintenance of the Enterprise Information Security Policies
and Standards, pursuant to Section 2 of Chapter 7D of the General Laws, which requires all executive
branch agencies to “adhere to the policies, procedures, and objectives established by the executive office
1. Agencies marked as “not under audit” are not included in this report. Additionally, EOTSS’s Information Security Risk
Management Standard IS.010 states the following regarding its scope: “Executive Department agencies and offices are
required to implement procedures that ensure their personnel comply with the requirements herein to safeguard
information.”
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
6
of technology services and security.” EOTSS states in its Information Security Risk Management Standard
IS.010 that this standard “applies to the Executive Department including all executive offices, and all
boards, commissions, agencies, departments, divisions, councils, and bureaus.” This report outlines our
audit of the following executive branch agencies regarding cybersecurity awareness training:
 EOTSS itself;
 the Bureau of the State House (BSH);
 the Civil Service Commission (CSC);
 the Department of Labor Standards (DLS);
 the Department of Mental Health (DMH);
 the Department of Public Health (DPH);
 the Department of Revenue (DOR);
 the Group Insurance Commission (GIC);
 the Massachusetts Department of Transportation (MassDOT);
 the Massachusetts Parole Board (MPB);
 the Registry of Motor Vehicles (RMV); and
 the State 911 Department (911).
The table below shows the state appropriations for each of these executive branch agencies.2 (Note that
911 does not receive state appropriations. Instead, it receives funding through an annual surcharge of
$1.50 on all telephone lines capable of accessing the 911 system. These funds are kept by 911 in a trust
fund account.)
2. This table shows state appropriations exclusively; however, some agencies receive additional funding from other sources.
State appropriations include a variety of different spending categories, including personnel, technology, and pass-through
spending. As an example, GIC (line item 1108-5100) received $4,385,239, $4,385,240, and $4,738,587 in state appropriations
in fiscal years 2021, 2022, and 2023, respectively. GIC’s state appropriations include group insurance premium and plan costs
(line item 1108-5200), which accounted for $1,747,367,959, $1,826,778,807, and $1,921,206,747 in state appropriations in
fiscal years 2021, 2022, and 2023, respectively. GIC’s state appropriations also include the State Retiree Benefits Trust Fund
(line item 1599-6152), which accounted for $500,000,000 in state appropriations in fiscal years 2021 and 2022 and
$525,000,000 in state appropriations in fiscal year 2023. See the GIC’s Historical Budget Summary for more information.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
7
Agency State Appropriations
Fiscal Year 2021
State Appropriations
Fiscal Year 2022
State Appropriations
Fiscal Year 2023
EOTSS $3,105,778 $3,105,778 $3,204,513
BSH $3,677,814 $3,927,814 $4,569,197
CSC $623,938 $625,406 $843,762
DLS $3,949,551 $4,349,551 $4,628,025
DMH $911,642,258 $951,956,760 $1,018,768,861
DPH $769,034,718 $819,954,348 $938,273,734
DOR $1,356,399,209 $1,399,872,660 $1,483,244,288
MassDOT $613,006,824 $635,459,988 $752,237,634
GIC $2,263,612,328 $2,344,120,760 $2,463,402,384
MPB $21,908,514 $20,943,687 $21,649,317
RMV $182,380,000 $131,573,000 $131,653,000
State Colleges and Universities
The state colleges and universities in Massachusetts work to improve higher education, support economic
development and growth, and support communities across the Commonwealth. The following state
colleges and universities (which were established in accordance with Section 5 of Chapter 15A of the
General Laws) are a system of public institutions of higher education, and were subjects of this audit:
 Framingham State University (FSU);
 Holyoke Community College (HCC);
 Massachusetts Bay Community College (MBCC);
 Massasoit Community College (MCC);
 North Shore Community College (NSCC);
 Northern Essex Community College (NECC); and
 Westfield State University (WSU).
The table below shows the state appropriations for each of these state colleges and universities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
8
Agency State Appropriations
Fiscal Year 2021
State Appropriations
Fiscal Year 2022
State Appropriations
Fiscal Year 2023
FSU $32,545,150 $33,193,587 $36,087,625
HCC $22,697,040 $23,207,079 $23,851,448
MBCC $17,779,141 $18,136,472 $18,746,043
MCC $24,064,288 $24,474,243 $25,391,675
NSCC $24,154,641 $24,600,186 $25,517,333
NECC $21,986,040 $22,385,471 $23,251,578
WSU $30,992,952 $31,621,476 $34,336,799
Regional Transit Authorities
Regional transit authorities provide public transportation services in different communities within
Massachusetts, meeting the specific transit needs of each community. The following regional transit
authorities were established in accordance with Section 2 of Chapter 161B of the General Laws and were
subjects of this audit:
 the Cape Ann Transportation Authority (CATA);
 the Cape Cod Regional Transit Authority (CCRTA);
 the Martha’s Vineyard Regional Transit Authority (VTA); and
 the Nantucket Regional Transit Authority (NRTA).
The table below shows the operating revenues for each of these regional transit authorities.
Agency Operating Revenues
Fiscal Year 2021
Operating Revenues
Fiscal Year 2022
Operating Revenues
Fiscal Year 2023
CATA $13,642,963 $2,604,218 $512,110
CCRTA $9,083,000 $1,456,000 $1,139,000
VTA $1,289,000 $1,779,000 $1,798,000
NRTA $389,492 $578,464 $614,688
Cybersecurity Awareness Training
EOTSS has established policies and procedures that apply to all Commonwealth agencies within the
executive branch. These policies and procedures require executive branch agencies to implement
procedures that ensure that their employees comply with the requirements in EOTSS’s aforementioned
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
9
policies and procedures. EOTSS recommends, but does not require, non-executive branch agencies to
follow its policies and procedures. Section 6.2 of EOTSS’s Information Security Risk Management Standard
IS.010 states,
The objective of the Commonwealth information security training is to educate users on their
responsibility to help protect the confidentiality, availability and integrity of the Commonwealth’s
information assets. Commonwealth Offices and Agencies must ensure that all personnel are trained
on all relevant rules and regulations for cybersecurity.
To ensure that employees in all Commonwealth agencies within the executive branch are clear on their
responsibilities, EOTSS’s policies and procedures require that all newly hired employees3 must complete
an initial cybersecurity awareness training course within 30 days of their orientation, and that all existing
employees4 complete an annual refresher cybersecurity awareness course.
3. For the purposes of this audit report, we use the term newly hired employees to refer to employees who were hired during
the audit period, unless stated otherwise.
4. For the purposes of this audit report, we use the term existing employees to refer to employees who were hired before the
start of the audit period (July 1, 2021), unless stated otherwise.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
10
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of cybersecurity awareness training at the Executive Office of
Technology Services and Security (EOTSS). Pursuant to our governing statute, Section 12 of Chapter 11 of
the General Laws, our audit covers multiple entities’ compliance with EOTSS’s cybersecurity training
standards. Specifically, Section 12 of Chapter 11 states, “Each entity may be audited separately as a part
of a larger organizational entity or as a part of an audit covering multiple entities.” As such, cybersecurity
awareness training testing was completed at 22 other executive branch agencies, state colleges and
universities, and regional transit authorities, for the period July 1, 2021 through April 30, 2023.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objective.
Below is our audit objective, indicating the question we intended our audit to answer; the conclusion we
reached regarding our objective; and, if applicable, where our objective is discussed in the audit findings.
Objective Conclusion
1. Did EOTSS and other executive branch agencies, state colleges and universities, and
regional transit authorities ensure that their employees completed cybersecurity
awareness training in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information
Security Risk Management Standard IS.010?
No; see Findings
1, 2, 3, and 4
To accomplish our audit objective, we gained an understanding of the aspects of EOTSS’s internal control
environment relevant to our objective by interviewing EOTSS staff members and management and by
reviewing EOTSS’s Information Security Risk Management Standard IS.010.
To obtain sufficient, appropriate evidence to address our audit objective, we performed the procedures
described below.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
11
Cybersecurity Awareness Training
We separated the 23 agencies we reviewed as part of this audit into three categories based on agency
type: EOTSS and other executive branch agencies, state colleges or universities, and regional transit
authorities.
 The first category comprises EOTSS and 11 other executive branch agencies: the Bureau of the
State House (BSH), the Civil Service Commission (CSC), the Department of Labor Standards (DLS),
the Department of Mental Health (DMH), the Department of Public Health (DPH), the Department
of Revenue (DOR), the Massachusetts Department of Transportation (MassDOT), the Group
Insurance Commission (GIC), the Massachusetts Parole Board (MPB), the Registry of Motor
Vehicles (RMV), and the State 911 Department (911).
 The second category comprises seven state colleges and universities: Framingham State
University (FSU), Holyoke Community College (HCC), Massachusetts Bay Community College
(MBCC), Massasoit Community College (MCC), North Shore Community College (NSCC), Northern
Essex Community College (NECC), and Westfield State University (WSU).
 The third category comprises four regional transit authorities: the Cape Ann Transportation
Authority (CATA), the Cape Cod Regional Transit Authority (CCRTA), the Martha’s Vineyard
Regional Transit Authority (VTA), and the Nantucket Regional Transit Authority (NRTA).
To determine whether EOTSS and these other executive branch agencies, state colleges and universities,
and regional transit authorities ensured that their employees completed cybersecurity awareness training
in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information Security Risk Management Standard
IS.010, we took the actions described below.
EOTSS and Other Executive Branch Agencies
To determine whether EOTSS and the 11 other executive branch agencies included in this audit
ensured that their newly hired employees completed initial cybersecurity awareness training within
30 days of orientation, we analyzed the evidence for cybersecurity awareness training completion
(i.e., transcript reports5
) by comparing each employee’s start date and training completion date for
all 2,662 newly hired employees across these executive branch agencies.
To determine whether these executive branch agencies ensured that their existing employees
completed annual refresher cybersecurity awareness training, we analyzed the evidence for
cybersecurity awareness training completion (i.e., transcript reports) by comparing each employee’s
5. We analyzed the cybersecurity awareness training transcript reports from EOTSS and the other executive branch agencies.
These reports included fields such as the training due date and the training completion date.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
12
training completion date and training due date for all 12,236 existing employees across these
executive branch agencies.
To further substantiate the results of the above procedures, we also selected a random, statistical
sample6 of 24 employee training certificates of completion out of the population of 14,898 newly
hired and existing employees, using a 90% confidence level,7 a 0% expected error rate,8 and a 10%
tolerable error rate.9 Our sample comprised the following:
 from EOTSS, BSH, CSC, DLS, GIC, MPB, RMV, and 911: 1 employee training certificate of
completion from each agency;
 from DOR: 2 employee training certificates of completion;
 from DPH and MassDOT: 4 employee training certificates of completion from each agency;
and
 from DMH: 6 employee training certificates of completion.
We selected these sample numbers based on the number of active employees each agency had during
the audit period.
We did not note any exceptions in our testing corresponding to BSH and 911. Therefore, we concluded
that, during the audit period, BSH and 911 met the relevant criteria regarding this matter.
For the other executive branch agencies included in this audit, we did note exceptions during our
testing. See Findings 1 and 2 for issues we identified with the cybersecurity awareness training
provided by EOTSS and the other executive branch agencies included in this audit.
6. Auditors use statistical sampling to select items for audit testing when a population is large and contains similar items.
Auditors generally use a statistical software program to choose a random sample when sampling is used. The results of testing
using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections
about entire populations.
7. Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are
representative of the population (parameter), expressed as a percentage.
8. Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the
auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.
9. The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while
still using the sample to conclude that the results from the sample have achieved the objective.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
13
State Colleges and Universities
To determine whether the state colleges and universities included in this audit ensured that their
employees completed cybersecurity awareness training, we took the actions described below.
We inspected the cybersecurity awareness training certificates of completion using a judgmental,10
nonstatistical sample of 70 employee training certificates of completion out of the population of
10,094. Our sample comprised 10 employee training certificates of completion from each of the seven
state colleges and universities included in this audit. Of the 10 employee training certificates of
completion from each state college or university, we judgmentally selected 3 existing non-student
employees, 4 newly hired non-student employees, and 3 existing student employees.
Also, we determined whether the state colleges and universities included in this audit ensured that
the newly hired employees from our sample completed initial training within 30 days of orientation
by comparing the dates of their orientations to the dates of their certificates of completion.
See Finding 3 for issues we identified with the cybersecurity awareness training provided by the state
colleges and universities included in this audit.
Regional Transit Authorities
To determine whether the regional transit authorities included in this audit ensured that their
employees completed cybersecurity awareness training, we took the actions described below.
We inspected the cybersecurity awareness training certificates of completion using a judgmental,
nonstatistical sample of 23 employee training certificates of completion out of the population of 55.
Our sample comprised the following:
 from CATA: 3 employee training certificates of completion (which represents its full
population of employees);
 from NRTA: 4 employee training certificates of completion (which represents its full
population of employees); and
10. Auditors use judgmental sampling to select items for audit testing when a population is very small, the population items are
not similar enough, or there are specific items in the population that the auditors want to review. Auditors use their
knowledge and judgment to select the most appropriate sample. For example, an auditor might select items from areas of
high risk. The results of testing using judgmental sampling cannot be used to make conclusions or projections about entire
populations; however, they can be used to identify specific issues, risks, or weaknesses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
14
 from CCRTA and VTA: 8 employee training certificates of completion from each agency.
Of the 8 employee training certificates of completion from CCRTA and VTA, we judgmentally selected
2 newly hired employees and 6 existing employees. Additionally, we determined whether these
regional transit authorities ensured that the newly hired employees from our sample completed initial
training within 30 days of orientation by comparing the dates of their orientations to the dates of their
certificates of completion.
We did not note any exceptions in our testing corresponding to NRTA. Therefore, we concluded that,
during the audit period, NRTA met the relevant criteria regarding this matter.
For the other regional transit authorities included in this audit, we noted exceptions during our
testing. See Finding 4 for issues we identified with the cybersecurity awareness training provided by
the regional transit authorities included in this audit.
We used a combination of statistical and nonstatistical sampling methods for testing, and we did not
project the results of our testing to any corresponding populations.
Data Reliability Assessment
To determine the reliability of the employee lists from EOTSS and each of the 22 other executive branch
agencies, state colleges and universities, and regional transit authorities included in this audit (see the list
of auditees included in this report, by category), we took the actions described below.
We interviewed EOTSS management who were knowledgeable about these lists. We reviewed
MassAchieve11 system controls for access control, configuration management, contingency planning,
segregation of duties, and security management. We checked that the variable formats of each agency’s
employee list (e.g., dates, unique identifiers, or abbreviations) were accurate. For each agency’s employee
list, we ensured that there was no abbreviation of data fields, no missing data (e.g., hidden rows or
columns, blank cells, or incomplete records), and no duplicate records and that all values corresponded
with expected values.
To determine the completeness and accuracy of each agency’s employee list, we took the actions
described below.
11. MassAchieve is a training platform used by executive branch agencies to administer cybersecurity awareness training.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
15
EOTSS and Other Executive Branch Agencies
 EOTSS: We selected random samples of 20 employees from EOTSS’s employee list and traced
their names to CTHRU, the Commonwealth’s statewide payroll open records system. We also
selected random samples of 20 employees from CTHRU and traced their names back to
EOTSS’s employee list.
 BSH and CSC: We selected random samples of five employees from each executive branch
agency’s employee list and traced their names to CTHRU. We also selected random samples
of five employees from each agency from CTHRU and traced their names back to each
agency’s employee list.
 DLS, GIC, MPB, and 911: We selected random samples of 10 employees from each executive
branch agency’s employee list and traced their names to CTHRU. We also selected random
samples of 10 employees from each agency from CTHRU and traced their names back to each
agency’s employee list.
 DMH, DPH, DOR, MassDOT, and RMV: We selected random samples of 20 employees from
each executive branch agency’s employee list and traced their names to CTHRU. We also
selected random samples of 20 employees from each agency from CTHRU and traced their
names back to each agency’s employee list.
State Colleges and Universities
 FSU, HCC, MBCC, MCC, NSCC, NECC, and WSU: We selected random samples of 20 employees
from each state college’s/university’s employee list and traced their names to CTHRU. We
also selected random samples of 20 employees from each state college/university from
CTHRU and traced their names back to each state college/university’s employee list.
Regional Transit Authorities
 CATA: We selected the total population of three employees and traced their names to CATA’s
open payroll webpage. We also selected the total population of three employees from CATA’s
open payroll webpage and traced their names back to CATA’s employee list.
 CCRTA and VTA: We selected random samples of five employees from each regional transit
authority’s employee list and traced their names to each agency’s open payroll webpage. We
also selected random samples of five employees from each regional transit authority’s open
payroll webpage and traced their names back to each agency’s employee list.
 NRTA: We selected the total population of four employees and traced their names to NRTA’s
open payroll webpage. We also selected the total population of four employees from NRTA’s
open payroll webpage and traced their names back to NRTA’s employee list.
Based on the results of the data reliability assessment procedures described above, we determined that
the information we obtained for the audit period was sufficiently reliable for the purposes of our audit.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
16
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. EOTSS did not ensure that all of its employees completed cybersecurity
awareness training.
The Executive Office of Technology Services and Security (EOTSS) did not ensure that all of its employees
who were active during the audit period completed initial and annual refresher cybersecurity awareness
training.
The original due date for the training was August 31, 2022, but EOTSS executive management requested
and received an extension from the Human Resources Division (HRD), which extended the due date for
all executive branch agencies to October 14, 2022. HRD communicated this new deadline to executive
branch managers through its Managers’ Corner Newsletter.
The table below shows our findings for EOTSS. Note that this table reflects the extended October 14, 2022
due date.
Cybersecurity
Awareness
Training Type
On-Time Training
Completion
Percentage
Total Number
of Employees
Tested
Number of Employees
Who Completed
Training Late
Number of Employees
Who Did Not Complete
Training
Initial 67.8% 115 28 9
Annual Refresher 99.8% 411 — 1
If EOTSS does not ensure that all of its employees complete cybersecurity awareness training, then EOTSS
may expose itself to an increased risk of cybersecurity attacks and financial and/or reputational losses.
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
17
Reasons for Issue
EOTSS management explained that contract employees undergo a different onboarding process
compared to non-contact employees. EOTSS processes contract employees’ training assignments in
batches and must create training accounts manually. This process is time-consuming and typically occurs
only once or twice per month. Additionally, EOTSS management noted that they do not have access to
training transcripts for former employees.
Recommendations
EOTSS should strengthen their policy to improve oversight of executive branch state agencies,
including their timely completion of cybersecurity awareness trainings.
EOTSS should ensure that all employee training transcript for all employees are maintained and
include records regarding cybersecurity awareness training completion.
EOTSS should ensure that all of its employees complete cybersecurity awareness training within 30
days of orientation and annually thereafter.
EOTSS should establish procedures to monitor employee cybersecurity awareness training completion
rates throughout the training cycle and use historical data retained by HRD to ensure that employees
meet training deadlines.
Auditee’s Response
Security awareness training is a critical component of the Commonwealth’s security compliance
strategy. Mandatory cybersecurity training must be completed within 30 days of employee
orientation. The new hire 30-day training completion requirement is tied to employee orientation,
rather than date of hire, to accommodate business processes related to onboarding and
credentialing into the training system. Further, the process for onboarding and credentialing
contract employees is different than the process for non-contract employees. Contractors are
assigned training in “batches” once or twice per month. The new hire 30-day training completion
requirement is purposefully tied to orientation date, as opposed to new hire date to accommodate
for such business processes. [The Office of the State Auditor] relied on hire date, rather than
employee orientation/onboarding date to calculate the 30-day deadline.
Moving forward, EOTSS will evaluate its internal processes to identify areas for improvement
related to new hire orientation and contractor onboarding.
Additionally, EOTSS will work with necessary partners to explore whether there is a technical
solution to accessing transcript data of former agency employees.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
18
Auditor’s Reply
We agree with EOTSS’s statement that “security awareness training is a critical component of the
Commonwealth’s security compliance strategy,” and for this reason, we believe that all employees,
regardless of classification, should complete their initial training within 30 days. The data provided by
EOTSS in response to our data requests in this audit did not include new hire orientation dates, it included
new hire start dates.
Additionally, while we acknowledge that EOTSS has established policies and procedures applicable to all
Commonwealth agencies within the executive branch, based on the findings below respective to those
executive branch agencies, we believe there is a need for EOTSS to enhance its oversight of these agencies
to ensure greater compliance with the Enterprise Information Security Policies and Standards.12
Based on its response, EOTSS has indicated that it will take steps to address our concerns on this matter.
We will follow up on this during our post-audit review process in approximately six months.
2. CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure
that all of their employees completed cybersecurity awareness training.
The following executive branch agencies did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: the Civil Service Commission (CSC), the
Department of Labor Standards (DLS), the Department of Mental Health (DMH), the Department of Public
Health (DPH), the Department of Revenue (DOR), the Massachusetts Department of Transportation
(MassDOT), the Group Insurance Commission (GIC), the Massachusetts Parole Board (MPB), and the
Registry of Motor Vehicles (RMV).
Regarding the completion rates for the initial cybersecurity awareness training, we observed that 445
newly hired employees completed training late, while 601 did not complete training at all. Regarding the
completion rates for the annual refresher cybersecurity awareness training, we observed that 156 existing
employees completed training late, while 951 did not complete training at all.
12. The Enterprise Information Security Policies and Standards is the compilation of policies and standards that all executive
branch agencies are required to follow. Information Security Risk Management Standard IS.010 is just one of these policies.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
19
The table and graph below show our findings for these agencies regarding initial cybersecurity awareness
training.
On-Time Cybersecurity Awareness Training Completion Rates for Executive
Branch Agencies: Newly Hired Employees
0.0%
54.3%
66.4%
97.8%
44.1%
66.7%
76.2%
67.8%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CSC DMH DPH DOR MassDOT GIC MPB RMV
Agency On-Time Initial
Training Completion
Percentage
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training Late
Number of Tested
Employees Who Did Not
Complete Training
CSC 00.0% 1 — 1
DMH 54.3% 905 148 266
DPH 66.4% 524 83 93
DOR 97.8% 229 — 5
MassDOT 44.1% 742 185 230
GIC 66.7% 3 1 —
MPB 76.2% 21 5 —
RMV 67.8% 90 23 6
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
20
The table and graph below show our findings for these agencies regarding annual refresher cybersecurity
awareness training.
On-Time Cybersecurity Awareness Training Completion Rates for Executive
Branch Agencies: Existing Employees
70.0%
98.3%
90.9% 86.9%
99.5%
89.2%
99.3%
91.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CSC DLS DMH DPH DOR MassDOT MPB RMV
Agency On-Time Annual
Refresher Training
Completion Percentage
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training Late
Number of Tested
Employees Who Did
Not Complete Training
CSC 70.0% 10 — 3
DLS 98.3% 58 1 —
DMH 90.9% 3246 30 265
DPH 86.9% 2911 26 355
DOR 99.5% 1356 — 7
MassDOT 89.2% 3455 91 284
MPB 99.3% 151 — 1
RMV 91.0% 488 8 36
If executive branch agencies do not ensure that all of their employees complete cybersecurity awareness
training, then they may expose themselves to an increased risk of cybersecurity attacks and financial
and/or reputational losses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
21
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following executive branch agencies provided us with the following
reasons for noncompliance:
 CSC management stated that contracted attorneys and interns were not on the list of employees
required to complete the cybersecurity awareness training.
 DLS management stated that they are not sure what the reason was for the late training
completion of the employee from our finding, other than the employee overlooked the training
due date. DLS management noted that this employee is no longer with DLS.
 DMH management sent us an email on February 16, 2024 regarding the employees from our
finding, stating that these are “Employees who do not have [computer] Network Access—these
staff are exempt.”
 DPH management stated that the staff members from our finding started their cybersecurity
awareness training but did not complete the full training.
 DOR management stated that some of the employees from our finding had job duties that did not
require them to have computer network access, while others separated from DOR shortly after
their training due date had passed, leaving no time for DOR to enforce training completion.
 MassDOT management and RMV management stated that employees missed the training
deadline and that interns did not receive cybersecurity awareness training because MassAchieve
did not assign them training.
 GIC management stated that the employee from our finding left the agency shortly after starting
and did not complete the training before their departure.
 MPB management stated that newly hired employees have assigned joint orientation/training
days which may have been scheduled past the 30 days from hire dates for some staff. Regarding
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
22
refresher training, it appears that one employee found not to have completed the training,
completed only 2 out of the 5 required sections of the cybersecurity training.
Recommendations
The aforementioned nine executive branch agencies should do the following:
provide cybersecurity awareness training (both an initial training within 30 days of orientation and an
annual refresher training thereafter) to all full-time employees, contractors, and interns;
establish procedures to monitor employee cybersecurity awareness training completion rates
throughout the training cycle and use historical data retained by HRD to ensure employees meet
training deadlines; and
implement additional controls to ensure that the new hire onboarding process includes all required
coursework regarding cybersecurity awareness training.
We appreciate the following responses provided by the executive branch agencies:
Auditee’s Response: CSC
CSC appreciates receiving clarification from the Office of the State Auditor that seasonal interns
and contract employees are required to complete the cybersecurity awareness training. In
response, CSC had the seasonal interns and contract employee at the time immediately complete
the required cybersecurity training. Going forward, any CSC interns and contract employees will be
required to complete the same initial and refresher cybersecurity training as all regular CSC
employees, ensuring 100% compliance with this requirement.
Auditor’s Reply: CSC
Based on its response, CSC has taken measures to address our concerns regarding this matter.
Auditee’s Response: DLS
[DLS] management agrees with the finding. As [the Office of the State Auditor] has affirmed, we
now have a program to ensure employees are trained in a timely manner. This is demonstrated by
the 100% completion for new hires and near completion for existing employees. Of the two existing
employees whose training was not completed by the deadline, one staff was one (1) day late due
to her supervisor leaving and the new supervisor not receiving the alerts, while the other staff is
no longer an [Executive Office of Labor and Workforce Development] employee. Regardless, we
will continue to reinforce timely completion by sending email reminders.
Auditor’s Reply: DLS
Based on its response, DLS will take measures to address our concerns regarding this matter.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
23
Auditee’s Response: DMH
As DMH indicated to the [Office of the State Auditor] during the audit, 417 of the newly hired
individuals are contract employees who do not have any network access. Consequently, they do
not need cybersecurity training. In fact, providing this training would unnecessarily expend
resources and increase security risk, as DMH would need to create network access solely to provide
the training.
DMH recognizes that the [Office of the State Auditor] assesses compliance with the policy or
standard as written, and that it reads Section 6.2 of EOTSS’s Information Security Risk Management
Standards as requiring cybersecurity training for “all personnel.” Indeed, Section 6.2 states that
“all personnel” must be trained. The immediately preceding sentence, however, states that the
objective of the cybersecurity training is to educate “users” on their cybersecurity responsibilities.
Respectfully, DMH views the word “personnel” in the second sentence as referring to the “users”
referred to in the first sentence. Thus, per DMH’s reading, only “users” must be trained. . . .
The data used for this finding had some limitations, as indicated during the audit. Some employees
were hired after the end date that the 2021 annual cybersecurity training was due; some left state
service and then returned after the due date for the 2021 cybersecurity training; and, on account
of system limitations, DMH was unable to determine dates that some staff left DMH. DMH
understands that data of the sort required and assessed here typically has limitations, and that the
Auditor’s Office needs to utilize data as provided, but the number here likely is not accurate.
Auditor’s Reply: DMH
Section 2 of Chapter 7D of the Massachusetts General Laws mandates that all executive branch state
agencies, including DMH, “adhere to the policies, procedures and objectives established by the executive
office of technology services and security.” DMH must ensure that contractors are trained in compliance
with EOTSS’s Information Security Risk Management Standard IS.010.
Regarding the definition of “personnel,” we maintain that EOTSS’s Information Security Risk Management
Standard IS.010 states, “All new personnel must complete an Initial Security Awareness Training course,”
and that EOTSS does not provide an exemption to this policy for employees who lack access to computers.
We urge DMH to implement an alternative method for employees without system access to complete
their training, such as offering a paper-based training option. We recognize that some agencies may
disagree with EOTSS standards, but nonetheless, these standards exist. Cybersecurity awareness policies
are not just guidelines; they are essential safeguards in today’s digital landscape. Comprehensive
employee training and shared responsibility are critical to mitigating potential cyber threats. It is
important to consistently assess and reinforce cybersecurity measures to ensure that policies are
effective, compliance is maintained, and public trust in the agency’s ability to safely manage data is
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
24
upheld. These policies exist to protect both individuals and organization, fostering a secure and safe digital
environment.
Regarding the data’s limitations, we conducted a data reliability assessment on the information DMH
provided to us, ensuring the completeness and accuracy of DMH’s employee list. As we have
recommended, we believe that DMH should establish procedures to (1) monitor employee cybersecurity
awareness training completion rates throughout the training cycle, (2) accurately track the dates when
employees leave the agency, and (3) use historical data retained by HRD to ensure that employees meet
training deadlines.
Auditee’s Response: DPH
1. Provide cybersecurity awareness training (both an initial training within 30 days of orientation
and an annual refresher training thereafter) to all full-time employees, contractors, and interns.
a. The training is offered through MassAchieve within 30 days of start and annually.
b. DPH has increased staffing in this area and developed and implemented a robust system
of reminders for all staff who are incompliant starting in December of each year.
c. We promote completion of this training by alerting staff to the consequence of shut-off by
EOTSS.
d. This past fiscal year we achieved near perfect completion with less than 10 shut offs.
2. Establish procedures to monitor employee completion throughout the training cycle to ensure
that staff are meeting the training deadlines.
a. Our staff run reports monthly and have empowered each bureau, office and hospital to
run their own custom-built reports.
b. We established standard communications to go out to supervisors and incompliant staff.
We appreciate the insights provided by the audit and are addressing these findings promptly.
Auditor’s Reply: DPH
Based on its response, DPH has taken measures to address our concerns regarding this matter.
Auditee’s Response: DOR
DOR agrees with the results of the audit. The employees who did not complete the training during
the audit period were employees with no access to computers or were separated from DOR shortly
after hire.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
25
DOR will continue to utilize MassAchieve to track employee completion throughout the training
cycle.
In [fiscal year 2024], DOR implemented the process of “paper training,” where Employees with no
access to computers and/or systems will take the training in person, in a class organized by their
managers, and sign an acknowledgement that they have received, taken and understand the
training. Information will be uploaded to MassAchieve.
DOR will incorporate cybersecurity awareness training into the new hire process, where the course
is added to DOR’s Learning Management System (LMS—DOR’s internal training system). LMS
system also will be used to track completion and follow up with new hires that have not completed
the training. Information will be uploaded to MassAchieve.
Auditor’s Reply: DOR
Based on its response, DOR has taken, and will continue to take, measures to address our concerns
regarding this matter.
Auditee’s Response: MassDOT and RMV
As of 2024, MassDOT has transitioned to using only the MassAchieve LMS, eliminating confusion
for employees regarding where to find and complete assigned training. Furthermore, statewide
improvements, such as increased frequency of reminders from HRD, have helped improve
performance. Additionally, EOTSS has followed through on removing access to those who do not
complete cybersecurity training on time. MassDOT has used this consequence to effect in our
messaging to further incentivize timely completion of cybersecurity training and has collaborated
with EOTSS as needed to reinstate access for individuals who had their access removed due to
non-compliance. . . .
In the 2023–24 training cycle MassDOT implemented procedures to continue to support the
agency’s efforts in meeting its compliance obligation. This includes earlier distribution of targeted
activity reports, making it easier for managers to identify those yet to complete training. Reports
are shared on an increasing cadence as the training deadline approaches.
Auditor’s Reply: MassDOT and RMV
Based on their response, MassDOT and RMV have taken measures to address our concerns regarding this
matter.
Auditee Response: GIC
GIC was given the opportunity to respond to a draft version of this audit report and did not provide a
written response.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
26
Auditee’s Response: MPB
MPB concurs with [the Office of the State Auditor’s] recommendations to (1) provide cybersecurity
awareness training (both an initial training within 30 days of orientation and an annual refresher
training thereafter) to all full-time employees, contractors, and interns; and (2) establish
procedures to monitor employee completion throughout the training cycle to ensure that staff are
meeting the training deadlines.
To improve timely completion of cybersecurity training for new hires, MPB will modify its existing
“Checklist for Employee Orientation” form to specify due dates for completion of cybersecurity
training and include an acknowledgement receipt upon completion.
Bi-weekly Managers’ Meetings will be utilized to further monitor adherence to the training
deadlines.
Auditor’s Reply: MPB
Based on its response, MPB will take measures to address our concerns regarding this matter.
3. Seven state colleges and universities did not ensure that all of their
employees completed cybersecurity awareness training.
The following state colleges and universities did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: Framingham State University (FSU), Holyoke
Community College (HCC), Massachusetts Bay Community College (MBCC), Massasoit Community College
(MCC), North Shore Community College (NSCC), Northern Essex Community College (NECC), and Westfield
State University (WSU).
The table and graph below show our findings for these state colleges and universities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
27
On-Time Cybersecurity Awareness Training Completion Rates for State
Colleges and Universities: Sample of All Employees
40.0%
60.0%
50.0% 50.0%
80.0%
30.0%
70.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
FSU HCC MBCC MCC NSCC NECC WSU
State College
or University
On-Time Training
Completion
Percentage*
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training
Late
Number of Tested
Employees Who Did
Not Complete Training
FSU 40.0% 10 — 6
HCC 60.0% 10 — 4
MBCC 50.0% 10 — 5
MCC 50.0% 10 — 5
NSCC 80.0% 10 — 2
NECC 30.0% 10 — 7
WSU 70.0% 10 — 3
* Note that this table is based on the sample of employees from each state college or university, not the population of
employees.
If state colleges and universities do not ensure that all of their employees complete cybersecurity
awareness training, then they may expose themselves to an increased risk of cybersecurity attacks and
financial and/or reputational losses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
28
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following state colleges and universities provided us with the following
reasons for noncompliance:
 FSU management stated that its internal policy only recommended cybersecurity awareness
training for its employees, instead of requiring it.
 HCC management stated that student employees did not have access to HCC’s computer network
(which is only accessible with staff member accounts), so therefore, providing them with
cybersecurity awareness training would not be required.
 MBCC management stated the following:
 Two student employees from our finding “never received student [employee] accounts, so
they were missed in getting training assigned as part of the onboarding” (from an email MBCC
sent to us on February 15, 2024);
 Two employees from our finding “did not elect to complete their training. . . . As a result, their
employment with [MBCC] was discontinued” (from an email MBCC sent to us on February 15,
2024);
 One newly hired employee from our finding “started before the training program was in place,
so [they] would not have had the option for [initial] training” (from an email MBCC sent to us
on February 15, 2024); and
 One newly hired employee from our finding joined MBCC while the college was conducting
annual refresher cybersecurity awareness training, so MBCC enrolled this employee in the
annual refresher training rather than being trained on the same content twice in a short
period of time by first being enrolled in the initial cybersecurity awareness training.
 MCC management stated that its internal policy only recommended cybersecurity awareness
training for its employees, instead of requiring it.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
29
 NSCC management stated that two newly hired employees did not complete the cybersecurity
awareness training because its auto-enrollment process failed briefly in September 2022, leading
to NSCC’s inability to enroll newly hired employees into the training during this period.
 NECC management stated that it has a written cybersecurity awareness training policy, but that
the policy is not enforced. Management also stated that they are not allowed to limit user access
for employees who do not complete cybersecurity awareness training.
 WSU management stated that they were not aware that contractors, part-time employees, or
seasonal employees were required to complete the cybersecurity awareness training.
Recommendations
The aforementioned seven state colleges and universities should update their cybersecurity
awareness training policies to require this training for all employees.
The aforementioned seven state colleges and universities should update their cybersecurity
awareness training policies to include consequences for non-completion (e.g., restriction of access
until they complete the training).
Auditees’ Responses
FSU
We are in agreement with the merits of the [EOTSS] Standard and the University is now
aligned with the goals of the cybersecurity awareness training. To that end, since the
completion of the field work associated with this audit, but prior to the receipt of this draft
report, FSU developed and formally adopted campus policy consistent with the Information
Security Risk Management Standard. Appendix A contains the text of this Policy on
Cybersecurity Training for Employees established on July 17, 2024. The policy is currently
in effect and will begin full implementation in October 2024 pursuant to the establishment
of a bargained labor agreement that permits initial onboarding cybersecurity training and
then subsequent annual training, including prescriptive penalties or remediations for
noncompliance.
This local policy will achieve the same goals and mitigate the risks identified in the
recommendations associated with Finding 3. We remain committed to the protection of
the information technology assets and information retained by the University and share
the mutual desire to remain vigilant to new and emerging threats to these digital assets
and networks.
HCC
Upon learning that all HCC work study students, regardless of their need to access the
network, must complete the cybersecurity training within 30 days of their assignment, HCC
implemented the following policies and procedures:
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
30
Policy: HCC’s policy now mandates that all work study students will be notified they need
to complete mandatory cybersecurity training within 30 days of starting their work
assignment.
Consequences: Failure to complete the required training within 30 days of their work
assignment will result in revoking their work study assignment/job until the training is
completed.
MBCC
[MBCC writes] in response to your email of July 19, 2024, regarding the recent audit of
cybersecurity training at [MBCC]. Thank you for sharing the audit results and providing us
with the opportunity to respond.
The two student employees mentioned did not receive student employee accounts and
thus were not assigned training during onboarding. As part of our employee onboarding
process, all MBCC employees receive an account and are enrolled in the new hire
cybersecurity training program. This issue was identified in November 2023 due to this
audit, and since then, MBCC has taken steps to ensure the enforcement of this process.
Two employees chose not to complete their training, leading to the termination of their
employment with MBCC, underscoring the institution’s commitment to mandatory training.
One employee joined before the training program was established. The program is now
fully operational, requiring all employees to complete it within 30 days of starting. If not,
they are granted an additional 30 days then this [is] escalated to senior management and
their access is restricted until it is completed.
Lastly, one newly hired employee started with MBCC during the annual cybersecurity
awareness training. As the on boarding training is identical, they were not enrolled twice.
Going forward we will ensure that they are enrolled in both.
Thank you again for the audit. Our policy states that all employees must complete the
cybersecurity training, but this audit helped us identify areas for improvement. We have
taken the necessary steps to remediate areas of concern. Going forward we anticipate we
will be in full compliance with the State requirements.
MCC
The college fully acknowledges the need for, and importance of, cybersecurity training for
all employees.
Massasoit Community College’s leadership is currently developing language to amend the
existing Written Information Security Program (WISP) with the recommendations of the
recent Executive Office of Technology Services and Security performance audit.
The college will be collaborating with the Unions, through impact bargaining, to ensure
proper checks and balances are in place, that new hire training and annual re-training are
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
31
conducted in a timely manner, and that, if necessary, reasonable gradated consequences
for non-compliance are in place.
NSCC
The College agrees that cybersecurity training is critical and important. The College
management and especially the [information technology] department has put a great deal
of effort into a collaborative process ensuring that cybersecurity training is ongoing and
annual, as demonstrated in our highest completion rate (80%) of those tested in the [Office
of the State Auditor] draft report. Since that audit the College has gone further with tighter
process improvements which now disables employee accounts that have not completed
either the new employee training or annual training within the allotted time frames.
Disabled accounts are reenabled upon request and employees are granted an additional
week to complete the required training. Our training completion rate now stands at 97%.
NECC
At NECC we specifically value and understand the importance of Cybersecurity training.
Recently we experienced a cyber incident caused by user error. Had it not been for the
systems we have in place; this threat would have had significant impact on our operation.
We also worked with EOTSS after the incident to discuss lessons learned from the attack,
working with vendors and the Commonwealth.
In order to better comply with EOTSS’s Information Security Risk Management Standard
IS.010, and industry’s best practices, we have developed a revised Cybersecurity Training
Process. . . . NECC is implementing the process starting in the Fall. This process may be
subject to impact bargaining with our [Massachusetts Community College Council] and
[American Federation of State, County and Municipal Employees] union members.
Thank you again for the opportunity to respond to this audit and please do not hesitate to
contact me should you have any additional questions.
WSU
Westfield State’s current Security Education Training and Awareness (SETA Program)
already requires training as part of the campus onboarding program. . . . For the faculty
collective bargaining unit, [Massachusetts State College Association], training was impact
bargained and the final agreement was completed on March 21, 2024. As a result,
beginning in the fall 2024, cyber security training will be required for faculty. . . .
The University’s Access Control Guidelines already allows for the suspension of access to
information technology resources for non-compliance. Efforts are currently underway to
formalize the consequences with Office of Information and Instructional Technology and
the Human Resources Office. Progressive discipline actions may require further impact
bargaining.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
32
Auditor’s Reply
We appreciate the responses provided by the seven state colleges and universities we audited. The issue
we identified is that these state colleges and universities did not consistently provide cybersecurity
training to their employees. We regard EOTSS’s Information Security Risk Management Standard IS.010
as the baseline for best practices in cybersecurity awareness training across the Commonwealth’s
agencies, and therefore, we used this as our audit criteria. According to Section 8.18 of the US
Government Accountability Office’s Generally Accepted Government Auditing Standards, “Examples of
criteria include: . . . (c) technically developed standards or norms; . . . (f) defined business practices; . . .
and (h) benchmarks against which performance is compared, including performance of other entities or
sectors.”
As noted above within the auditees’ responses, many colleges and universities have already started
addressing our concerns in this area.
4. CATA, CCRTA, and VTA did not ensure that all of their employees completed
cybersecurity awareness training.
The following regional transit authorities did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: the Cape Ann Transportation Authority (CATA),
the Cape Cod Regional Transit Authority (CCRTA), and the Martha’s Vineyard Regional Transit Authority
(VTA).
The table and graph below show our findings for these regional transit authorities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
33
On-Time Cybersecurity Awareness Training Completion Rates for Regional
Transit Authorities: Sample of All Employees
66.7%
25.0%
50.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CATA CCRTA VTA
Regional
Transit
Authority
On-Time Training
Completion
Percentage*
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training
Late
Number of Tested
Employees Who Did
Not Complete Training
CATA 66.7% 3 — 1
CCRTA 25.0% 8 — 6
VTA 50.0% 8 — 4
* Note that this table is based on the sample of employees from each regional transit authority, not the population of
employees.
If regional transit authorities do not ensure that all of their employees complete cybersecurity awareness
training, then they may expose themselves to an increased risk of cybersecurity attacks and financial
and/or reputational losses.
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
34
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following regional transit authorities provided us with the following
reasons for noncompliance:
 CATA management stated that the employee from our finding overlooked the email reminders
for the cybersecurity awareness training and did not know they could complete training after the
due date.
 CCRTA management stated that not all employees participated in the cybersecurity awareness
training, as it was given only to staff members with access to sensitive customer or agency data.
 VTA management stated that some employees did not have computer network access, and
therefore, VTA did not require them to take cybersecurity awareness training.
Recommendations
The aforementioned three regional transit authorities should do the following:
update their cybersecurity awareness training policies to require this training for all employees and
update their cybersecurity training policies to include consequences for non-completion (e.g.,
restriction of access until training is completed).
Auditees’ Responses
CATA
The Cape Ann Transportation Authority agrees with the recommendations.
CCRTA
The [Office of the State Auditor] audit findings are based on a limited compliance review
conducted in accordance with the EOTSS IS.010 cybersecurity policy, which the CCRTA did
not opt to adopt as permitted under the policy (AUTHORITY Section 2, 2.1:
“Notwithstanding any general or special law, rule, regulation, executive order, policy or
procedure to the contrary, all executive department agencies shall, and other state
agencies may, adhere to the policies, procedures and objectives established by the
executive office of technology services and security with respect to activities concerning
information technology.”).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
35
VTA
VTA stated that 4 of the 8 employees selected did not have computer network access as
part of their job duties.
Auditor’s Reply
We appreciate the responses provided by the regional transit authorities we audited. The issue we
identified is that these regional transit authorities did not consistently provide cybersecurity training to
their employees. We regard EOTSS’s Information Security Risk Management Standard (IS.010) as the
baseline for best practices in cybersecurity awareness training across the Commonwealth’s agencies, and
therefore we used this as our audit criteria. Per Generally Accepted Government Auditing Standards 8.18,
examples of criteria include: (C) technically developed standards or norms, (f) defined business practices,
and (h) benchmarks for performance comparison, including those of other entities or sectors.
We also note here that EOTSS’s Information Security Risk Management Standard IS.010 is applicable to
the use of information systems and resources by all Commonwealth agencies within the executive branch,
encompassing, as it states, “all executive offices, and all boards, commissions, agencies, [and]
departments.” This EOTSS standard is designed to safeguard information and serves as a minimum
requirement for cybersecurity awareness training.
Regarding training employees who do not have computer network access, we maintain that EOTSS’s
Information Security Risk Management Standard IS.010 states, “All new personnel must complete an
Initial Security Awareness Training course” and that EOTSS does not provide an exemption to this policy
for employees who lack access to computer systems. We urge the regional transit authorities to
implement an alternative method to complete training for employees without system access, such as
offering a paper-based training option.
As noted above within the auditees’ responses, many RTAs have already started addressing our concerns in this area.

Official Audit Report – Issued November 8, 2024
Audit of Cybersecurity Awareness Training
Compliance Across Multiple State Agencies
For the period July 1, 2021 through April 30, 2023
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
November 8, 2024
Jason Snyder, Secretary and Commonwealth Chief Information Officer
Executive Office of Technology Services and Security
1 Ashburton Place, 8th Floor
Boston, MA 02108
Dear Mr. Snyder:
I am pleased to provide to you the results of the enclosed performance audit. Pursuant to our governing
statute, Section 12 of Chapter 11 of the Massachusetts General Laws, our audit covers multiple entities’
compliance with the Executive Office of Technology Services and Security’s cybersecurity training
standards. Specifically, the following entities were included as part of this comprehensive audit:
Executive Branch Agencies State Colleges and Universities Regional Transit Authorities
Executive Office of Technology
Services and Security
Framingham State University Cape Ann Transportation Authority
Bureau of the State House Holyoke Community College Cape Cod Regional Transit
Authority
Civil Service Commission Massachusetts Bay Community
College
Martha’s Vineyard Regional Transit
Authority
Department of Labor Standards Massasoit Community College Nantucket Regional Transit
Authority
Department of Mental Health North Shore Community College
Department of Public Health Northern Essex Community College
Department of Revenue Westfield State University (WSU)
Massachusetts Department of
Transportation
Group Insurance Commission
Massachusetts Parole Board
Registry of Motor Vehicles
State 911 Department
As is typically the case, this report details the audit objectives, scope, methodology, findings, and
recommendations for the audit period, July 1, 2021 through April 30, 2023. As you know, my audit team
discussed the contents of this report with agency managers. This report reflects those comments.
I appreciate you and all your efforts at the Executive Office of Technology Services and Security. The
cooperation and assistance provided to my staff during the audit went a long way toward a smooth
process. Thank you for encouraging and making available your team. I am available to discuss this audit if
you or your team have any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ............................................................................................... 10
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 16
1. EOTSS did not ensure that all of its employees completed cybersecurity awareness training................... 16
2. CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure that all of their employees
completed cybersecurity awareness training............................................................................................ 18
3. Seven state colleges and universities did not ensure that all of their employees completed cybersecurity
awareness training. .................................................................................................................................. 26
4. CATA, CCRTA, and VTA did not ensure that all of their employees completed cybersecurity awareness
training. .................................................................................................................................................... 32
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
List of Abbreviations
ii
LIST OF ABBREVIATIONS
911 State 911 Department
BSH Bureau of the State House
CATA Cape Ann Transportation Authority
CCRTA Cape Cod Regional Transit Authority
CSC Civil Service Commission
DLS Department of Labor Standards
DMH Department of Mental Health
DOR Department of Revenue
DPH Department of Public Health
EOTSS Executive Office of Technology Services and Security
FSU Framingham State University
GIC Group Insurance Commission
HCC Holyoke Community College
HRD Human Resources Division
MassDOT Massachusetts Department of Transportation
MBCC Massachusetts Bay Community College
MCC Massasoit Community College
MPB Massachusetts Parole Board
NECC Northern Essex Community College
NRTA Nantucket Regional Transit Authority
NSCC North Shore Community College
RMV Registry of Motor Vehicles
VTA Martha’s Vineyard Regional Transit Authority
WSU Westfield State University
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Executive Office of Technology Services and Security
(EOTSS), as well as 22 other executive branch agencies, state colleges and universities, and regional transit
authorities. This audit covers the period July 1, 2021 through April 30, 2023 and includes the following
agencies:
Executive Branch Agencies State Colleges and Universities Regional Transit Authorities
Executive Office of Technology
Services and Security (EOTSS)
Framingham State University (FSU) Cape Ann Transportation Authority
(CATA)
Bureau of the State House (BSH) Holyoke Community College (HCC) Cape Cod Regional Transit
Authority (CCRTA)
Civil Service Commission (CSC) Massachusetts Bay Community
College (MBCC)
Martha’s Vineyard Regional Transit
Authority (VTA)
Department of Labor Standards
(DLS)
Massasoit Community College
(MCC)
Nantucket Regional Transit
Authority (NRTA)
Department of Mental Health
(DMH)
North Shore Community College
(NSCC)
Department of Public Health (DPH) Northern Essex Community College
(NECC)
Department of Revenue (DOR) Westfield State University (WSU)
Massachusetts Department of
Transportation (MassDOT)
Group Insurance Commission (GIC)
Massachusetts Parole Board (MPB)
Registry of Motor Vehicles (RMV)
State 911 Department (911)
The purpose of our audit was to determine whether EOTSS and the above executive branch agencies,
state colleges and universities, and regional transit authorities ensured that their employees completed
cybersecurity awareness training in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information
Security Risk Management Standard IS.010.
Below is a summary of our findings, the effects of those finds, and our recommendations, with links to
each page listed.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
2
Finding 1
Page 16
EOTSS did not ensure that all of its employees completed cybersecurity awareness training.
Effect If EOTSS does not ensure that all of its employees complete cybersecurity awareness
training, then EOTSS may expose itself to an increased risk of cybersecurity attacks and
financial and/or reputational losses.
Recommendations
Page 17
1. EOTSS should strengthen their policy to improve oversight of executive branch state
agencies, including their timely completion of cybersecurity awareness trainings.
2. EOTSS should ensure that all employee training transcripts for all employees are
maintained and include records regarding cybersecurity awareness training
completion.
3. EOTSS should ensure that all of its employees complete cybersecurity awareness
training within 30 days of orientation and annually thereafter.
4. EOTSS should establish procedures to monitor employee cybersecurity awareness
training completion rates throughout the training cycle and use historical data retained
by HRD to ensure that employees meet training deadlines.
Finding 2
Page 18
CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure that all of their
employees completed cybersecurity awareness training.
Effect If executive branch agencies do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendation
Page 22
The aforementioned nine executive branch agencies should do the following:
1. provide cybersecurity awareness training (both an initial training within 30 days of
orientation and an annual refresher training thereafter) to all full-time employees,
contractors, and interns;
2. establish procedures to monitor employee cybersecurity awareness training
completion rates throughout the training cycle and use historical data retained by HRD
to ensure employees meet training deadlines; and
3. implement additional controls to ensure that the new hire onboarding process includes
all relevant coursework regarding cybersecurity awareness training.
Finding 3
Page 26
Seven state colleges and universities did not ensure that all of their employees completed
cybersecurity awareness training.
Effect If state colleges and universities do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendations
Page 29
1. The aforementioned seven state colleges and universities should update their
cybersecurity awareness training policies to require this training for all employees.
2. The aforementioned seven state colleges and universities should update their
cybersecurity awareness training policies to include consequences for non-completion
(e.g., restriction of access until they complete the training).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Executive Summary
3
Finding 4
Page 32
CATA, CCRTA, and VTA did not ensure that all of their employees completed cybersecurity
awareness training.
Effect If regional transit authorities do not ensure that all of their employees complete
cybersecurity awareness training, then they may expose themselves to an increased risk of
cybersecurity attacks and financial and/or reputational losses.
Recommendations
Page 34
The aforementioned three regional transit authorities should do the following:
1. update their cybersecurity awareness training policies to require this training for all
employees and
2. update their cybersecurity training policies to include consequences for noncompletion (e.g., restriction of access until training is completed).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
4
OVERVIEW OF AUDITED ENTITY
The Executive Office of Technology Services and Security (EOTSS), located at 1 Ashburton Place in Boston,
was established in 2017 in accordance with Section 2 of Chapter 7D of the Massachusetts General Laws.
According to its website, EOTSS was created to “improve data security, safeguard privacy, and promote
better service delivery across the Commonwealth.” EOTSS operates under the direction of the
Commonwealth’s chief information officer, who is appointed by the Governor.
According to its website,
The Executive Office of Technology Services and Security (EOTSS) seeks to provide secure and
quality digital information, services, and tools to customers and constituents when and where they
need them. . . . EOTSS provides responsive digital and security services that enable taxpayers,
motorists, businesses, visitors, families, and other citizens to do business with the
Commonwealth. . . . EOTSS also oversees and manages the enterprise technology and digital
infrastructure and services for over 125 state agencies and over 43,000 state employees. . . . Since
its creation, EOTSS has made critical investments in infrastructure resiliency, unifying cybersecurity
operations, and deploying a Standard Operating Environment (SOE) and technology architecture
across all agencies. The organization has also collaborated with agencies to improve the centralized
delivery of digital services for constituents, schools, businesses, government agencies, and
municipalities.
According to its website, EOTSS employed 452 full-time employees as of May 24, 2023.
Multi-Agency Approach
This report covers 22 additional agencies’ compliance with EOTSS’s cybersecurity awareness training
standard. We separated them out into three categories (other executive branch agencies in addition to
EOTSS, state colleges and universities, and regional transit authorities) for the purposes of this report.
The organization chart below shows the applicability of EOTSS guidance for the agencies in this report.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
5
Applicability of Information Security Risk Management Standard IS.0101
EOTSS and Other Executive Branch Agencies
EOTSS is responsible for the development and maintenance of the Enterprise Information Security Policies
and Standards, pursuant to Section 2 of Chapter 7D of the General Laws, which requires all executive
branch agencies to “adhere to the policies, procedures, and objectives established by the executive office
1. Agencies marked as “not under audit” are not included in this report. Additionally, EOTSS’s Information Security Risk
Management Standard IS.010 states the following regarding its scope: “Executive Department agencies and offices are
required to implement procedures that ensure their personnel comply with the requirements herein to safeguard
information.”
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
6
of technology services and security.” EOTSS states in its Information Security Risk Management Standard
IS.010 that this standard “applies to the Executive Department including all executive offices, and all
boards, commissions, agencies, departments, divisions, councils, and bureaus.” This report outlines our
audit of the following executive branch agencies regarding cybersecurity awareness training:
 EOTSS itself;
 the Bureau of the State House (BSH);
 the Civil Service Commission (CSC);
 the Department of Labor Standards (DLS);
 the Department of Mental Health (DMH);
 the Department of Public Health (DPH);
 the Department of Revenue (DOR);
 the Group Insurance Commission (GIC);
 the Massachusetts Department of Transportation (MassDOT);
 the Massachusetts Parole Board (MPB);
 the Registry of Motor Vehicles (RMV); and
 the State 911 Department (911).
The table below shows the state appropriations for each of these executive branch agencies.2 (Note that
911 does not receive state appropriations. Instead, it receives funding through an annual surcharge of
$1.50 on all telephone lines capable of accessing the 911 system. These funds are kept by 911 in a trust
fund account.)
2. This table shows state appropriations exclusively; however, some agencies receive additional funding from other sources.
State appropriations include a variety of different spending categories, including personnel, technology, and pass-through
spending. As an example, GIC (line item 1108-5100) received $4,385,239, $4,385,240, and $4,738,587 in state appropriations
in fiscal years 2021, 2022, and 2023, respectively. GIC’s state appropriations include group insurance premium and plan costs
(line item 1108-5200), which accounted for $1,747,367,959, $1,826,778,807, and $1,921,206,747 in state appropriations in
fiscal years 2021, 2022, and 2023, respectively. GIC’s state appropriations also include the State Retiree Benefits Trust Fund
(line item 1599-6152), which accounted for $500,000,000 in state appropriations in fiscal years 2021 and 2022 and
$525,000,000 in state appropriations in fiscal year 2023. See the GIC’s Historical Budget Summary for more information.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
7
Agency State Appropriations
Fiscal Year 2021
State Appropriations
Fiscal Year 2022
State Appropriations
Fiscal Year 2023
EOTSS $3,105,778 $3,105,778 $3,204,513
BSH $3,677,814 $3,927,814 $4,569,197
CSC $623,938 $625,406 $843,762
DLS $3,949,551 $4,349,551 $4,628,025
DMH $911,642,258 $951,956,760 $1,018,768,861
DPH $769,034,718 $819,954,348 $938,273,734
DOR $1,356,399,209 $1,399,872,660 $1,483,244,288
MassDOT $613,006,824 $635,459,988 $752,237,634
GIC $2,263,612,328 $2,344,120,760 $2,463,402,384
MPB $21,908,514 $20,943,687 $21,649,317
RMV $182,380,000 $131,573,000 $131,653,000
State Colleges and Universities
The state colleges and universities in Massachusetts work to improve higher education, support economic
development and growth, and support communities across the Commonwealth. The following state
colleges and universities (which were established in accordance with Section 5 of Chapter 15A of the
General Laws) are a system of public institutions of higher education, and were subjects of this audit:
 Framingham State University (FSU);
 Holyoke Community College (HCC);
 Massachusetts Bay Community College (MBCC);
 Massasoit Community College (MCC);
 North Shore Community College (NSCC);
 Northern Essex Community College (NECC); and
 Westfield State University (WSU).
The table below shows the state appropriations for each of these state colleges and universities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
8
Agency State Appropriations
Fiscal Year 2021
State Appropriations
Fiscal Year 2022
State Appropriations
Fiscal Year 2023
FSU $32,545,150 $33,193,587 $36,087,625
HCC $22,697,040 $23,207,079 $23,851,448
MBCC $17,779,141 $18,136,472 $18,746,043
MCC $24,064,288 $24,474,243 $25,391,675
NSCC $24,154,641 $24,600,186 $25,517,333
NECC $21,986,040 $22,385,471 $23,251,578
WSU $30,992,952 $31,621,476 $34,336,799
Regional Transit Authorities
Regional transit authorities provide public transportation services in different communities within
Massachusetts, meeting the specific transit needs of each community. The following regional transit
authorities were established in accordance with Section 2 of Chapter 161B of the General Laws and were
subjects of this audit:
 the Cape Ann Transportation Authority (CATA);
 the Cape Cod Regional Transit Authority (CCRTA);
 the Martha’s Vineyard Regional Transit Authority (VTA); and
 the Nantucket Regional Transit Authority (NRTA).
The table below shows the operating revenues for each of these regional transit authorities.
Agency Operating Revenues
Fiscal Year 2021
Operating Revenues
Fiscal Year 2022
Operating Revenues
Fiscal Year 2023
CATA $13,642,963 $2,604,218 $512,110
CCRTA $9,083,000 $1,456,000 $1,139,000
VTA $1,289,000 $1,779,000 $1,798,000
NRTA $389,492 $578,464 $614,688
Cybersecurity Awareness Training
EOTSS has established policies and procedures that apply to all Commonwealth agencies within the
executive branch. These policies and procedures require executive branch agencies to implement
procedures that ensure that their employees comply with the requirements in EOTSS’s aforementioned
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Overview of Audited Entity
9
policies and procedures. EOTSS recommends, but does not require, non-executive branch agencies to
follow its policies and procedures. Section 6.2 of EOTSS’s Information Security Risk Management Standard
IS.010 states,
The objective of the Commonwealth information security training is to educate users on their
responsibility to help protect the confidentiality, availability and integrity of the Commonwealth’s
information assets. Commonwealth Offices and Agencies must ensure that all personnel are trained
on all relevant rules and regulations for cybersecurity.
To ensure that employees in all Commonwealth agencies within the executive branch are clear on their
responsibilities, EOTSS’s policies and procedures require that all newly hired employees3 must complete
an initial cybersecurity awareness training course within 30 days of their orientation, and that all existing
employees4 complete an annual refresher cybersecurity awareness course.
3. For the purposes of this audit report, we use the term newly hired employees to refer to employees who were hired during
the audit period, unless stated otherwise.
4. For the purposes of this audit report, we use the term existing employees to refer to employees who were hired before the
start of the audit period (July 1, 2021), unless stated otherwise.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
10
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of cybersecurity awareness training at the Executive Office of
Technology Services and Security (EOTSS). Pursuant to our governing statute, Section 12 of Chapter 11 of
the General Laws, our audit covers multiple entities’ compliance with EOTSS’s cybersecurity training
standards. Specifically, Section 12 of Chapter 11 states, “Each entity may be audited separately as a part
of a larger organizational entity or as a part of an audit covering multiple entities.” As such, cybersecurity
awareness training testing was completed at 22 other executive branch agencies, state colleges and
universities, and regional transit authorities, for the period July 1, 2021 through April 30, 2023.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objective.
Below is our audit objective, indicating the question we intended our audit to answer; the conclusion we
reached regarding our objective; and, if applicable, where our objective is discussed in the audit findings.
Objective Conclusion
1. Did EOTSS and other executive branch agencies, state colleges and universities, and
regional transit authorities ensure that their employees completed cybersecurity
awareness training in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information
Security Risk Management Standard IS.010?
No; see Findings
1, 2, 3, and 4
To accomplish our audit objective, we gained an understanding of the aspects of EOTSS’s internal control
environment relevant to our objective by interviewing EOTSS staff members and management and by
reviewing EOTSS’s Information Security Risk Management Standard IS.010.
To obtain sufficient, appropriate evidence to address our audit objective, we performed the procedures
described below.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
11
Cybersecurity Awareness Training
We separated the 23 agencies we reviewed as part of this audit into three categories based on agency
type: EOTSS and other executive branch agencies, state colleges or universities, and regional transit
authorities.
 The first category comprises EOTSS and 11 other executive branch agencies: the Bureau of the
State House (BSH), the Civil Service Commission (CSC), the Department of Labor Standards (DLS),
the Department of Mental Health (DMH), the Department of Public Health (DPH), the Department
of Revenue (DOR), the Massachusetts Department of Transportation (MassDOT), the Group
Insurance Commission (GIC), the Massachusetts Parole Board (MPB), the Registry of Motor
Vehicles (RMV), and the State 911 Department (911).
 The second category comprises seven state colleges and universities: Framingham State
University (FSU), Holyoke Community College (HCC), Massachusetts Bay Community College
(MBCC), Massasoit Community College (MCC), North Shore Community College (NSCC), Northern
Essex Community College (NECC), and Westfield State University (WSU).
 The third category comprises four regional transit authorities: the Cape Ann Transportation
Authority (CATA), the Cape Cod Regional Transit Authority (CCRTA), the Martha’s Vineyard
Regional Transit Authority (VTA), and the Nantucket Regional Transit Authority (NRTA).
To determine whether EOTSS and these other executive branch agencies, state colleges and universities,
and regional transit authorities ensured that their employees completed cybersecurity awareness training
in accordance with Sections 6.2.3 and 6.2.4 of EOTSS’s Information Security Risk Management Standard
IS.010, we took the actions described below.
EOTSS and Other Executive Branch Agencies
To determine whether EOTSS and the 11 other executive branch agencies included in this audit
ensured that their newly hired employees completed initial cybersecurity awareness training within
30 days of orientation, we analyzed the evidence for cybersecurity awareness training completion
(i.e., transcript reports5
) by comparing each employee’s start date and training completion date for
all 2,662 newly hired employees across these executive branch agencies.
To determine whether these executive branch agencies ensured that their existing employees
completed annual refresher cybersecurity awareness training, we analyzed the evidence for
cybersecurity awareness training completion (i.e., transcript reports) by comparing each employee’s
5. We analyzed the cybersecurity awareness training transcript reports from EOTSS and the other executive branch agencies.
These reports included fields such as the training due date and the training completion date.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
12
training completion date and training due date for all 12,236 existing employees across these
executive branch agencies.
To further substantiate the results of the above procedures, we also selected a random, statistical
sample6 of 24 employee training certificates of completion out of the population of 14,898 newly
hired and existing employees, using a 90% confidence level,7 a 0% expected error rate,8 and a 10%
tolerable error rate.9 Our sample comprised the following:
 from EOTSS, BSH, CSC, DLS, GIC, MPB, RMV, and 911: 1 employee training certificate of
completion from each agency;
 from DOR: 2 employee training certificates of completion;
 from DPH and MassDOT: 4 employee training certificates of completion from each agency;
and
 from DMH: 6 employee training certificates of completion.
We selected these sample numbers based on the number of active employees each agency had during
the audit period.
We did not note any exceptions in our testing corresponding to BSH and 911. Therefore, we concluded
that, during the audit period, BSH and 911 met the relevant criteria regarding this matter.
For the other executive branch agencies included in this audit, we did note exceptions during our
testing. See Findings 1 and 2 for issues we identified with the cybersecurity awareness training
provided by EOTSS and the other executive branch agencies included in this audit.
6. Auditors use statistical sampling to select items for audit testing when a population is large and contains similar items.
Auditors generally use a statistical software program to choose a random sample when sampling is used. The results of testing
using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections
about entire populations.
7. Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are
representative of the population (parameter), expressed as a percentage.
8. Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the
auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.
9. The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while
still using the sample to conclude that the results from the sample have achieved the objective.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
13
State Colleges and Universities
To determine whether the state colleges and universities included in this audit ensured that their
employees completed cybersecurity awareness training, we took the actions described below.
We inspected the cybersecurity awareness training certificates of completion using a judgmental,10
nonstatistical sample of 70 employee training certificates of completion out of the population of
10,094. Our sample comprised 10 employee training certificates of completion from each of the seven
state colleges and universities included in this audit. Of the 10 employee training certificates of
completion from each state college or university, we judgmentally selected 3 existing non-student
employees, 4 newly hired non-student employees, and 3 existing student employees.
Also, we determined whether the state colleges and universities included in this audit ensured that
the newly hired employees from our sample completed initial training within 30 days of orientation
by comparing the dates of their orientations to the dates of their certificates of completion.
See Finding 3 for issues we identified with the cybersecurity awareness training provided by the state
colleges and universities included in this audit.
Regional Transit Authorities
To determine whether the regional transit authorities included in this audit ensured that their
employees completed cybersecurity awareness training, we took the actions described below.
We inspected the cybersecurity awareness training certificates of completion using a judgmental,
nonstatistical sample of 23 employee training certificates of completion out of the population of 55.
Our sample comprised the following:
 from CATA: 3 employee training certificates of completion (which represents its full
population of employees);
 from NRTA: 4 employee training certificates of completion (which represents its full
population of employees); and
10. Auditors use judgmental sampling to select items for audit testing when a population is very small, the population items are
not similar enough, or there are specific items in the population that the auditors want to review. Auditors use their
knowledge and judgment to select the most appropriate sample. For example, an auditor might select items from areas of
high risk. The results of testing using judgmental sampling cannot be used to make conclusions or projections about entire
populations; however, they can be used to identify specific issues, risks, or weaknesses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
14
 from CCRTA and VTA: 8 employee training certificates of completion from each agency.
Of the 8 employee training certificates of completion from CCRTA and VTA, we judgmentally selected
2 newly hired employees and 6 existing employees. Additionally, we determined whether these
regional transit authorities ensured that the newly hired employees from our sample completed initial
training within 30 days of orientation by comparing the dates of their orientations to the dates of their
certificates of completion.
We did not note any exceptions in our testing corresponding to NRTA. Therefore, we concluded that,
during the audit period, NRTA met the relevant criteria regarding this matter.
For the other regional transit authorities included in this audit, we noted exceptions during our
testing. See Finding 4 for issues we identified with the cybersecurity awareness training provided by
the regional transit authorities included in this audit.
We used a combination of statistical and nonstatistical sampling methods for testing, and we did not
project the results of our testing to any corresponding populations.
Data Reliability Assessment
To determine the reliability of the employee lists from EOTSS and each of the 22 other executive branch
agencies, state colleges and universities, and regional transit authorities included in this audit (see the list
of auditees included in this report, by category), we took the actions described below.
We interviewed EOTSS management who were knowledgeable about these lists. We reviewed
MassAchieve11 system controls for access control, configuration management, contingency planning,
segregation of duties, and security management. We checked that the variable formats of each agency’s
employee list (e.g., dates, unique identifiers, or abbreviations) were accurate. For each agency’s employee
list, we ensured that there was no abbreviation of data fields, no missing data (e.g., hidden rows or
columns, blank cells, or incomplete records), and no duplicate records and that all values corresponded
with expected values.
To determine the completeness and accuracy of each agency’s employee list, we took the actions
described below.
11. MassAchieve is a training platform used by executive branch agencies to administer cybersecurity awareness training.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Audit Objectives, Scope, and Methodology
15
EOTSS and Other Executive Branch Agencies
 EOTSS: We selected random samples of 20 employees from EOTSS’s employee list and traced
their names to CTHRU, the Commonwealth’s statewide payroll open records system. We also
selected random samples of 20 employees from CTHRU and traced their names back to
EOTSS’s employee list.
 BSH and CSC: We selected random samples of five employees from each executive branch
agency’s employee list and traced their names to CTHRU. We also selected random samples
of five employees from each agency from CTHRU and traced their names back to each
agency’s employee list.
 DLS, GIC, MPB, and 911: We selected random samples of 10 employees from each executive
branch agency’s employee list and traced their names to CTHRU. We also selected random
samples of 10 employees from each agency from CTHRU and traced their names back to each
agency’s employee list.
 DMH, DPH, DOR, MassDOT, and RMV: We selected random samples of 20 employees from
each executive branch agency’s employee list and traced their names to CTHRU. We also
selected random samples of 20 employees from each agency from CTHRU and traced their
names back to each agency’s employee list.
State Colleges and Universities
 FSU, HCC, MBCC, MCC, NSCC, NECC, and WSU: We selected random samples of 20 employees
from each state college’s/university’s employee list and traced their names to CTHRU. We
also selected random samples of 20 employees from each state college/university from
CTHRU and traced their names back to each state college/university’s employee list.
Regional Transit Authorities
 CATA: We selected the total population of three employees and traced their names to CATA’s
open payroll webpage. We also selected the total population of three employees from CATA’s
open payroll webpage and traced their names back to CATA’s employee list.
 CCRTA and VTA: We selected random samples of five employees from each regional transit
authority’s employee list and traced their names to each agency’s open payroll webpage. We
also selected random samples of five employees from each regional transit authority’s open
payroll webpage and traced their names back to each agency’s employee list.
 NRTA: We selected the total population of four employees and traced their names to NRTA’s
open payroll webpage. We also selected the total population of four employees from NRTA’s
open payroll webpage and traced their names back to NRTA’s employee list.
Based on the results of the data reliability assessment procedures described above, we determined that
the information we obtained for the audit period was sufficiently reliable for the purposes of our audit.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
16
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. EOTSS did not ensure that all of its employees completed cybersecurity
awareness training.
The Executive Office of Technology Services and Security (EOTSS) did not ensure that all of its employees
who were active during the audit period completed initial and annual refresher cybersecurity awareness
training.
The original due date for the training was August 31, 2022, but EOTSS executive management requested
and received an extension from the Human Resources Division (HRD), which extended the due date for
all executive branch agencies to October 14, 2022. HRD communicated this new deadline to executive
branch managers through its Managers’ Corner Newsletter.
The table below shows our findings for EOTSS. Note that this table reflects the extended October 14, 2022
due date.
Cybersecurity
Awareness
Training Type
On-Time Training
Completion
Percentage
Total Number
of Employees
Tested
Number of Employees
Who Completed
Training Late
Number of Employees
Who Did Not Complete
Training
Initial 67.8% 115 28 9
Annual Refresher 99.8% 411 — 1
If EOTSS does not ensure that all of its employees complete cybersecurity awareness training, then EOTSS
may expose itself to an increased risk of cybersecurity attacks and financial and/or reputational losses.
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
17
Reasons for Issue
EOTSS management explained that contract employees undergo a different onboarding process
compared to non-contact employees. EOTSS processes contract employees’ training assignments in
batches and must create training accounts manually. This process is time-consuming and typically occurs
only once or twice per month. Additionally, EOTSS management noted that they do not have access to
training transcripts for former employees.
Recommendations
EOTSS should strengthen their policy to improve oversight of executive branch state agencies,
including their timely completion of cybersecurity awareness trainings.
EOTSS should ensure that all employee training transcript for all employees are maintained and
include records regarding cybersecurity awareness training completion.
EOTSS should ensure that all of its employees complete cybersecurity awareness training within 30
days of orientation and annually thereafter.
EOTSS should establish procedures to monitor employee cybersecurity awareness training completion
rates throughout the training cycle and use historical data retained by HRD to ensure that employees
meet training deadlines.
Auditee’s Response
Security awareness training is a critical component of the Commonwealth’s security compliance
strategy. Mandatory cybersecurity training must be completed within 30 days of employee
orientation. The new hire 30-day training completion requirement is tied to employee orientation,
rather than date of hire, to accommodate business processes related to onboarding and
credentialing into the training system. Further, the process for onboarding and credentialing
contract employees is different than the process for non-contract employees. Contractors are
assigned training in “batches” once or twice per month. The new hire 30-day training completion
requirement is purposefully tied to orientation date, as opposed to new hire date to accommodate
for such business processes. [The Office of the State Auditor] relied on hire date, rather than
employee orientation/onboarding date to calculate the 30-day deadline.
Moving forward, EOTSS will evaluate its internal processes to identify areas for improvement
related to new hire orientation and contractor onboarding.
Additionally, EOTSS will work with necessary partners to explore whether there is a technical
solution to accessing transcript data of former agency employees.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
18
Auditor’s Reply
We agree with EOTSS’s statement that “security awareness training is a critical component of the
Commonwealth’s security compliance strategy,” and for this reason, we believe that all employees,
regardless of classification, should complete their initial training within 30 days. The data provided by
EOTSS in response to our data requests in this audit did not include new hire orientation dates, it included
new hire start dates.
Additionally, while we acknowledge that EOTSS has established policies and procedures applicable to all
Commonwealth agencies within the executive branch, based on the findings below respective to those
executive branch agencies, we believe there is a need for EOTSS to enhance its oversight of these agencies
to ensure greater compliance with the Enterprise Information Security Policies and Standards.12
Based on its response, EOTSS has indicated that it will take steps to address our concerns on this matter.
We will follow up on this during our post-audit review process in approximately six months.
2. CSC, DLS, DMH, DPH, DOR, MassDOT, GIC, MPB, and RMV did not ensure
that all of their employees completed cybersecurity awareness training.
The following executive branch agencies did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: the Civil Service Commission (CSC), the
Department of Labor Standards (DLS), the Department of Mental Health (DMH), the Department of Public
Health (DPH), the Department of Revenue (DOR), the Massachusetts Department of Transportation
(MassDOT), the Group Insurance Commission (GIC), the Massachusetts Parole Board (MPB), and the
Registry of Motor Vehicles (RMV).
Regarding the completion rates for the initial cybersecurity awareness training, we observed that 445
newly hired employees completed training late, while 601 did not complete training at all. Regarding the
completion rates for the annual refresher cybersecurity awareness training, we observed that 156 existing
employees completed training late, while 951 did not complete training at all.
12. The Enterprise Information Security Policies and Standards is the compilation of policies and standards that all executive
branch agencies are required to follow. Information Security Risk Management Standard IS.010 is just one of these policies.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
19
The table and graph below show our findings for these agencies regarding initial cybersecurity awareness
training.
On-Time Cybersecurity Awareness Training Completion Rates for Executive
Branch Agencies: Newly Hired Employees
0.0%
54.3%
66.4%
97.8%
44.1%
66.7%
76.2%
67.8%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CSC DMH DPH DOR MassDOT GIC MPB RMV
Agency On-Time Initial
Training Completion
Percentage
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training Late
Number of Tested
Employees Who Did Not
Complete Training
CSC 00.0% 1 — 1
DMH 54.3% 905 148 266
DPH 66.4% 524 83 93
DOR 97.8% 229 — 5
MassDOT 44.1% 742 185 230
GIC 66.7% 3 1 —
MPB 76.2% 21 5 —
RMV 67.8% 90 23 6
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
20
The table and graph below show our findings for these agencies regarding annual refresher cybersecurity
awareness training.
On-Time Cybersecurity Awareness Training Completion Rates for Executive
Branch Agencies: Existing Employees
70.0%
98.3%
90.9% 86.9%
99.5%
89.2%
99.3%
91.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CSC DLS DMH DPH DOR MassDOT MPB RMV
Agency On-Time Annual
Refresher Training
Completion Percentage
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training Late
Number of Tested
Employees Who Did
Not Complete Training
CSC 70.0% 10 — 3
DLS 98.3% 58 1 —
DMH 90.9% 3246 30 265
DPH 86.9% 2911 26 355
DOR 99.5% 1356 — 7
MassDOT 89.2% 3455 91 284
MPB 99.3% 151 — 1
RMV 91.0% 488 8 36
If executive branch agencies do not ensure that all of their employees complete cybersecurity awareness
training, then they may expose themselves to an increased risk of cybersecurity attacks and financial
and/or reputational losses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
21
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following executive branch agencies provided us with the following
reasons for noncompliance:
 CSC management stated that contracted attorneys and interns were not on the list of employees
required to complete the cybersecurity awareness training.
 DLS management stated that they are not sure what the reason was for the late training
completion of the employee from our finding, other than the employee overlooked the training
due date. DLS management noted that this employee is no longer with DLS.
 DMH management sent us an email on February 16, 2024 regarding the employees from our
finding, stating that these are “Employees who do not have [computer] Network Access—these
staff are exempt.”
 DPH management stated that the staff members from our finding started their cybersecurity
awareness training but did not complete the full training.
 DOR management stated that some of the employees from our finding had job duties that did not
require them to have computer network access, while others separated from DOR shortly after
their training due date had passed, leaving no time for DOR to enforce training completion.
 MassDOT management and RMV management stated that employees missed the training
deadline and that interns did not receive cybersecurity awareness training because MassAchieve
did not assign them training.
 GIC management stated that the employee from our finding left the agency shortly after starting
and did not complete the training before their departure.
 MPB management stated that newly hired employees have assigned joint orientation/training
days which may have been scheduled past the 30 days from hire dates for some staff. Regarding
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
22
refresher training, it appears that one employee found not to have completed the training,
completed only 2 out of the 5 required sections of the cybersecurity training.
Recommendations
The aforementioned nine executive branch agencies should do the following:
provide cybersecurity awareness training (both an initial training within 30 days of orientation and an
annual refresher training thereafter) to all full-time employees, contractors, and interns;
establish procedures to monitor employee cybersecurity awareness training completion rates
throughout the training cycle and use historical data retained by HRD to ensure employees meet
training deadlines; and
implement additional controls to ensure that the new hire onboarding process includes all required
coursework regarding cybersecurity awareness training.
We appreciate the following responses provided by the executive branch agencies:
Auditee’s Response: CSC
CSC appreciates receiving clarification from the Office of the State Auditor that seasonal interns
and contract employees are required to complete the cybersecurity awareness training. In
response, CSC had the seasonal interns and contract employee at the time immediately complete
the required cybersecurity training. Going forward, any CSC interns and contract employees will be
required to complete the same initial and refresher cybersecurity training as all regular CSC
employees, ensuring 100% compliance with this requirement.
Auditor’s Reply: CSC
Based on its response, CSC has taken measures to address our concerns regarding this matter.
Auditee’s Response: DLS
[DLS] management agrees with the finding. As [the Office of the State Auditor] has affirmed, we
now have a program to ensure employees are trained in a timely manner. This is demonstrated by
the 100% completion for new hires and near completion for existing employees. Of the two existing
employees whose training was not completed by the deadline, one staff was one (1) day late due
to her supervisor leaving and the new supervisor not receiving the alerts, while the other staff is
no longer an [Executive Office of Labor and Workforce Development] employee. Regardless, we
will continue to reinforce timely completion by sending email reminders.
Auditor’s Reply: DLS
Based on its response, DLS will take measures to address our concerns regarding this matter.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
23
Auditee’s Response: DMH
As DMH indicated to the [Office of the State Auditor] during the audit, 417 of the newly hired
individuals are contract employees who do not have any network access. Consequently, they do
not need cybersecurity training. In fact, providing this training would unnecessarily expend
resources and increase security risk, as DMH would need to create network access solely to provide
the training.
DMH recognizes that the [Office of the State Auditor] assesses compliance with the policy or
standard as written, and that it reads Section 6.2 of EOTSS’s Information Security Risk Management
Standards as requiring cybersecurity training for “all personnel.” Indeed, Section 6.2 states that
“all personnel” must be trained. The immediately preceding sentence, however, states that the
objective of the cybersecurity training is to educate “users” on their cybersecurity responsibilities.
Respectfully, DMH views the word “personnel” in the second sentence as referring to the “users”
referred to in the first sentence. Thus, per DMH’s reading, only “users” must be trained. . . .
The data used for this finding had some limitations, as indicated during the audit. Some employees
were hired after the end date that the 2021 annual cybersecurity training was due; some left state
service and then returned after the due date for the 2021 cybersecurity training; and, on account
of system limitations, DMH was unable to determine dates that some staff left DMH. DMH
understands that data of the sort required and assessed here typically has limitations, and that the
Auditor’s Office needs to utilize data as provided, but the number here likely is not accurate.
Auditor’s Reply: DMH
Section 2 of Chapter 7D of the Massachusetts General Laws mandates that all executive branch state
agencies, including DMH, “adhere to the policies, procedures and objectives established by the executive
office of technology services and security.” DMH must ensure that contractors are trained in compliance
with EOTSS’s Information Security Risk Management Standard IS.010.
Regarding the definition of “personnel,” we maintain that EOTSS’s Information Security Risk Management
Standard IS.010 states, “All new personnel must complete an Initial Security Awareness Training course,”
and that EOTSS does not provide an exemption to this policy for employees who lack access to computers.
We urge DMH to implement an alternative method for employees without system access to complete
their training, such as offering a paper-based training option. We recognize that some agencies may
disagree with EOTSS standards, but nonetheless, these standards exist. Cybersecurity awareness policies
are not just guidelines; they are essential safeguards in today’s digital landscape. Comprehensive
employee training and shared responsibility are critical to mitigating potential cyber threats. It is
important to consistently assess and reinforce cybersecurity measures to ensure that policies are
effective, compliance is maintained, and public trust in the agency’s ability to safely manage data is
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
24
upheld. These policies exist to protect both individuals and organization, fostering a secure and safe digital
environment.
Regarding the data’s limitations, we conducted a data reliability assessment on the information DMH
provided to us, ensuring the completeness and accuracy of DMH’s employee list. As we have
recommended, we believe that DMH should establish procedures to (1) monitor employee cybersecurity
awareness training completion rates throughout the training cycle, (2) accurately track the dates when
employees leave the agency, and (3) use historical data retained by HRD to ensure that employees meet
training deadlines.
Auditee’s Response: DPH
1. Provide cybersecurity awareness training (both an initial training within 30 days of orientation
and an annual refresher training thereafter) to all full-time employees, contractors, and interns.
a. The training is offered through MassAchieve within 30 days of start and annually.
b. DPH has increased staffing in this area and developed and implemented a robust system
of reminders for all staff who are incompliant starting in December of each year.
c. We promote completion of this training by alerting staff to the consequence of shut-off by
EOTSS.
d. This past fiscal year we achieved near perfect completion with less than 10 shut offs.
2. Establish procedures to monitor employee completion throughout the training cycle to ensure
that staff are meeting the training deadlines.
a. Our staff run reports monthly and have empowered each bureau, office and hospital to
run their own custom-built reports.
b. We established standard communications to go out to supervisors and incompliant staff.
We appreciate the insights provided by the audit and are addressing these findings promptly.
Auditor’s Reply: DPH
Based on its response, DPH has taken measures to address our concerns regarding this matter.
Auditee’s Response: DOR
DOR agrees with the results of the audit. The employees who did not complete the training during
the audit period were employees with no access to computers or were separated from DOR shortly
after hire.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
25
DOR will continue to utilize MassAchieve to track employee completion throughout the training
cycle.
In [fiscal year 2024], DOR implemented the process of “paper training,” where Employees with no
access to computers and/or systems will take the training in person, in a class organized by their
managers, and sign an acknowledgement that they have received, taken and understand the
training. Information will be uploaded to MassAchieve.
DOR will incorporate cybersecurity awareness training into the new hire process, where the course
is added to DOR’s Learning Management System (LMS—DOR’s internal training system). LMS
system also will be used to track completion and follow up with new hires that have not completed
the training. Information will be uploaded to MassAchieve.
Auditor’s Reply: DOR
Based on its response, DOR has taken, and will continue to take, measures to address our concerns
regarding this matter.
Auditee’s Response: MassDOT and RMV
As of 2024, MassDOT has transitioned to using only the MassAchieve LMS, eliminating confusion
for employees regarding where to find and complete assigned training. Furthermore, statewide
improvements, such as increased frequency of reminders from HRD, have helped improve
performance. Additionally, EOTSS has followed through on removing access to those who do not
complete cybersecurity training on time. MassDOT has used this consequence to effect in our
messaging to further incentivize timely completion of cybersecurity training and has collaborated
with EOTSS as needed to reinstate access for individuals who had their access removed due to
non-compliance. . . .
In the 2023–24 training cycle MassDOT implemented procedures to continue to support the
agency’s efforts in meeting its compliance obligation. This includes earlier distribution of targeted
activity reports, making it easier for managers to identify those yet to complete training. Reports
are shared on an increasing cadence as the training deadline approaches.
Auditor’s Reply: MassDOT and RMV
Based on their response, MassDOT and RMV have taken measures to address our concerns regarding this
matter.
Auditee Response: GIC
GIC was given the opportunity to respond to a draft version of this audit report and did not provide a
written response.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
26
Auditee’s Response: MPB
MPB concurs with [the Office of the State Auditor’s] recommendations to (1) provide cybersecurity
awareness training (both an initial training within 30 days of orientation and an annual refresher
training thereafter) to all full-time employees, contractors, and interns; and (2) establish
procedures to monitor employee completion throughout the training cycle to ensure that staff are
meeting the training deadlines.
To improve timely completion of cybersecurity training for new hires, MPB will modify its existing
“Checklist for Employee Orientation” form to specify due dates for completion of cybersecurity
training and include an acknowledgement receipt upon completion.
Bi-weekly Managers’ Meetings will be utilized to further monitor adherence to the training
deadlines.
Auditor’s Reply: MPB
Based on its response, MPB will take measures to address our concerns regarding this matter.
3. Seven state colleges and universities did not ensure that all of their
employees completed cybersecurity awareness training.
The following state colleges and universities did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: Framingham State University (FSU), Holyoke
Community College (HCC), Massachusetts Bay Community College (MBCC), Massasoit Community College
(MCC), North Shore Community College (NSCC), Northern Essex Community College (NECC), and Westfield
State University (WSU).
The table and graph below show our findings for these state colleges and universities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
27
On-Time Cybersecurity Awareness Training Completion Rates for State
Colleges and Universities: Sample of All Employees
40.0%
60.0%
50.0% 50.0%
80.0%
30.0%
70.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
FSU HCC MBCC MCC NSCC NECC WSU
State College
or University
On-Time Training
Completion
Percentage*
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training
Late
Number of Tested
Employees Who Did
Not Complete Training
FSU 40.0% 10 — 6
HCC 60.0% 10 — 4
MBCC 50.0% 10 — 5
MCC 50.0% 10 — 5
NSCC 80.0% 10 — 2
NECC 30.0% 10 — 7
WSU 70.0% 10 — 3
* Note that this table is based on the sample of employees from each state college or university, not the population of
employees.
If state colleges and universities do not ensure that all of their employees complete cybersecurity
awareness training, then they may expose themselves to an increased risk of cybersecurity attacks and
financial and/or reputational losses.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
28
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following state colleges and universities provided us with the following
reasons for noncompliance:
 FSU management stated that its internal policy only recommended cybersecurity awareness
training for its employees, instead of requiring it.
 HCC management stated that student employees did not have access to HCC’s computer network
(which is only accessible with staff member accounts), so therefore, providing them with
cybersecurity awareness training would not be required.
 MBCC management stated the following:
 Two student employees from our finding “never received student [employee] accounts, so
they were missed in getting training assigned as part of the onboarding” (from an email MBCC
sent to us on February 15, 2024);
 Two employees from our finding “did not elect to complete their training. . . . As a result, their
employment with [MBCC] was discontinued” (from an email MBCC sent to us on February 15,
2024);
 One newly hired employee from our finding “started before the training program was in place,
so [they] would not have had the option for [initial] training” (from an email MBCC sent to us
on February 15, 2024); and
 One newly hired employee from our finding joined MBCC while the college was conducting
annual refresher cybersecurity awareness training, so MBCC enrolled this employee in the
annual refresher training rather than being trained on the same content twice in a short
period of time by first being enrolled in the initial cybersecurity awareness training.
 MCC management stated that its internal policy only recommended cybersecurity awareness
training for its employees, instead of requiring it.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
29
 NSCC management stated that two newly hired employees did not complete the cybersecurity
awareness training because its auto-enrollment process failed briefly in September 2022, leading
to NSCC’s inability to enroll newly hired employees into the training during this period.
 NECC management stated that it has a written cybersecurity awareness training policy, but that
the policy is not enforced. Management also stated that they are not allowed to limit user access
for employees who do not complete cybersecurity awareness training.
 WSU management stated that they were not aware that contractors, part-time employees, or
seasonal employees were required to complete the cybersecurity awareness training.
Recommendations
The aforementioned seven state colleges and universities should update their cybersecurity
awareness training policies to require this training for all employees.
The aforementioned seven state colleges and universities should update their cybersecurity
awareness training policies to include consequences for non-completion (e.g., restriction of access
until they complete the training).
Auditees’ Responses
FSU
We are in agreement with the merits of the [EOTSS] Standard and the University is now
aligned with the goals of the cybersecurity awareness training. To that end, since the
completion of the field work associated with this audit, but prior to the receipt of this draft
report, FSU developed and formally adopted campus policy consistent with the Information
Security Risk Management Standard. Appendix A contains the text of this Policy on
Cybersecurity Training for Employees established on July 17, 2024. The policy is currently
in effect and will begin full implementation in October 2024 pursuant to the establishment
of a bargained labor agreement that permits initial onboarding cybersecurity training and
then subsequent annual training, including prescriptive penalties or remediations for
noncompliance.
This local policy will achieve the same goals and mitigate the risks identified in the
recommendations associated with Finding 3. We remain committed to the protection of
the information technology assets and information retained by the University and share
the mutual desire to remain vigilant to new and emerging threats to these digital assets
and networks.
HCC
Upon learning that all HCC work study students, regardless of their need to access the
network, must complete the cybersecurity training within 30 days of their assignment, HCC
implemented the following policies and procedures:
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
30
Policy: HCC’s policy now mandates that all work study students will be notified they need
to complete mandatory cybersecurity training within 30 days of starting their work
assignment.
Consequences: Failure to complete the required training within 30 days of their work
assignment will result in revoking their work study assignment/job until the training is
completed.
MBCC
[MBCC writes] in response to your email of July 19, 2024, regarding the recent audit of
cybersecurity training at [MBCC]. Thank you for sharing the audit results and providing us
with the opportunity to respond.
The two student employees mentioned did not receive student employee accounts and
thus were not assigned training during onboarding. As part of our employee onboarding
process, all MBCC employees receive an account and are enrolled in the new hire
cybersecurity training program. This issue was identified in November 2023 due to this
audit, and since then, MBCC has taken steps to ensure the enforcement of this process.
Two employees chose not to complete their training, leading to the termination of their
employment with MBCC, underscoring the institution’s commitment to mandatory training.
One employee joined before the training program was established. The program is now
fully operational, requiring all employees to complete it within 30 days of starting. If not,
they are granted an additional 30 days then this [is] escalated to senior management and
their access is restricted until it is completed.
Lastly, one newly hired employee started with MBCC during the annual cybersecurity
awareness training. As the on boarding training is identical, they were not enrolled twice.
Going forward we will ensure that they are enrolled in both.
Thank you again for the audit. Our policy states that all employees must complete the
cybersecurity training, but this audit helped us identify areas for improvement. We have
taken the necessary steps to remediate areas of concern. Going forward we anticipate we
will be in full compliance with the State requirements.
MCC
The college fully acknowledges the need for, and importance of, cybersecurity training for
all employees.
Massasoit Community College’s leadership is currently developing language to amend the
existing Written Information Security Program (WISP) with the recommendations of the
recent Executive Office of Technology Services and Security performance audit.
The college will be collaborating with the Unions, through impact bargaining, to ensure
proper checks and balances are in place, that new hire training and annual re-training are
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
31
conducted in a timely manner, and that, if necessary, reasonable gradated consequences
for non-compliance are in place.
NSCC
The College agrees that cybersecurity training is critical and important. The College
management and especially the [information technology] department has put a great deal
of effort into a collaborative process ensuring that cybersecurity training is ongoing and
annual, as demonstrated in our highest completion rate (80%) of those tested in the [Office
of the State Auditor] draft report. Since that audit the College has gone further with tighter
process improvements which now disables employee accounts that have not completed
either the new employee training or annual training within the allotted time frames.
Disabled accounts are reenabled upon request and employees are granted an additional
week to complete the required training. Our training completion rate now stands at 97%.
NECC
At NECC we specifically value and understand the importance of Cybersecurity training.
Recently we experienced a cyber incident caused by user error. Had it not been for the
systems we have in place; this threat would have had significant impact on our operation.
We also worked with EOTSS after the incident to discuss lessons learned from the attack,
working with vendors and the Commonwealth.
In order to better comply with EOTSS’s Information Security Risk Management Standard
IS.010, and industry’s best practices, we have developed a revised Cybersecurity Training
Process. . . . NECC is implementing the process starting in the Fall. This process may be
subject to impact bargaining with our [Massachusetts Community College Council] and
[American Federation of State, County and Municipal Employees] union members.
Thank you again for the opportunity to respond to this audit and please do not hesitate to
contact me should you have any additional questions.
WSU
Westfield State’s current Security Education Training and Awareness (SETA Program)
already requires training as part of the campus onboarding program. . . . For the faculty
collective bargaining unit, [Massachusetts State College Association], training was impact
bargained and the final agreement was completed on March 21, 2024. As a result,
beginning in the fall 2024, cyber security training will be required for faculty. . . .
The University’s Access Control Guidelines already allows for the suspension of access to
information technology resources for non-compliance. Efforts are currently underway to
formalize the consequences with Office of Information and Instructional Technology and
the Human Resources Office. Progressive discipline actions may require further impact
bargaining.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
32
Auditor’s Reply
We appreciate the responses provided by the seven state colleges and universities we audited. The issue
we identified is that these state colleges and universities did not consistently provide cybersecurity
training to their employees. We regard EOTSS’s Information Security Risk Management Standard IS.010
as the baseline for best practices in cybersecurity awareness training across the Commonwealth’s
agencies, and therefore, we used this as our audit criteria. According to Section 8.18 of the US
Government Accountability Office’s Generally Accepted Government Auditing Standards, “Examples of
criteria include: . . . (c) technically developed standards or norms; . . . (f) defined business practices; . . .
and (h) benchmarks against which performance is compared, including performance of other entities or
sectors.”
As noted above within the auditees’ responses, many colleges and universities have already started
addressing our concerns in this area.
4. CATA, CCRTA, and VTA did not ensure that all of their employees completed
cybersecurity awareness training.
The following regional transit authorities did not ensure that all of their employees completed
cybersecurity awareness training during the audit period: the Cape Ann Transportation Authority (CATA),
the Cape Cod Regional Transit Authority (CCRTA), and the Martha’s Vineyard Regional Transit Authority
(VTA).
The table and graph below show our findings for these regional transit authorities.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
33
On-Time Cybersecurity Awareness Training Completion Rates for Regional
Transit Authorities: Sample of All Employees
66.7%
25.0%
50.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CATA CCRTA VTA
Regional
Transit
Authority
On-Time Training
Completion
Percentage*
Total Number of
Employees
Tested
Number of Tested
Employees Who
Completed Training
Late
Number of Tested
Employees Who Did
Not Complete Training
CATA 66.7% 3 — 1
CCRTA 25.0% 8 — 6
VTA 50.0% 8 — 4
* Note that this table is based on the sample of employees from each regional transit authority, not the population of
employees.
If regional transit authorities do not ensure that all of their employees complete cybersecurity awareness
training, then they may expose themselves to an increased risk of cybersecurity attacks and financial
and/or reputational losses.
Authoritative Guidance
EOTSS’s Information Security Risk Management Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. This course shall be conducted via web-based learning or in
class training and shall be included in the new hire orientation checklist. The New Hire
Security Awareness course must be completed within 30 days of new hire orientation.
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
34
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training. Once implemented, automatic email reminders will be sent
to personnel 12 months after course completion, alerting personnel to annual refresher
training completion deadlines.
Reasons for Issue
Management from each of the following regional transit authorities provided us with the following
reasons for noncompliance:
 CATA management stated that the employee from our finding overlooked the email reminders
for the cybersecurity awareness training and did not know they could complete training after the
due date.
 CCRTA management stated that not all employees participated in the cybersecurity awareness
training, as it was given only to staff members with access to sensitive customer or agency data.
 VTA management stated that some employees did not have computer network access, and
therefore, VTA did not require them to take cybersecurity awareness training.
Recommendations
The aforementioned three regional transit authorities should do the following:
update their cybersecurity awareness training policies to require this training for all employees and
update their cybersecurity training policies to include consequences for non-completion (e.g.,
restriction of access until training is completed).
Auditees’ Responses
CATA
The Cape Ann Transportation Authority agrees with the recommendations.
CCRTA
The [Office of the State Auditor] audit findings are based on a limited compliance review
conducted in accordance with the EOTSS IS.010 cybersecurity policy, which the CCRTA did
not opt to adopt as permitted under the policy (AUTHORITY Section 2, 2.1:
“Notwithstanding any general or special law, rule, regulation, executive order, policy or
procedure to the contrary, all executive department agencies shall, and other state
agencies may, adhere to the policies, procedures and objectives established by the
executive office of technology services and security with respect to activities concerning
information technology.”).
Audit No. 2023-0884-3I Executive Office of Technology Services and Security
Detailed Audit Findings with Auditee’s Response
35
VTA
VTA stated that 4 of the 8 employees selected did not have computer network access as
part of their job duties.
Auditor’s Reply
We appreciate the responses provided by the regional transit authorities we audited. The issue we
identified is that these regional transit authorities did not consistently provide cybersecurity training to
their employees. We regard EOTSS’s Information Security Risk Management Standard (IS.010) as the
baseline for best practices in cybersecurity awareness training across the Commonwealth’s agencies, and
therefore we used this as our audit criteria. Per Generally Accepted Government Auditing Standards 8.18,
examples of criteria include: (C) technically developed standards or norms, (f) defined business practices,
and (h) benchmarks for performance comparison, including those of other entities or sectors.
We also note here that EOTSS’s Information Security Risk Management Standard IS.010 is applicable to
the use of information systems and resources by all Commonwealth agencies within the executive branch,
encompassing, as it states, “all executive offices, and all boards, commissions, agencies, [and]
departments.” This EOTSS standard is designed to safeguard information and serves as a minimum
requirement for cybersecurity awareness training.
Regarding training employees who do not have computer network access, we maintain that EOTSS’s
Information Security Risk Management Standard IS.010 states, “All new personnel must complete an
Initial Security Awareness Training course” and that EOTSS does not provide an exemption to this policy
for employees who lack access to computer systems. We urge the regional transit authorities to
implement an alternative method to complete training for employees without system access, such as
offering a paper-based training option.
As noted above within the auditees’ responses, many RTAs have already started addressing our concerns
in this area.

Official Audit Report – Issued June 9, 2023
Committee for Public Counsel Services
For the period January 1, 2019 through December 31, 2021
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
June 9, 2023
Anthony Benedetti, Chief Counsel
Committee for Public Counsel Services
75 Federal Street, 6th Floor
Boston, MA 02110
Dear Mr. Benedetti:
I am pleased to provide to you the results of the enclosed performance audit of the Committee for Public
Counsel Services. As is typically the case, this report details the audit objectives, scope, methodology,
findings, and recommendations for the audit period, January 1, 2019 through December 31, 2021. As you
know, my audit team discussed the contents of this report with agency managers. This report reflects
those comments.
I appreciate you and all your efforts at the Committee for Public Counsel Services. The cooperation and
assistance provided to my staff during the audit went a long way toward a smooth process. Thank you for
encouraging and making available your team. I am available to discuss this audit if you or your team have
any questions.
Sincerely,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2022-1104-3J Committee for Public Counsel Services
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 6
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 11
1. The Committee for Public Counsel Services did not ensure that interns receive cybersecurity awareness
training. .................................................................................................................................................... 11
2. CPCS did not have a business continuity and disaster recovery plan......................................................... 12
3. CPCS’s internal control plan was not updated with a 2019 coronavirus component. ................................ 14
Audit No. 2022-1104-3J Committee for Public Counsel Services
List of Abbreviations
ii
LIST OF ABBREVIATIONS
COVID-19 2019 coronavirus
CPCS Committee for Public Counsel Services
CTR Office of the Comptroller of the Commonwealth
EOTSS Executive Office of Technology Services and Security
ICP internal control plan
IT information technology
MMARS Massachusetts Management Accounting and Reporting System
Audit No. 2022-1104-3J Committee for Public Counsel Services
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Committee for Public Counsel Services (CPCS) for the
period January 1, 2019 through December 31, 2021. The objectives of this audit were to determine the
following:
 whether CPCS employees received cybersecurity awareness training and whether employees
signed acknowledgment forms regarding computer usage in accordance with Sections 6.2.3, 6.2.4,
and 6.2.8 of the Executive Office of Technology Services and Security’s (EOTSS’s) Information
Security Risk Management Standard IS.010, effective October 15, 2018;
 whether CPCS updated its business continuity and disaster recovery plan in accordance with
Section 6.1.1.4 of EOTSS’s Business Continuity and Disaster Recovery Standard IS.005, effective
October 15, 2018; and
 whether CPCS updated its internal control plan (ICP), as required by the Office of the Comptroller
of the Commonwealth’s “[2019 Coronavirus, or COVID-19] Pandemic Response Internal Controls
Guidance.”
Below is a summary of our findings and recommendations, with links to each page listed.
Finding 1
Page 11
CPCS did not ensure that interns receive cybersecurity awareness training.
Recommendations
Page 11
1. CPCS should require that interns receive cybersecurity awareness training.
2. CPCS should have a system to document the receipt of emails from interns who watch
the cybersecurity awareness training video.
Finding 2
Page 12
CPCS did not have a business continuity and disaster recovery plan.
Recommendation
Page 12
CPCS should develop, document, and test a business continuity and disaster recovery plan
to implement.
Finding 3
Page 14
CPCS’s ICP was not updated with a COVID-19 component.
Recommendation
Page 14
CPCS should establish policies and procedures to ensure that its ICP is updated annually
and when significant changes occur.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Committee for Public Counsel Services (CPCS) was established by Chapter 673 of the Acts of 1983.
According to its website, CPCS is governed by a 15-member committee “appointed by the Governor, the
Speaker of the House of Representatives, the President of the Senate, and the Massachusetts Supreme
Judicial Court.” The committee is responsible for planning, overseeing, and coordinating criminal and
non-criminal legal services to people who cannot afford an attorney in the Commonwealth.
CPCS senior staff comprises the chief counsel, the director of administration and operations, the general
counsel, the chief financial officer, the chief information officer, the chief human resources officer, the
equity and inclusion director, and the communications director. In addition, the chief counsel is assisted
by five deputy chief counsels, who oversee the legal divisions listed below.
CPCS is composed of five legal and five operations divisions. The legal divisions are Children and Family
Law, Mental Health Litigation, Private Counsel, Public Defender, and Youth Advocacy. The operations
divisions are Administration and Finance, General Counsel, Human Resource, Information Technology,
and Training.
According to CPCS, in addition to the main office at 75 Federal Street in Boston, there are 20 regional
offices in 17 communities1 in the Commonwealth. During our audit period, CPCS had approximately 868
employees2 and 16 unpaid interns.
There were 664,761 new cases assigned to CPCS public defenders and private attorneys in calendar years
2019, 2020, and 2021, which encompass four fiscal years.
1. The 17 communities with regional offices are Boston, Brockton, Fall River, Framingham, Holyoke, Hyannis, Lawrence, Lowell,
Malden, New Bedford, Northampton, Pittsfield, Quincy, Roxbury, Salem, Springfield, and Worcester. Brockton, Roxbury, and
Springfield each have two offices, and the others each have one.
2. This number of employees includes employees who retired or resigned during the audit period.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Overview of Audited Entity
3
CPCS Public Defender New Cases
Division
Fiscal Year
2019
Fiscal Year
2020
Fiscal Year
2021
Fiscal Year
2022
Total New
Assignments
Children and Family Law 2,269 1,486 1,362 1,610 6,727
Mental Health Litigation 1,063 1,123 947 1,086 4,219
Youth Advocacy 1,756 1,728 1122 1,590 6,196
Public Defender 26,443 20,728 16,399 18,964 82,534
Total New Assignments 31,531 25,065 19,830 23,250 99,676
CPCS Private Attorney New Notices of Assignment of Counsel Issued
Division
Fiscal Year
2019
Fiscal Year
2020
Fiscal Year
2021
Fiscal Year
2022 Total New Notices
Children and Family Law 19,626 16,570 15,486 15,845 67,527
Mental Health Litigation 13,601 12,655 12,340 12,771 51,367
Youth Advocacy 5,403 5,266 4,791 5,452 20,912
Private Counsel 127,755 104,802 92,330 100,392 425,279
Total New Assignments 166,385 139,293 124,947 134,460 565,085
Software
CPCS uses over 15 types of software for public attorney billing, case management, office space
reservation, customer service, employee expense management, information technology project
management and help desk, data storage, private counsel billing, and court costs.
Cybersecurity Threat
According to CPCS management, on February 27, 2019, CPCS suffered a cyberattack. The agency was able
to restore its data in 10 business days. The attack was contained, and the system was cleared of any
remaining infected technology. CPCS shut down its intranet and email servers for those 10 business days,
but this did not affect court cases.
During these 10 business days, CPCS worked in conjunction with the Office of the Comptroller of the
Commonwealth (CTR) and hired a consultant to determine the impact of the attack and provide
recommendations. CPCS also hired another consultant to perform a vulnerability assessment.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Overview of Audited Entity
4
Based on the consultants’ recommendations, CPCS applied new protective measures for patching,
detecting breaches of, and monitoring their data centers and network. Information security awareness
and training were improved for all employees.
Cybersecurity Awareness Training
According to CPCS’s internal control plan (ICP), dated June 30, 2020, regarding cybersecurity awareness,
CPCS follows the Enterprise Information Security Policies and Standards established by the Executive
Office of Technology Services and Security’s (EOTSS’s) Enterprise Security Office, which are available and
recommended for all Commonwealth agencies to use for guidance.
CPCS requires its employees to sign a Certification of Receipt Personnel Policies Manual form.
Section 5.2.2 of CPCS’s “Personnel Policies Manual” is its acceptable use policy. An acceptable use policy
documents the responsibilities of personnel and that employees must comply with the applicable code of
conduct when using Commonwealth-owned IT systems to preserve the confidentiality, integrity, and
availability of CPCS’s information assets.
CPCS uses KnowBe4 software to administer cybersecurity awareness training and phishing tests.3 This
software retains CPCS employees’ test results. Newly hired employees receive cybersecurity awareness
training during orientation. However, interns do not attend orientation and, as of May 2021, CPCS started
requiring interns to watch a cybersecurity awareness training video and email the Human Resource
Department once they have watched the video.
Business Continuity and Disaster Recovery
According to CPCS’s ICP, dated June 30, 2020, regarding its business continuity and disaster recovery plan,
CPCS follows the Enterprise Information Security Policies and Standards established by EOTSS’s Enterprise
Security Office, which are available and recommended for all Commonwealth agencies to use for
guidance.
EOTSS’s Business Continuity and Disaster Recovery Standard IS.005 requires an agency to “establish
procedures for the continuation of critical business processes in the event of any organizational or
3. Phishing is when someone sends an email pretending to be a legitimate business or a person the recipient knows to obtain
sensitive data, such as the recipient’s bank account number. A phishing test lets organizations send a realistic, but fake,
phishing email to employees to see how they respond.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Overview of Audited Entity
5
information technology (“IT”) infrastructure failure and define the related controls and acceptable
practices.”
CTR’s Pandemic Response Guidance
On September 30, 2020, CTR provided guidance for state agencies in response to the 2019 coronavirus
(COVID-19) pandemic. The guidance helped state agencies that were experiencing significant changes
identify their goals, objectives, and risks associated with the COVID-19 pandemic. Objectives included
telework, return-to-work plans, changes to business processes, and safety protocols for staff members
and visitors.
American Rescue Plan Act of 2021
The American Rescue Plan Act of 2021, signed on March 11, 2021, was a federal stimulus bill to aid public
health and economic recovery from the COVID-19 pandemic. On December 17, 2021, CPCS received a
total of $4,500,000 in American Rescue Plan Act of 2021 funds, allocated by Chapter 102 of the
Massachusetts Acts and Resolves of 2021.
Chapter 102 stipulates $2,000,000 to address pandemic-related backlogged cases in CPCS’s Public
Defender Division; $1,000,000 to temporarily fund staffing levels to address an increased need for legal
representation in CPCS’s Children and Family Law Program; and $1,500,000 for the modernization of
CPCS’s billing system. As of the conclusion of the audit period (which was December 31, 2021, two weeks
after the funds were received), CPCS had not expended any of the $4,500,000.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Audit Objectives, Scope, and Methodology
6
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Committee for Public Counsel
Services (CPCS) for the period January 1, 2019 through December 31, 2021.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer, the
conclusion we reached regarding each objective, and where each objective is discussed in the audit
findings.
Objective Conclusion
1. Did CPCS employees receive cybersecurity awareness training and sign
acknowledgment forms in accordance with Sections 6.2.3, 6.2.4, and 6.2.8 of the
Executive Office of Technology Services and Security’s (EOTSS’s) Information Security
Risk Management Standard IS.010, effective October 15, 2018?
No; see Finding 1
2. Did CPCS update its business continuity and disaster recovery plan in accordance with
Section 6.1.1.4 of EOTSS’s Business Continuity and Disaster Recovery Standard IS.005,
effective October 15, 2018?
No; see Finding 2
3. Did CPCS update its internal control plan (ICP) as required by the Office of the
Comptroller of the Commonwealth’s (CTR’s) “[2019 Coronavirus, or COVID-19]
Pandemic Response Internal Controls Guidance”?
No; see Finding 3
To achieve our audit objectives, we gained an understanding of CPCS’s internal control environment
related to the objectives by reviewing applicable agency policies and procedures, as well as conducting
inquiries with CPCS’s staff and management. We evaluated the design of controls over cybersecurity
awareness training, computer use acknowledgment forms, the business continuity and disaster recovery
plan, and the ICP.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Audit Objectives, Scope, and Methodology
7
Acceptable Use Policy
We obtained a list of all employees during the audit period from CPCS. From this list, we selected a
random, nonstatistical sample of 60 CPCS employees from a population of 884. We requested copies of
the 60 employees’ signed Certification of Receipt Personnel Policies forms (including computer usage)
from CPCS’s Human Resource Department and verified that there was a signature on each user’s form to
ensure that all users had signed forms and acknowledged the policies.
Cybersecurity Awareness Training
During our data reliability assessment, we tested all 23 CPCS employees, including 5 hired during the audit
period, who had access to the Massachusetts Management Accounting and Reporting System (MMARS)
during our audit period to ensure that they had received cybersecurity awareness training.
In addition, to determine whether CPCS newly hired employees without access to MMARS received
cybersecurity awareness training within 30 days of new-hire orientation, we selected a nonstatistical,
random sample of 35 CPCS newly hired employees from the list of employees without MMARS access
from a population of 145. For each newly hired employee in our sample, we requested the orientation
date from CPCS’s Human Resource Department, and we requested the cybersecurity awareness training
materials presented to the newly hired employees from CPCS’s Information Technology (IT) Department.
Also, we requested the cybersecurity awareness training certificates from CPCS’s IT Department to
determine whether these newly hired employees received annual cybersecurity awareness training, if
applicable.
To determine whether CPCS’s existing employees (who were hired before the audit period began) with
no MMARS access completed their cybersecurity awareness training, we selected a nonstatistical sample
of 50 existing CPCS employees from a population of 716 (which excludes the 18 existing employees with
MMARS access). For each existing employee in our sample, we examined their cybersecurity awareness
training certificate to determine whether they completed the annual training and the certificate was
documented in KnowBe4.
Phishing
We obtained KnowBe4 test results for all employees who received phishing emails as part of the test
during the audit period. We examined the test results and determined that 218 CPCS employees failed
Audit No. 2022-1104-3J Committee for Public Counsel Services
Audit Objectives, Scope, and Methodology
8
the phishing testing during our audit period. In addition, for employees with MMARS access who failed
the phishing tests, we determined how many times they each failed. We divided the 218 employees who
failed the phishing tests into two strata.
For stratum one, 172 employees failed once, and we targeted the 6 employees with MMARS access and
29 without MMARS access to determine whether these employees took additional cybersecurity
awareness training. We reviewed email notifications for additional training dates, interactive PowerPoint
presentations used for additional training, and cybersecurity awareness training certificates from
KnowBe4. No exceptions were noted with this testing.
For stratum two, there were 46 employees who failed more than once. For this stratum, we selected a
random, nonstatistical sample of 10 employees to determine whether they received additional
cybersecurity awareness training. We reviewed email notifications for additional training dates,
interactive PowerPoint presentations used for additional training, and cybersecurity awareness training
certificates from KnowBe4. No exceptions were noted with this testing.
ICP
We requested CPCS’s ICPs for fiscal years 2020, 2021, and 2022 to determine whether they were updated
with COVID-19 pandemic guidance as required by CTR’s “Internal Control Guide,” because COVID-19 had
caused a significant change to the work environment. We examined a copy of the ICP for fiscal year 2021
to determine whether it contained the components required by CTR’s “COVID-19 Pandemic Response
Internal Controls Guidance.”
Business Continuity and Disaster Recovery Plan
To determine whether CPCS had established a business continuity and disaster recovery plan, we
requested the business continuity and disaster recovery plan from CPCS management. CPCS management
provided an outline of the business continuity and disaster recovery plan that had not been approved by
CPCS management (see Finding 2).
When we used nonstatistical sampling methods for our audit objectives, we did not project the results
from the samples to the populations.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Audit Objectives, Scope, and Methodology
9
Data Reliability Assessment
CPCS Employee List
To determine the completeness and accuracy of the list of all CPCS employees during the audit period
generated from MMARS, we compared this list to an employee list provided by CPCS’s Human
Resource Department and an employee list provided by CPCS’s IT Department. In addition, for each
of these lists, we tested for duplicate data, missing data, and dates outside the audit period. No
exceptions were noted with this testing.
KnowBe4
To assess the reliability of CPCS’s phishing test records from KnowBe4, we tested for missing data,
duplicate data, and dates outside the audit period. For completeness and accuracy, we compared the
names of employees from KnowBe4 to our reconciled employee list.
We assessed the reliability of the cybersecurity awareness training and phishing test records obtained
from KnowBe4 using Service Organization Control reports4 to determine whether there were
exceptions in the testing performed for certain general IT controls (security management, access
control, configuration management, segregation of duties, and contingency planning). In addition, we
reviewed the peer review report of the agency that prepared the Service Organization Control reports.
MMARS
In 2018, the Office of the State Auditor performed a data reliability assessment of MMARS for the
period April 1, 2017 through March 31, 2018. The assessment focused on reviewing selected system
controls, including access controls, cybersecurity awareness, audit and accountability, configuration
management, identification and authentication, and personnel security.
During this audit, we asked CPCS management about the agency’s cybersecurity awareness policy and
personnel security policy and procedures. We requested cybersecurity awareness training certificates
for all 23 employees who had access to MMARS during the audit period.
4. These reports review the effectiveness of internal controls over an organization’s information systems and are conducted by
independent certified public accountants or accounting firms.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Audit Objectives, Scope, and Methodology
10
Based on the results of our data reliability assessments, we determined that the information obtained for
our audit period was sufficiently reliable for the purpose of our audit objectives.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Detailed Audit Findings with Auditee’s Response
11
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Committee for Public Counsel Services did not ensure that interns
receive cybersecurity awareness training.
The Committee for Public Counsel Services (CPCS) employed 24 interns during the audit period, of whom
20 did not receive cybersecurity awareness training.
CPCS is exposed to a higher risk of cybersecurity attacks and financial and/or reputation losses without
educating interns on their responsibility of protecting CPCS’s information by requiring training.
Authoritative Guidance
Section 6.2.3 of the Executive Office of Technology Services and Security’s (EOTSS’s) Information Security
Risk Management Standard IS.010 states, “All new personnel must complete an Initial Security Awareness
Training course. . . . The New Hire Security Awareness course must be completed within 30 days of new
hire orientation.”
Reasons for Issue
CPCS management stated that interns were not required to attend orientation, which included
cybersecurity awareness training. In addition, after implementing the cybersecurity awareness training
video requirement for interns in May 2021, CPCS did not have a system in place to document the receipt
of emails from interns who watched the video.
Recommendations
CPCS should require that interns receive cybersecurity awareness training.
CPCS should have a system to document the receipt of emails from interns who watch the
cybersecurity awareness training video.
Auditee’s Response
CPCS augmented its prior policies and procedures to ensure that all interns receive cybersecurity
awareness training, as it does for all agency employees. CPCS completed both development and
implementation of a new electronic platform to ensure all interns receive cybersecurity awareness
training during onboarding or within 30 days of hire. The electronic platform creates and maintains
a record that cybersecurity training was completed by all interns. CPCS management believes that
all users of its systems, including all short-term summer interns, must be educated concerning the
dangers posed by cybersecurity threats as well as the acceptable use and safeguarding of the
agency’s electronic resources.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Detailed Audit Findings with Auditee’s Response
12
Auditor’s Reply
Based on its response, CPCS has taken measures to address our concerns in this area.
2. CPCS did not have a business continuity and disaster recovery plan.
As of the end of our audit period, CPCS had not developed, documented, or tested a business continuity
and disaster recovery plan for its business and operational objectives, potential risks and exposures, and
the relative importance of the committee’s systems and data.
Without a business continuity and disaster recovery plan, employees may not be sufficiently trained in
performing recovery efforts, including those related to CPCS’s mission-critical applications. In addition,
CPCS has not assessed its ability to continue operations in the event of a business interruption, which
could lead to reputational loss, financial loss, or breach of data.
Authoritative Guidance
Section 6 of EOTSS’s Business Continuity and Disaster Recovery Standard IS.005 states,
Commonwealth Executive Offices and Agencies must establish a Business Continuity Program. . . .
6.1.1.4 Develop business continuity plans (BCP): Each agency shall develop BCPs for critical
business processes based on prioritization of likely disruptive events in light of their
probability, severity and consequences for information security identified through the
[business impact analysis] and risk assessment processes.
Reasons for Issue
CPCS management stated that in fiscal year 2019, the staff member assigned to write the business
continuity and disaster recovery plan took a leave of absence, and in fiscal year 2020, CPCS put out a
request for proposals for a vendor to prepare a business continuity and disaster recovery plan. In fiscal
year 2021, CPCS’s chief information officer resigned before CPCS awarded a contract to a vendor, and as
of the end of our audit period, CPCS was waiting for a new chief information officer to resume the process.
Recommendation
CPCS should develop, document, and test a business continuity and disaster recovery plan to implement.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Detailed Audit Findings with Auditee’s Response
13
Auditee’s Response
CPCS has documented a Continuity of Operations Plan (“COOP”). The COOP was developed by
CPCS senior management in collaboration with consultants with expertise in creating robust
Business Continuity Plans for government agencies. This process was completed last year, and the
formal plan has been presented to and reviewed by our governing board.
Further, senior management will be participating in table-top exercises in March 2023 to test and
enhance our crisis management skills and the agency’s resiliency in the event of a cyber (or other)
attack on agency systems which could impact both the provision of legal services to our clients and
the regular business operations of CPCS. These exercises will help to hone the skills required to
manage the agency during a major crisis or other disruptions with the smallest possible impact on
our clients and staff.
Finally, regarding disaster recovery, the [Information Technology, or IT] Department at CPCS
oversees several information security products and services to monitor and defend against ongoing
cybersecurity risks, including but not limited to:
 Firewall & Endpoint Protection
 Managed Detection and Response
 E-Mail & Web Filtering
 Multifactor Authentication
 Ongoing Cybersecurity Awareness Training and Phishing Testing
 Virtual Information Security Officer Strategic Services
Implementation and ongoing maintenance of these services is sponsored and supported by senior
leadership at the agency.
CPCS currently utilizes IT disaster recovery procedures to guide recovery, including but not limited
to tightly controlled access to the broader internet as well as a multi-level cloud and on-premises
backup strategy.
CPCS’ COOP includes an embedded business impact analysis summary to set strategic expectations
for recovery objectives. As an additional step, the agency is currently working to further formalize
its continuity planning by gathering and integrating business impact analyses from all practice and
operational areas into a set of detailed business continuity plans which will inform the IT disaster
recovery plan expected to be completed by the end of calendar year 2023.
Auditor’s Reply
Based on its response, CPCS is taking measures to address our concerns in this area.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Detailed Audit Findings with Auditee’s Response
14
3. CPCS’s internal control plan was not updated with a 2019 coronavirus
component.
CPCS’s internal control plan (ICP) was not updated with a 2019 coronavirus (COVID-19) component as
required by the Office of the Comptroller of the Commonwealth’s (CTR’s) “COVID-19 Pandemic Response
Internal Controls Guidance,” issued September 30, 2020. CPCS’s ICP was last updated in June 2020.
The absence of an up-to-date ICP may hinder CPCS from identifying vulnerabilities that could prevent it
from achieving its mission to provide legal assistance to those in need and ensure equal access to legal
representation to Massachusetts residents.
Authoritative Guidance
CTR’s “COVID-19 Pandemic Response Internal Controls Guidance,” dated September 30, 2020, states,
Department internal control plans must be based on risk assessments and updated annually, or
when significant changes occur. Because the COVID-19 Pandemic has affected all departments,
The Comptroller, in consultation with the State Auditor’s Office, is providing two options for
updating internal controls.
1. If the impact to your department is such that it can be reflected in your Internal Control
Plan (ICP), then update the ICP as you would for any other mid-year changes.
2. Departments experiencing a significant impact, and requiring the accumulation of
Substantial documentation (e.g. changes to business processes, requirements of
federal and state-specific laws or guidance, new funds or new programs), can draft a
separate COVID-19 Pandemic Response Plan Appendix to the ICP as an organized set
(hard or soft copies) of emails, documents, risk assessments, policies, and procedures.
CTR’s “Internal Control Guide,” revised June 25, 2015, states, “Accordingly, departments are obligated to
revise their ICPs whenever significant changes occur in objectives, risks, management structure, program
scope, etc. At the very least, the ICP must be reviewed and updated annually.”
Reasons for Issue
CPCS did not have policies and procedures to ensure that its ICP is updated annually and when significant
changes occurred.
Recommendation
CPCS should establish policies and procedures to ensure that its ICP is updated annually and when
significant changes occur.
Audit No. 2022-1104-3J Committee for Public Counsel Services
Detailed Audit Findings with Auditee’s Response
15
Auditee’s Response
CPCS has updated the agency’s Internal Control Plan. Further, CPCS has established systems,
policies, and procedures to ensure that the agency’s ICP is updated annually and when significant
changes occur.
The most recent update to the Internal Control Plan was developed by CPCS management and was
completed last year after undertaking a detailed risk assessment. Management believes internal
controls are fundamental to achieving the mission, goals, and objectives of the agency and to
mitigate potential risks. Further, CPCS management believes internal controls are an integral part
of the agency's values and essential to successful program and organizational operations. The
latest version of the ICP has been presented to and reviewed by our governing board.
In specific response to the COVID-19 Pandemic Emergency, the Chief Counsel designated the
Director of Administration and Operations as the CPCS COVID-19 Response leader in March 2020.
The Director oversaw all operational responses to COVID, delivered policy and other guidance via
thirty-three (33) emails to staff, held conference calls with agency managers and directors and
ensured that staff was regularly updated regarding operational and service changes. The Director
also formed a committee of staff from various positions across the Commonwealth to create, review
and recommend COVID policies and procedures which maintained the health and safety of staff
while serving our clients and fulfilling our agency mission.
For example, CPCS quickly made plans for necessary office coverage at our 20 office locations,
adopted systems to ensure that we maintained contact with incarcerated and hospitalized clients
by telephone and mail, and instituted virtual conferences. Each office was required to create a
reopening plan. All staff were trained on CPCS COVID policies and procedures. When the Governor
announced reopening requirements, the agency ensured that COVID Reopening Policies and
Procedures complied with the requirements specific to the type of business conducted in each
office. As the Governor’s reopening requirements were updated, the agency’s policies and
procedures were likewise updated.
Further, senior staff, including the Chief Counsel, Director of Administration and Operations,
General Counsel, Human Resources, IT, and CFO, held virtual town halls with agency managers
and staff to address agency policies and procedures during the pandemic to ensure the continued
zealous representation of clients as well as staff support.
Throughout the pandemic, CPCS continued to zealously represent clients, supported staff, and met
our fiduciary obligations to the Commonwealth as well as our financial obligation to ensure timely
payments to private attorneys and vendors.
Auditor’s Reply
Based on its response, CPCS has taken measures to address our concerns in this area.

Official Audit Report – Issued November 7, 2024
Department of Children and Families
For the period July 1, 2019 through December 31, 2023
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
November 7, 2024
Staverne Miller, Commissioner
Department of Children and Families
600 Washington Street, 6th Floor
Boston, MA 02111
Dear Ms. Miller:
I am pleased to provide to you the results of the enclosed performance audit of the Department of
Children and Families. As is typically the case, this report details the audit objectives, scope, methodology,
findings, and recommendations for the audit period, July 1, 2019 through December 31, 2023. As you
know, my audit team discussed the contents of this report with agency managers. This report reflects
those comments.
I appreciate you and all your efforts at the Department of Children and Families. The cooperation and
assistance provided to my staff during the audit went a long way toward a smooth process. Thank you for
encouraging and making available your team. I am available to discuss this audit if you or your team have
any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
cc: Kate Walsh, Secretary of the Executive Office of Health and Human Services
Maria Z. Mossaides, Child Advocate of the Office of the Child Advocate
Audit No. 2022-1058-3H Department of Children and Families
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ............................................................................................... 11
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 18
1. The Department of Children and Families did not always obtain or renew court approval before children
in its protective custody were administered antipsychotic medications. .................................................. 18
2. The Department of Children and Families did not properly maintain healthcare records in iFamilyNet for
children in its protective custody who received psychotropic medications............................................... 21
a. The Department of Children and Families did not document and/or update psychotropic
medications listed in children’s medical passports. ..............................................................22
b. The Department of Children and Families did not document follow-up doctor appointments
and recommended psychosocial services in iFamilyNet for children in its protective custody
receiving psychotropic medications.....................................................................................25
c. The Department of Children and Families did not document its consent in iFamilyNet for
children in its protective custody to receive psychotropic medications.................................26
3. The Department of Children and Families did not ensure that children received recommended
psychosocial services in conjunction with their prescriptions for psychotropic medications..................... 30
4. The Department of Children and Families did not ensure that all employees with access to COVID-19
funds received annual refresher cybersecurity awareness training........................................................... 32
OTHER MATTERS ................................................................................................................................................. 34
1. The Department of Children and Families should provide more oversight for children in its custody
receiving psychotropic medication in amounts and dosages that exceed United States Food and Drug
Administration recommendations. ........................................................................................................... 34
2. The Department of Children and Families should coordinate with other state agencies, law enforcement,
and other child-serving agencies to address how to detect and respond to human trafficking. ................ 36
3. The Department of Children and Families should collaborate with the Massachusetts Commission on
LGBTQ Youth to implement all recommendations from its annual report. ............................................... 39
Audit No. 2022-1058-3H Department of Children and Families
List of Abbreviations
ii
LIST OF ABBREVIATIONS
AACAP American Academy of Child and Adolescent Psychiatry
DCF Department of Children and Families
EOTSS Executive Office of Technology Services and Security
FDA US Food and Drug Administration
FFS fee-for-service
HHS OIG US Department of Health and Human Services Office of Inspector General
iFN iFamilyNet
MCO managed care organization
MMIS Medicaid Management Information System
OSA Office of the State Auditor
Audit No. 2022-1058-3H Department of Children and Families
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Department of Children and Families (DCF) for the
period July 1, 2019 through December 31, 2023.
In this performance audit, we examined DCF’s process related to psychotropic medications1 prescribed to
children in its protective custody. Specifically, we determined the following:
 whether DCF obtained the required court approval for antipsychotic medications and documented
its consent for psychotropic medications prescribed to children in its protective custody as required
by Sections 11.14(3)(a), (4)(a), and (6)(a) of Title 110 of the Code of Massachusetts Regulations and
DCF Policy 2010-001: Medical Examinations for Children Entering DCF Placement or Custody;
 whether children in DCF’s protective custody received follow-up visits and recommended
psychosocial services in conjunction with prescriptions for psychotropic medications in accordance
with the American Academy of Child and Adolescent Psychiatry’s 2005 “Position Statement on
Oversight of Psychotropic Medication Use for Children in State Custody: A Best Principles Guideline”
and the American Academy of Child and Adolescent Psychiatry’s 2015 Recommendations about the
Use of Psychotropic Medications for Children and Adolescents Involved in Child-Serving Systems;
 whether DCF maintained medical passports2 for children in its protective custody who received
psychotropic medications according to DCF Policy 85-003: Health Care Services to Children in
Placement, DCF Policy 86-011: Ongoing Casework and Documentation, DCF Policy 2010-001, and
Section 475 of the Social Security Act; and
 whether DCF provided oversight to children in its protective custody who received psychotropic
medications that exceeded the US Food and Drug Administration’s (FDA’s) recommended
maximum dosages.
In addition to examining DCF’s process related to psychotropic medications prescribed for children in its
protective custody, we determined the following:
 whether DCF updated its internal control plan to address the COVID-19 pandemic as required by
the Office of the Comptroller of the Commonwealth’s “COVID-19 Pandemic Response Internal
Controls Guidance”;
1. Psychotropic medications are used to treat mental health disorders such as schizophrenia, depression, bipolar disorder,
anxiety disorders, and attention deficit / hyperactivity disorder.
2. A medical passport is a record of healthcare services that a child receives or has received, including their current medications,
relevant mental health history, known mental health conditions, treatment programs, and appointments.
Audit No. 2022-1058-3H Department of Children and Families
Executive Summary
2
 whether DCF ensured that employees who had access to COVID-19 funds completed
cybersecurity awareness training in accordance with the Executive Office of Technology Services
and Security’s Information Security Risk Standard IS.010; and
 whether DCF made outreach efforts to ensure that it reached eligible youths who aged out of DCF
care to allocate funds from the John H. Chafee Foster Care Independence Program grant as
required by the grant agreement ACYF-CB-PI-21-04.
Below is a summary of our findings, the effects of those findings, and our recommendations, with links to
each page listed.
Finding 1
Page 18
DCF did not always obtain or renew court approval before children in its protective
custody were administered antipsychotic medications.
Effect If DCF does not obtain or renew court approvals for antipsychotic medications, which
includes presenting treatment plans to the courts, it cannot ensure that these treatment
plans are safe and appropriate for the children. In addition, this is removing the courts’
oversight of children in DCF protective custody, who are too young to consent to their
treatment plans and need a neutral, third party to ensure that any prescribed medications
are in the children’s best interest.
Recommendation
Page 19
DCF should add monitoring controls to its policies and procedures to ensure that any
Rogers guardianship orders are approved and renewed by the court.
Finding 2a
Page 22
DCF did not document and/or update psychotropic medications listed in children’s medical
passports.
Effect Without accurate and complete information, DCF and health providers may make
decisions that conflict with existing medical treatments or do not reflect children’s best
interests, such as overprescribing psychotropic medications, which can lead to adverse
side effects.
Finding 2b
Page 25
DCF did not document follow-up doctor appointments and recommended psychosocial
services in iFamilyNet (iFN) for children in its protective custody receiving psychotropic
medications.
Effect If DCF does not of not keep accurate and complete medical records in iFN, then children
in DCF’s protective custody may not receive the services needed to treat their conditions.
This may delay the growth, development, or recovery of the children who did not receive
needed care. Failure to keep accurate and complete medical records may also prevent DCF
from determining which medical treatments or providers are most effective or costefficient for serving the medical needs of children in its care.
Finding 2c
Page 26
DCF did not document its consent in iFN for children in its protective custody to receive
psychotropic medications.
Effect Without documentation of consent or court approval for prescriptions of psychotropic
medications, DCF cannot ensure that its social workers and/or medical social workers are
providing children in DCF protective custody with medical treatment that is legally
required.
Audit No. 2022-1058-3H Department of Children and Families
Executive Summary
3
Recommendation
Page 28
DCF should establish sufficient monitoring controls to ensure that children in its protective
custody have up-to-date and accurate health records in iFN and that its social workers
prevent these children from receiving medical care without approval, including the
following:
 DCF should review medical passports for children in its protective custody and
update them at least every six months or when there are changes to a child’s
prescription (e.g., new prescriptions, dosage changes, or discontinued
prescriptions).
 DCF should update iFN with all follow-up doctor appointments and psychosocial
services for children in its protective custody, including the type and frequency
of these appointments and services.
 DCF should document its consent for psychotropic medication for children in its
protective custody in iFN and store that consent in the same location in iFN for
quick and accurate reviews.
Finding 3
Page 30
DCF did not ensure that children received recommended psychosocial services in
conjunction with their prescriptions for psychotropic medications.
Effect If children do not receive the recommended therapy and psychosocial services with
psychotropic medications, treatment effectiveness can be negatively affected. Further
DCF cannot monitor the effectiveness of these medications and cannot identify and
mitigate any side effects that these children may experience. For example, 28 children
from both our samples had suicidal ideations.
Recommendation
Page 31
DCF should ensure that all children in its protective custody who are prescribed
psychotropic medications receive psychosocial services and DCF should implement
sufficient monitoring controls to ensure that these services are provided and that the
efficacy of these services is evaluated.
Finding 4
Page 32
DCF did not ensure that all employees with access to COVID-19 funds received annual
refresher cybersecurity awareness training.
Effect If DCF does not ensure that all its employees complete cybersecurity awareness training,
then it is exposed to a higher-than-acceptable risk of cyberattacks and financial and/or
reputational losses.
Recommendation
Page 32
DCF should develop and implement policies, procedures, and controls to ensure that all
its employees complete cybersecurity awareness training.
In addition, we identified an issue regarding DCF oversight of children in its custody receiving psychotropic
medication in amounts that exceeded the FDA’s recommended doses. For more information on this issue,
see the “Other Matters” section of this report.
During our audit, additional areas of concern that were outside the original scope of our objectives came
to our attention. Given the high-risk nature of these areas, we examined them while we were still engaged
with the auditee. These areas include human trafficking prevention measures, as well as DCF’s
implementation of recommendations by the Massachusetts Commission on LGBTQ Youth. The results of
this work are included within the “Other Matters” section of this audit report.
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
4
OVERVIEW OF AUDITED ENTITY
The Department of Children and Families (DCF), established by Section 1 of Chapter 18B of the
Massachusetts General Laws, provides services to children ages 0 through 21 who are at risk or who have
been victims of abuse or neglect, as well as their families.
According to its website, DCF “works in partnership with families and communities to keep children safe
from abuse and neglect.”
DCF services include adoption, guardianship, foster care, housing stabilization, and family support and
stabilization. DCF has a central office in Boston and four regional offices administered by regional directors
who oversee 29 local-area offices.
In fiscal year 2020, DCF provided support and services to approximately 48,000 children between the ages
of 0 and 21. DCF had an annual appropriation of approximately $1.05 billion for fiscal year 2020 and an
annual appropriation of approximately $1.09 billion for fiscal year 2021.
Federal law requires Massachusetts to have a plan for overseeing and coordinating healthcare services
for any child in foster care placement. According to Section 422(b)(15)(A)(v) of the Social Security Act, this
plan must include “an outline of the oversight of prescription medicines, including protocols for the
appropriate use and monitoring of psychotropic medications.”
DCF’s Protective Custody
Children are referred to DCF for services in several ways. Section 51A of Chapter 119 of the General Laws
requires professionals whose work brings them into contact with children to be designated as mandated
reporters. Mandated reporters are required to make an immediate oral report, and a subsequent written
report (called a 51A report), to DCF when, in their professional capacity, they have reasonable cause to
believe that a child under the age of 18 is suffering from abuse and/or neglect. If DCF considers the report
to have merit, it conducts what is called a 51B investigation. For children who are in immediate danger,
DCF can file a care and protection case in the Juvenile Court and request that a judge order the child’s
immediate removal from a household into DCF’s protective custody. Children and families may also come
to DCF’s attention from sources other than 51A reports, including cases referred by the Juvenile Court,
cases referred by the Probate Court, instances of infants surrendered under the Safe Haven Act, and
parents’ or other relatives’ requests for DCF services.
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
5
When a child is removed from a household and is in DCF’s protective custody, they are placed in DCFcontracted or DCF-operated settings, such as foster care, a shelter, a short-term or group care program,
or a community residential care facility.
During fiscal years 2020 and 2021, a total of 17,891 children under the age of 18 were in DCF’s protective
custody.
DCF’s iFamilyNet
DCF implemented the Statewide Child Welfare Information System, known as FamilyNet, in
February 1998. In 2009, DCF moved FamilyNet functionality to the web-based application, iFamilyNet
(iFN). iFN is the system of record for DCF. Starting in May 2016, DCF hired 29 medical social workers—one
for each local-area office—who are responsible for ensuring that each child’s healthcare records, such as
medical appointment information or medical passport information, remain up-to-date in the medical
section of iFN. Social workers input healthcare information for children, including their healthcare
providers, appointment dates, medical conditions, and medications, in the medical section of iFN. In
addition, social workers can upload healthcare records directly to iFN. These electronic healthcare records
enable the social worker to review a child’s healthcare information at any time.
Pediatric Behavioral Health Medication Initiative
The University of Massachusetts Chan Medical School leads the MassHealth Pharmacy Program in
collaboration with DCF and the Department of Mental Health. In 2014, this partnership created the
Pediatric Behavioral Health Medication Initiative to ensure safe and effective prescribing of behavioral
health medications, including psychotropic medications, for MassHealth members who are 18 years old
and younger. This initiative requires prior authorizations from MassHealth for certain behavioral
medication classes and/or specific medication combinations that have limited evidence for safety and
efficacy within the pediatric population. For example, pharmacy claims with any combination of four or
more behavioral health medications within a 45-day period require a prior authorization from
MassHealth.
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
6
Psychotropic Medications
Psychotropic medications are provided to patients with diagnosed mental health disorders. These
medicines may be prescribed to children in protective custody. During the audit period, 3,899 (22%) of
the 17,891 children in DCF’s protective custody were prescribed at least one psychotropic medication.
According to the American Academy of Child and Adolescent Psychiatry’s (AACAP’s) 2005 “Position
Statement on Oversight of Psychotropic Medication Use for Children in State Custody: A Best Principles
Guideline,” “Many children in state custody benefit from psychotropic medications as part of a
comprehensive mental health treatment plan.” For example, these medications can help control mental
health symptoms such as mood swings, anger outbursts, hallucinations, and delusions. Although there are
benefits to prescribing children in protective custody psychotropic medication, it is important that the
medication is only a part of an overall health treatment plan, which should include monitoring the side
effects of these medications and providing mental health services.
According to the US Food and Drug Administration’s (FDA’s) Approved Drug Products with Therapeutic
Equivalence Evaluations, referred to as the Orange Book,3 psychotropic medications can have potentially
serious side effects associated with them, including drowsiness, upset stomach, increased appetite and
weight gain, other metabolic abnormalities, allergic reactions, mania, seizures, low sodium, serotonin
syndrome, and suicidal ideation. According to AACAP’s 2015 Recommendations about the Use of
Psychotropic Medications for Children and Adolescents Involved in Child-Serving Systems, healthcare
professionals should routinely monitor children receiving this class of medication for adverse side effects
like these and avoid prescribing children too many medications.
We used the MassHealth Pediatric Behavior Health Initiative Medication List4 to determine what specific
psychotropic medications were prescribed to children in DCF’s protective custody. These medications
included the following:
 antianxiety and antidepressant medications, such as Zoloft, which are used to treat anxiety or
depression;
3. The Orange Book lists all medications that the FDA has deemed safe and effective. For each medication, the Orange Book
includes information such as side effects, warnings, dosage recommendations, indications, and more.
4. This medication list was created by MassHealth’s Pharmacy Program in collaboration with DCF and the Department of Mental
Health and includes medications prescribed for pediatric behavior health diagnoses.
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
7
 antipsychotic medications, such as Haldol, which are used to treat symptoms of some mental
disorders, including schizophrenia;
 mood stabilizing medications, such as lithium, which are used to treat mood disorders, such as
bipolar disorder;
 stimulants, such as Adderall, which are used to treat attention deficit / hyperactivity disorder; and
 other medications, such as Armodafinil, which are used to treat excessive drowsiness and/or
narcolepsy.
The table below shows the breakdown of psychotropic medication types and how many prescriptions for
each were filled for children in DCF’s protective custody who were in a fee-for-service (FFS) plan.
Medication Type Number of Prescriptions Filled
Antianxiety 1,065
Antidepressant 21,585
Antipsychotic 10,564
Mood stabilizer 10,776
Stimulant* 48,453
Other 1,244
Total 93,687
* Stimulants in this list included some hypertension medications that are also used to treat attention deficit / hyperactivity
disorder. These medications were listed on the MassHealth Pediatric Behavior Health Initiative Medication List for this
reason.
Psychotropic Medication Consent
When a healthcare provider recommends prescribing psychotropic medication(s) to a child in DCF’s
protective custody, DCF must consent to the prescription. A social worker engages with the child in
protective custody and the foster family or residential facility providing care regarding the prescribing
provider’s recommendations, and the social worker, doctor, and caregiver (i.e., foster family or program
provider) together develop a plan for the child’s well-being. Additionally, the social worker should
document DCF’s consent for the use of psychotropic medications, should ensure that the prescription(s)
are filled, and should document the consent in iFN throughout the child’s time in DCF’s protective custody.
Rogers Guardianship Order
Rogers guardianship order proceedings are named for the 1983 Massachusetts court case, Rogers v.
Commissioner of Mental Health, in which the court stated that antipsychotic medications are so intrusive,
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
8
and their side effects are potentially so severe, that a court must review the treatment plan and approve
their prescription and use for children in DCF’s protective custody.
In order to apply for a Rogers guardianship order, DCF must present the court with a Clinician’s Affidavit
as to Competency and Proposed Treatment Plan from the healthcare professional for the prescription and
administration of the proposed antipsychotic medication. This treatment plan includes the name of the
antipsychotic medication, the dosage, the dosage range, proposed alternative medications, risks of
potential side effects and/or adverse reactions, and the benefits of the medication.
If the judge approves the Rogers guardianship order, the treatment plan presented to the judge remains
in effect for a specified amount of time. After the specified amount of time has lapsed, or at least annually,
the Rogers guardianship order must be reviewed and renewed.
Medical Passport
All children placed in DCF’s protective custody are issued a physical medical passport. A medical passport
is a record of healthcare services that a child receives, including current medications, relevant mental
health history, known mental health conditions, all treatment programs, and appointments. The medical
passport remains with the child and in the possession of the foster family, group home, or residential
facility throughout the child’s time in protective custody or foster care placements. DCF requires its social
workers to review these physical medical passports every six months to keep the children’s related
medical records in iFN updated with their most recent healthcare information.
Psychosocial Services
Psychosocial services are mental health treatment services designed to reduce patients’ emotional or
behavioral symptoms and usually include general therapy, group therapy, and behavioral therapy. Such
therapies may be used instead of, or in combination with, psychotropic medications to treat children with
mental health conditions. AACAP’s 2015 Recommendations about the Use of Psychotropic Medications for
Children and Adolescents Involved in Child-Serving Systems recommends, “All youth with complex
behavioral needs, including youth in foster care, should receive a combination of evidence-based
psychosocial interventions and psychotropic medication when indicated, not just psychotropic medication
alone.”
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
9
FFS and Managed Care Program
MassHealth members can receive benefits on an FFS basis or through a managed care program.
MassHealth directly pays healthcare providers under the FFS model for medical services rendered to an
eligible MassHealth member. Healthcare providers can bill MassHealth directly through the Provider
Online Service Center, which connects to the Medicaid Management Information System, using its
MassHealth-issued provider identification.
MassHealth’s managed care program consists of two managed care organizations (MCOs), Tufts Health
Together and Boston Medical Center HealthNet Plan, both of which provide healthcare services to
members through MCO plans. Each MCO plan assigns members a group of doctors and other healthcare
providers who work together to provide members with coordinated healthcare services. The doctors and
other healthcare providers contractually agree to follow certain federal and state requirements about
how they provide services. MCO plan enrollees select a primary care physician to provide basic healthcare
and make any necessary specialist referrals. MassHealth pays the MCO a capitation payment,5 the amount
of which is based on a rating category assigned by the Executive Office of Health and Human Services, for
each member enrolled in the MCO plan. Rating categories are based on risk factors for each member,
such as whether the member needs facility-based care (e.g., skilled nursing facilities) or behavioral health
treatment.
Office of the Comptroller of the Commonwealth’s Pandemic Response
Guidance
On September 30, 2020, the Office of the Comptroller of the Commonwealth provided guidance in
response to the COVID-19 pandemic for state agencies. This guidance helped state agencies that were
experiencing significant changes to identify their goals, objectives, and risks associated with the COVID19 pandemic. Objectives included the following: telework; return-to-office plans; a risk assessment of the
impact of COVID-19 on department operations; changes to business processes; safety protocols for staff
members and visitors; and tracking of COVID-19-related awards and expenditures, which were tracked
separately from other federal, state, and local expenditures.
5. Medicaid programs make fixed monthly payments to MCOs for members enrolled in its Managed Care Program. Each
payment is made to MCOs to cover the cost of the healthcare services of the member, and the amount of each payment is
based on the healthcare needs of each member.
Audit No. 2022-1058-3H Department of Children and Families
Overview of Audited Entity
10
Cybersecurity Awareness Training
The Executive Office of Technology Services and Security (EOTSS) has established policies and procedures
that apply to all Commonwealth agencies within the executive branch. Section 6.2 of EOTSS’s Information
Security Risk Management Standard IS.010 states,
The objective of the Commonwealth information security training is to educate users on their
responsibility to help protect the confidentiality, availability and integrity of the Commonwealth’s
information assets. Commonwealth Offices and Agencies must ensure that all personnel are trained
on all relevant rules and regulations for cybersecurity.
To ensure that employees are clear on their responsibilities, EOTSS’s policies require that all employees
in state executive branch agencies complete a cybersecurity awareness training course every year. All
newly hired employees must complete an initial security awareness training course within 30 days of their
orientation.
John H. Chafee Foster Care Independence Program Grant
DCF received $7.9 million from the federal government through the John H. Chafee Foster Care
Independence Program to give direct financial assistance to help current and former foster children
recover from the pandemic. The Supporting Foster Youth and Families through the Pandemic Act
prohibited states from allowing children to age out6 of foster care during the pandemic. This law also
allocated money to distribute to young people in foster care and to former foster children who had aged
out of the system before the pandemic but were still under the age of 26. In the five-year Child and Family
Services Prevention Plan that Massachusetts submits to the federal government, the Commonwealth
commits to providing support to the child welfare system to promote the safety and well-being of children
within the Commonwealth.
6. On a child’s 18th birthday, they become a legal adult and can decide whether they want to stay in DCF care. If they decide to
stay, DCF continues to provide services to them, including helping design a transition plan for them; providing them a safe,
affordable place to live; getting them important documents such as photo identification, a Social Security card, and their birth
certificate; and helping them find local health services until they are 21 years old.
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
11
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor (OSA) has conducted a performance audit of certain activities of the Department of Children and
Families (DCF) for the period July 1, 2019 through December 31, 2023.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Did DCF obtain required court approval for antipsychotic medications and document
its consent for psychotropic medications prescribed to children in its protective custody
as required by Sections 11.14(3)(a), (4)(a), and (6)(a) of Title 110 of the Code of
Massachusetts Regulations and DCF Policy 2010-001: Medical Examinations for
Children Entering DCF Placement or Custody?
No; see Findings 1
and 2c
2. Did children in DCF’s protective custody receive follow-up visits and recommended
psychosocial services in conjunction with prescriptions for psychotropic medications in
accordance with the American Academy of Child and Adolescent Psychiatry’s 2005
“Position Statement on Oversight of Psychotropic Medication Use for Children in State
Custody: A Best Principles Guideline” and the American Academy of Child and
Adolescent Psychiatry’s 2015 Recommendations about the Use of Psychotropic
Medications for Children and Adolescents Involved in Child-Serving Systems?
No; see Findings 3
and 2b
3. Did DCF maintain medical passports for children in its protective custody who received
psychotropic medications according to DCF Policy 85-003: Health Care Services to
Children in Placement, DCF Policy 86-011: Ongoing Casework and Documentation, DCF
Policy 2010-001, and Section 475 of the Social Security Act?
No; see Finding 2a
4. Did DCF provide oversight to children in its protective custody who received
psychotropic medications that exceeded the US Food and Drug Administration’s
(FDA’s) recommended maximum dosages?
No; see Other
Matters
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
12
Objective Conclusion
5. Did DCF update its internal control plan to address the COVID-19 pandemic as required
by the Office of the Comptroller of the Commonwealth’s “COVID-19 Pandemic
Response Internal Controls Guidance,” and did DCF ensure that employees who had
access to COVID-19 funds completed cybersecurity awareness training in accordance
with the Executive Office of Technology Services and Security’s (EOTSS’s) Information
Security Risk Standard IS.010?
No; see Finding 4
6. Did DCF make outreach efforts to ensure that it reached eligible youths who aged out
of DCF care to allocate funds from the John H. Chafee Foster Care Independence
Program grant as required by the grant agreement ACYF-CB-PI-21-04?
Yes
To accomplish our audit objectives, we gained an understanding of the aspects of DCF’s internal control
environment relevant to our objectives by reviewing its internal control plan and applicable policies and
procedures, as well as by interviewing DCF employees and management.
To obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures
described below.
Sampling Strategies for Children in DCF’s Protective Custody
DCF provided us with a list of all 3,899 children in its protective custody who received at least one
psychotropic medication during the audit period. We separated this list based on whether the child was
enrolled in a fee-for-service (FFS) plan (3,204 children) or managed care organization (MCO) plan
(695 children). This separation allowed us to further refine the FFS population based on our detailed access
to claim information.
 First, we removed claims for children who were receiving psychotropic medications that exceeded
the FDA’s recommended maximum dosages, which totaled 299 children, bringing the population
from 3,204 to 2,905 (see the “Maximum Dosages” section of this report).
 We removed the claims for children in the FFS plan population of 2,905 who had fewer than 24
claims for psychotropic medications (meaning they were not prescribed the medication during
the length of the audit period). This gave us a population of 1,180 children in DCF’s protective
custody with an FFS plan who consistently received at least one psychotropic medication during
the audit period.
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
13
For Objectives 1, 2, and 3, we selected a random, statistical sample7 of 118 out of 1,180 children in DCF’s
protective custody enrolled in an FFS plan and who were prescribed at least one psychotropic medication,
using a 95% confidence level,8 a 50% expected error rate,9 and a 15% desired precision range.10
In addition, we selected a random, nonstatistical sample11 of 50 out of 695 children in DCF’s protective
custody enrolled in a managed care plan who were prescribed at least one psychotropic medication.
For Objective 4, we selected a random, nonstatistical sample of 40 out of 299 children in DCF’s protective
custody in an FFS plan who, based on Medicaid Management Information System (MMIS) data, received
psychotropic medications that exceeded the FDA’s recommended maximum dosages.
Rogers Guardianship Orders and Department Consent
To determine whether DCF obtained required court approval for antipsychotic medications and
documented its consent for psychotropic medications prescribed to children in its protective custody, we
took the following actions:
We met with DCF officials to go through its iFamilyNet (iFN) system to gain an understanding of where all
the Rogers guardianship orders should be located in iFN. Next, we reviewed the MMIS All Services Report12
for each child listed in both our FFS and MCO samples and identified the claims for psychotropic
medication(s) for these children to determine whether each child was prescribed an antipsychotic and/or
another type of psychotropic medication. (As previously stated, any child prescribed an antipsychotic
medication required a Rogers guardianship order, and any child prescribed another psychotropic
7. Auditors use statistical sampling to select items for audit testing when a population is large (usually over 1,000) and contains
similar items. Auditors generally use a statistics software program to choose a random sample when statistical sampling is
used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make
conclusions or projections about entire populations.
8. Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are
representative of the population (parameter), expressed as a percentage.
9. Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the
auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.
10. Desired precision range is the range of likely values within which the true population value should lie; also called confidence
interval. For example, if the interval is 90%, the auditor will set an upper confidence limit and a lower confidence where 90%
of transactions fall within those limits.
11. Auditors use judgmental (i.e., nonstatistical) sampling to select items for audit testing when a population is very small, the
population items are not similar enough, or there are specific items in the population that the auditors want to review.
Auditors use their knowledge and judgment to select the most appropriate sample. For example, an auditor might select
items from areas of high risk. The results of testing using judgmental sampling cannot be used to make conclusions or
projections about entire populations; however, they can be used to identify specific issues, risks, or weaknesses.
12. Each MMIS All Services Report documents the healthcare services that MassHealth has paid for a specific member, including
pharmacy services.
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
14
medication required DCF consent. We determined whether Rogers guardianship orders and/or DCF
consent forms were included in iFN. In addition, for the Rogers guardianship orders not documented in
iFN, we requested and reviewed hard copies of the Rogers guardianship orders from DCF, who reached
out to juvenile courts in order to provide them to us.
Based on the results of our testing, we determined that, during the audit period, DCF did not always obtain
or renew required court approval before children in its protective custody were administered
antipsychotic medications and did not properly maintain Rogers guardianship orders in iFN. See Findings 1
and 2c for more information.
Psychosocial Services and Follow-up Visits
To determine whether children in DCF’s protective custody who were prescribed psychotropic
medications received follow-up doctor appointments and recommended psychosocial services, we took
the following actions:
 We inspected the MMIS All Services Reports for the children in both our FFS and MCO samples
and identified all psychosocial service claims and follow-up doctor appointment claims.
 We compared the services in the MMIS All Services Reports to what DCF documented in iFN by
searching social worker notes, encounter forms, and medical passports for evidence that the
children in our samples received psychosocial services and follow-up doctor appointments in
conjunction with prescribed psychotropic medications.
Based on the results of our testing, we determined that, during the audit period, children did not always
receive recommended psychosocial services in conjunction with their prescriptions for psychotropic
medications and DCF did not properly maintain healthcare records in iFN. See Findings 2b and 3 for more
information.
Medical Passports
To determine whether DCF maintained medical passports for children in its protective custody who
received psychotropic medications, we took the following actions. We inspected the medical passport in
iFN for each child in both our FFS and MCO samples to determine whether the medications and dosages
listed on each medical passport matched the medications and dosages in each child’s MMIS All Services
Report.
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
15
Based on the results of our testing, we determined that DCF did not properly maintain healthcare records
in iFN during the audit period. See Finding 3 for more information.
Maximum Dosages
To determine whether DCF provided oversight to children in its protective custody who received
psychotropic medications that exceeded the FDA’s recommended maximum dosages, we reviewed each
child’s full medical record in iFN for the sample of 40 and determined whether children were over the
maximum dosage. We further determined whether DCF documented its consent for the psychotropic
medication and the reasoning for the high dosage.
Based on the results of our testing, we determined that DCF should provide more oversight for children
in its custody receiving psychotropic medication in amounts and dosages that exceed the FDA’s
recommendations. See Other Matters for more information.
Internal Control Plan and Cybersecurity Awareness Training
To determine whether DCF updated its internal control plan to address the COVID-19 pandemic, as
required by the Office of the Comptroller of the Commonwealth’s “COVID-19 Pandemic Response Internal
Controls Guidance,” we reviewed DCF’s fiscal year 2020 internal control plan.
To determine whether DCF ensured that employees who had access to COVID-19 funds during the audit
period completed annual refresher and/or initial cybersecurity awareness training in accordance with
EOTSS’s Information Security Risk Management Standard IS.010, we obtained and inspected transcript
reports of cybersecurity awareness training for all 10 employees who had access to COVID-19 funds.
Based on the results of our testing, we determined that, during the audit period, DCF updated its internal
control plan to address the COVID-19 pandemic. However, DCF did not ensure that all employees with
access to COVID-19 funds received annual refresher cybersecurity awareness training during the audit
period. See Finding 4 for more information.
John H. Chafee Foster Care Independence Program Grant
To determine whether DCF made outreach efforts to ensure that it reached eligible youths who aged out
of DCF care in order to allocate funds from the John H. Chafee Foster Care Independence Program grant,
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
16
we interviewed DCF management and requested and reviewed DCF’s plan for reaching out to youths
eligible for these funds.
We noted no exceptions in our testing; therefore, we determined that, during the audit period, DCF made
outreach efforts to ensure that it reached eligible youths who left DCF care to allocate funds from the
John H. Chafee Foster Care Independence Program grant.
We used a combination of statistical and nonstatistical sampling methods for testing, and we did not
project the results of our testing to any corresponding populations.
Data Reliability Assessment
We received from DCF a list of children in DCF’s protective custody who were prescribed at least one
psychotropic medication during the audit period and separated the list by children who were enrolled in
an FFS plan and children who were enrolled in an MCO plan. We performed validity and integrity tests
first on the list of children enrolled in an FFS plan, including (1) testing for blank fields, (2) scanning for
duplicate records, and (3) tracing a sample of 20 children on the list to protective custody court
documents. We also performed validity and integrity testing on the children enrolled in a MCO plan,
including (1) testing for blank fields, (2) scanning for duplicate records, and (3) tracing a sample of 25
children on the list to their MCO plan from MMIS. To verify the completeness of the list provided by DCF,
we attempted to extract a list from MMIS of all children in DCF’s protective custody who were prescribed
at least one psychotropic medication. However, MMIS does not distinguish different levels of custody,
and the focus of our audit objective was children in DCF’s protective custody. Therefore, we used the list
provided by DCF as it was the only source of data available.
To determine the reliability of the data from MMIS, we relied on the work performed by OSA, completed
in 2020, that tested certain information system controls in MMIS. As part of that work, OSA reviewed
existing information, tested selected system controls, and interviewed knowledgeable agency officials
about the data. As part of this audit, we performed validity and integrity tests on all MMIS data, including
(1) testing for blank fields, (2) scanning for duplicate records, (3) looking for dates outside of the audit
period, and (4) determining whether each child’s age on the date of service was between 0 and 17.
We also received from DCF a list of employees who had access to COVID-19 funds. We performed validity
and integrity tests on this list, including (1) testing for duplicates, (2) testing for blank fields, and (3) tracing
Audit No. 2022-1058-3H Department of Children and Families
Audit Objectives, Scope, and Methodology
17
the list back to a list that we generated from the Human Resources Compensation Management System,
which is the Commonwealth’s official payroll system.
Based on the results of the data reliability assessment procedures described above, we determined that
the information we obtained was sufficiently reliable for the purposes of our audit.
Other Matters
During this audit, additional areas of concern that were outside the original scope of our objectives came
to our attention. Given the high-risk nature of these areas, we examined them while we were still engaged
with the auditee. These areas include human trafficking prevention measures, as well as DCF’s
implementation of recommendations by the Massachusetts Commission on LGBTQ Youth.
We emailed DCF to determine what corrective measures the agency took, or was taking, to address the
findings and concerns referenced in the US Department of Health and Human Services Office of Inspector
General (HHS OIG) report that cited Massachusetts as one of five states where there was no evidence that
children in foster care were screened for human trafficking after they had gone missing from, and later
returned to, foster care. We also inquired about several related follow-up questions with DCF that
addressed the detection and prevention of human trafficking.
In addition, we inquired about the Massachusetts Commission on LGBTQ Youth’s 2023 annual report with
DCF and whether it had implemented the recommendations
The results of this work are included within the “Other Matters” section of this audit report.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
18
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Department of Children and Families did not always obtain or renew
court approval before children in its protective custody were administered
antipsychotic medications.
The Department of Children and Families (DCF) did not always obtain or renew required court approval
before children in its protective custody were administered antipsychotic medications. Our fee-for-service
(FFS) and managed care organization (MCO) samples included 36 children13 who were prescribed one or
more antipsychotic medications, therefore requiring a Rogers guardianship order. Of these 36 children,
we found that 4 children14 were administered antipsychotic medication without a required Rogers
guardianship order from the court.
We also found that DCF did not always obtain court approval for antipsychotic medications in a timely
manner. Specifically, we determined that six15 children in our samples were administered antipsychotic
medications through an expired Rogers guardianship order. One of these children received medication
for eight months past the expiration date for their Rogers guardianship order and without an updated
order.
If DCF does not obtain or renew court approvals for antipsychotic medications, which includes presenting
treatment plans to the courts, it cannot ensure that these treatment plans are safe and appropriate for
the children. In addition, this is removing the courts’ oversight of children in DCF protective custody, who
are too young to consent to their treatment plans and need a neutral, third party to ensure that any
prescribed medications are in the children’s best interest.
Authoritative Guidance
According to Section 11.14 of Title 110 of the Code of Massachusetts Regulations,
(2) No Consent by Department. The Department shall not consent to the administration of
antipsychotic medication for any individual, but shall in all cases seek . . . prior judicial approval
for children in Department custody and wards of the Department. . . .
13. This number represents 25 children needing Rogers guardianship orders from our FFS sample of 118 and 11 children needing
Rogers guardianship orders from our MCO sample of 50. The remaining members in each sample did not require a Rogers
guardianship order, as they were not prescribed an antipsychotic medication.
14. This number represents one child from our FFS sample and three children from our MCO sample.
15. This number represents five children from our FFS sample and one child from our MCO sample.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
19
(4) Judicial Approval for Wards and Children in Department Custody.
(a) When any individual, organization, facility, or medical provider seeks the Department’s
consent to medicate with antipsychotic drugs a child, who is a ward of the Department
or who is in Department custody, the Department shall seek prior judicial approval for
administration of such drugs even if the child’s biological parents have consented to
the medication.
Reasons for Issues
DCF indicated in an email to us on July 21, 2023 the following reasons for the issues with the Rogers
Guardianship Orders:
 There were delays with the court because of the COVID-19 pandemic.
 The child was only in protective custody for 72 hours (although iFamilyNet [iFN] indicated
different and no supporting evidence was provided to us).
 DCF was unaware the child was taking the antipsychotic medication.
 The child never took the medication, but the prescription was filled.
In addition to the above reasons, DCF’s policies and procedures do not include monitoring controls to
ensure that DCF applies, and receives approval, for Rogers guardianship order and renews them in a timely
manner.
Recommendation
DCF should add monitoring controls to its policies and procedures to ensure that any Rogers guardianship
orders are approved and renewed by the court.
Auditee’s Response
DCF agrees with this recommendation and will make improvements to its electronic case records
system to better manage and track dates of court approvals for antipsychotic medications. In
Review of the case records for the 4 children identified as having no Rogers Orders issued shows
that the Department adhered to existing regulations and policies for 2 of the children, as we have
outlined below:
 The child was not in protective custody. The Department followed existing
regulation which governs the administration of medication when children come into
custody and shortly thereafter was returned to the parents’ custody obviating the need
for an order. Child came into DCF custody on previously prescribed antipsychotic
medication on December 3, 2019. Per Departmental regulation [Section 11.14(4)(b)
of Title 110 of the Code of Massachusetts Regulations]: “Where antipsychotic
medications have been previously prescribed for a child who is a ward of the
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
20
Department or who is in the custody of the Department, and that child is currently
being treated with antipsychotic drugs without judicial authorization, the Department
shall initiate the process for judicial review and application of substituted judgment.
Pending judicial review the Department shall not discontinue the prescribed treatment
with antipsychotic drugs, because interruption or discontinuance of the treatment
might cause severe medical complications and might violate the individual's legal right
to treatment.” The Department adhered to this regulation, and then the child was
returned to parents’ custody on December 19, 2019 after a custody hearing. Therefore,
no Rogers order entered; the regulation required DCF to maintain the child on
medication, however because the child was in DCF custody only 16 days, judicial
review of the medication was not possible prior to the child returning to parents’
custody.
 The child never took the medication, but the prescription was filled. The child
was never placed on antipsychotic medication, although it was discussed with the
provider. Therefore, no Rogers Order entered. In support of this fact, the Department
provided a screen shot of the Social Worker’s dictation, which stated: “Once the
provider was told about Rogers being needed, they didn’t write it and the med never
started.”
 For the other 2 cases, the Department acknowledges that there were delays in
scheduling that should have been mitigated or eliminated.
 With respect to the 6 cases identified as having delayed extensions of existing Rogers
Orders, the Department believes that the [Office of the State Auditor (OSA)] has
incorrectly identified one of the children as having a gap in the Rogers order as
outlined:
 OSA identified a gap in the Rogers orders for a child in placement under
a Child Requiring Assistance (CRA) application. CRA from 8/2019 to 6/2020,
then child returned to parent; C&P custody began in 3/2021. DCF does not consent
for CRA children - parents’ consent. No Rogers would have been issued between
8/2019 and 6/2020 during CRA custody. A Rogers Order is only required for
children in DCF custody as a result of a Care & Protection Petition. - Orders
previously provided to OSA are attached to this response.
 For two other cases, there were situations that involved extenuating circumstances
beyond the Department’s control:
 No gap, but delayed Rogers following emergency prescription due to
parents’ failure to appear in court. Child was prescribed antipsychotic
medication as an emergency while placed in an inpatient hospital at the time that
the Department. The Rogers hearing was scheduled for temporary custody
hearing, but that was continued for 10 days due to parents’ failure to appear in
court. The Rogers hearing was completed at the next court date.
 Gap related to Rogers extension due to failure of Guardian Ad Litem
(GAL) to provide the court with an updated affidavit. Antipsychotic drugs
had been prescribed for some time. Due to issues with the GAL not submitting an
updated affidavit, the judge vacated the Order previously authorized on
11/10/2020, revoked the GAL and appointed a new GAL. Antipsychotic medications
need to be tapered and cannot be stopped abruptly in most cases. Rogers was
reinstated on 12/16/2020 after new GAL appointed.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
21
For the remaining 3 cases, the Department acknowledges that there were delays in scheduling that
should have been mitigated or eliminated.
The Department agrees with the Auditor’s recommendation to establish better monitoring controls
and is working to make several enhancements to the electronic legal case records in iFamilyNet.
One of the suggestions that has been made is to better utilize the Department’s electronic case
filing system to better manage and track dates of judicial approval for prescribed antipsychotic
medications.
Auditor’s Reply
We commend DCF for working on steps to better manage the tracking of court approvals for children
receiving antipsychotic medications.
Regarding the two cases DCF disagrees with us about—it should be noted that, although DCF states that
one child was returned to their parents shortly after entering custody, we were not provided evidence for
when the child was returned. Based on the information in iFN, the child was in protective custody for over
two years. In addition, regarding the child that DCF states never took the medication, but the prescription
was filled, we note that a prescription should not be filled before obtaining a Rogers guardianship order.
An open question remains regarding what happened to the medication that was received but not
consumed. For the one case with a delay in extension for its existing Rogers order that DCF objected to,
we were not provided any supporting documentation to show the child was under CRA. According to iFN
the child has been in custody since August 2019.
2. The Department of Children and Families did not properly maintain
healthcare records in iFamilyNet for children in its protective custody who
received psychotropic medications.
DCF did not properly maintain healthcare records in for children in its protective custody who received
psychotropic medications. During the audit period, 117 of the iFN 118 children who were prescribed at
least one psychotropic medication in our FFS sample had incomplete or missing mental health and
psychosocial service information. In addition, 49 of the 50 children who were prescribed at least one
psychotropic medication in our MCO sample had incomplete or missing mental health and psychosocial
service information.
As a result of not updating iFN to reflect up-to-date healthcare records, DCF cannot provide adequate
oversight and ensure that the health and mental health needs of all children in its protective custody are
being met. In addition, DCF and providers could be creating a treatment plan that is not safe or effective
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
22
for a child, because there is important information missing that would affect this child’s healthcare (for
example, a history of depression or scheduled follow-up appointments to check the dosage of a new
medication).
Below is a summary of the specific issues we identified in iFN.
Specific Healthcare Records Number of Documentation
Issues—FFS Plan
Number of Documentation
Issues—MCO
Medical passports 104 out of 118 43 out of 50
Follow-up doctor appointments and
psychosocial services 116 out of 118 46 out of 50
Rogers guardianship order / department
consent for psychotropic medications 109 out of 118 47 out of 50
a. The Department of Children and Families did not document and/or
update psychotropic medications listed in children’s medical passports.
DCF did not list and/or update the psychotropic medications prescribed to children in their medical
passports in iFN. Specifically, 104 of the 118 children who were prescribed at least one psychotropic
medication in our FFS sample had medical passports that did not list any of their psychotropic
medication prescriptions or had incomplete information about their prescriptions. In addition, 43 of
the 50 children in our MCO sample had medical passports that did not list any psychotropic
medication prescriptions or had incomplete information about their prescriptions.
Without accurate and complete information, DCF and health providers may make decisions that
conflict with existing medical treatments or do not reflect children’s best interests, such as
overprescribing psychotropic medications, which can lead to adverse side effects.
Authoritative Guidance
According to DCF Policy 86-011: Ongoing Casework and Documentation,
It is the policy of the Department that the Social Worker documents casework activity for
each family, in the family’s case record in FamilyNet. Each client and collateral contact is
documented in dictation and entered into FamilyNet as soon as possible. It is expected
that dictation will be entered into FamilyNet no later than one month following the contact.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
23
According to DCF Policy 85-003: Health Care Services to Children in Placement,
All children in placement will have a medical passport containing pertinent and available
medical, dental, mental health and developmental information prior to or at the time of
placement. . . . This information contained in the medical passport will be reviewed in
conjunction with the Service Plan every 6 months at Foster Care Review and will be
updated when warranted. . . .
PROCEDURES SUBSEQUENT TO PLACEMENT. . . .
3. Encounter Form. The Social Worker ensures that the medical and dental
appointments are documented by use of the encounter form. . . . Upon receipt of
the second page of the encounter form, from either the physician or substitute
care provider, the Social Worker completes the form and submits it for data entry
or directly enters information into FamilyNet. . . . After data entry into FamilyNet,
the encounter form should be placed in the special document section envelope
with the copy of the passport. . . .
6. Case Review. In preparing for a case review the Social Worker reviews the
child(ren)’s current health care status by identifying any recent medical/dental
problems and whether the child(ren) has received any necessary routine care and
follow-up treatment. The Social Worker ensures that current medical information
is available in the case record. This includes:
 an up-to-date copy of the medical passport in the case record by either copying
the substitute care providers medical passport or adding to the case record
medical passport;
 current encounter forms in the case record and up to date information in
FamilyNet;
 current evaluation, test, and treatment results in the case record.
According to DCF Policy 2010-001: Medical Examinations for Children Entering DCF Placement or
Custody,
The information that the Social Worker documents in the medical sections of FamilyNet
includes, but is not limited to:
 Name, address and telephone number of the primary medical practitioner;
 Names and dates of medical or oral health examinations or tests, the practitioner
who completed the examinations or tests and any recommendations, findings or
treatments;
 Medical, oral health and behavioral health conditions that have been observed or
diagnosed;
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
24
 Medications that are prescribed;
 Known allergies;
 Immunizations that have been given; and
 Health-related equipment that is being used.
According to Section 475 of the Social Security Act,
(1) The term “case plan” means a written document . . . and includes at least the
following: . . .
(C) The health and education records of the child, including the most recent
information available regarding . . .
(v) the child’s known medical problems;
(vi) the child’s medications; and
(vii) any other relevant health and education information concerning the child
determined to be appropriate by the State agency.
Auditee’s Response
DCF agrees with the recommendation for additional monitoring controls to ensure up-todate and accurate documentation of a child’s health care in iFN. DCF is working to address
controls around these documentation issues, and will review the existing policy, purpose,
and guidelines for the medical passport as well as the documentation requirements for all
medical visits.
DCF agrees with the OSA's broad concern that children in DCF custody must not receive
medications that are contraindicated based on other medications they are taking.
Sometimes, providers do have access to the child's complete electronic medical record. For
continuity of care, DCF makes every effort to keep a child with their medical provider of
origin. If a child’s medical provider changes, DCF requests that record to transfer
immediately to a new provider.
While the medical passport was never intended to be a substitute for the child’s office
medical record, DCF recognizes the importance of maintaining up to date records regarding
psychotropic medications and other medical treatment in a child's iFN record and is
exploring how we can make better use of technology to do so. In February 2024 and
monthly thereafter, the Department began batch loading medication data based on
MassHealth pharmacy claims. MassHealth Pharmacy Claims Data is used to create
medication records for children in custody. The information can be viewed on the
Medical/Behavior Info page which is available in the Person Demographics for the child. In
addition, we are exploring ways to use MassHealth claims to capture other data such as
visit dates and diagnoses.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
25
Auditor’s Reply
We commend DCF for taking steps to improve its recordkeeping practices. We will follow up on this
issue in approximately six months as part of our post-audit review.
b. The Department of Children and Families did not document follow-up
doctor appointments and recommended psychosocial services in
iFamilyNet for children in its protective custody receiving psychotropic
medications.
DCF did not document follow-up doctor appointments and recommended psychosocial services in iFN
for children in its protective custody who received psychotropic medication. For 116 of the 118
children in our FFS sample and 46 of the 50 children in our MCO sample, we were unable to determine
the type and frequency of therapy provided to the children or whether they received follow-up doctor
appointments at all.
If DCF does not keep accurate and complete medical records in iFN, then children in DCF’s protective
custody may not receive the services needed to treat their conditions. This may delay the growth,
development, or recovery of the children who did not receive needed care. Failure to keep accurate
and complete medical records may also prevent DCF from determining which medical treatments or
providers are most effective or cost-efficient for serving the medical needs of children in its care.
Authoritative Guidance
According to the American Academy of Child and Adolescent Psychiatry’s (AACAP’s) 2015
Recommendations about the Use of Psychotropic Medications for Children and Adolescents Involved
in Child-Serving Systems,
All youth with complex behavioral needs, including youth in foster care, should receive a
combination of evidence-based psychosocial interventions and psychotropic medication
when indicated, not just psychotropic medication alone.
Auditee’s Response
DCF agrees with the recommendation for additional monitoring controls to ensure up-todate and accurate health records in iFamilyNet and medical passports. DCF is working to
address controls around these documentation issues, and will review the existing policy,
purpose, and guidelines for the medical passport as well as the documentation
requirements for all medical visits.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
26
As one of DCF’s critical objectives is continuous improvement, there is an opportunity for
the Department to review the existing policy regarding health care services for children in
placement, the purpose, use and contents of the medical passport, and the utility of the
current, paper based, encounter form. This will allow for a review of updated medical
recommendations and any [information technology] enhancements that may be necessary.
In addition, DCF would like to note some additional circumstances and barriers
continuously being worked on by the agency: Many children receive services via Family
Support Services (“FSS”) . . . and will not be captured in claims. There are children who
are stable on the ADHD medications, for example, who receive school supports through
an IEP or 504, which also may not be clear in iFN. That child may not need additional
outside support such as individual therapy.
Review of Medicaid claims data does not encompass the breadth of psychosocial services
a child may be receiving.
Auditor’s Reply
We commend DCF for taking steps to improve its recordkeeping practices. We agree that DCF cannot
rely solely on MassHealth claims to determine whether children in its custody received recommended
psychosocial services, and we did not recommend this in our audit. We reiterate our recommendation
that DCF should ensure that children in its protective custody have up-to-date and accurate health
records in iFN.
c. The Department of Children and Families did not document its consent
in iFamilyNet for children in its protective custody to receive
psychotropic medications.
DCF did not properly document its consent or court approval for the prescribing of psychotropic
medications16 for children in its protective custody. Specifically, 109 (92%) out of 118 of the children
who were prescribed at least one psychotropic medication from our FFS sample did not have required
documentation of DCF’s consent or court approval for psychotropic medications. In addition, 47 (94%)
out of 50 of the children who were prescribed at least one psychotropic medication from our MCO
sample did not have the required documentation of DCF’s consent or court approval for psychotropic
medications.
16. DCF’s consent is required for a child to be prescribed most psychotropic medications, while court approval is required for a
child to be prescribed antipsychotic medications (see the “Rogers Guardianship Order” section of this report for more
information). Our samples combined antipsychotic mediations with other classes of psychotropic medications.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
27
Without documentation of consent or court approval for prescriptions of psychotropic medications,
DCF cannot ensure that its social workers and/or medical social workers are providing children in DCF
protective custody with medical treatment that is legally required.
Authoritative Guidance
According to Section 11.14(4)(a) of Title 110 of the Code of Massachusetts Regulations,
When any individual, organization, facility, or medical provider seeks the Department’s
consent to medicate with antipsychotic drugs a child, who is a ward of the Department or
who is in Department custody, the Department shall seek prior judicial approval for
administration of such drugs even if the child’s biological parents have consented to the
medication.
According to the “Roles of Foster/Pre-Adoptive Parents or Other Substitute Care Providers and Social
Workers” section of DCF Policy 2010-001, “The foster/pre-adoptive parent (or other substitute care
provider) . . . arranges for the child to receive medical, behavior health and oral health care that is
recommended by the medical practitioner and consented to by the Department.”
According to DCF Policy 86-011: Ongoing Casework and Documentation,
It is the policy of the Department that the Social Worker documents casework activity for
each family, in the family’s case record in FamilyNet. Each client and collateral contact is
documented in dictation and entered into FamilyNet as soon as possible. It is expected
that dictation will be entered into FamilyNet no later than one month following the contact.
AACAP’s 2005 “Position Statement on Oversight of Psychotropic Medication Use for Children in State
Custody: A Best Principles Guideline” states,
State child welfare agencies, the juvenile court, or other state or county agencies
empowered by law to consent for treatment with psychotropic medications, should create
a website to provide ready access for clinicians, foster parents, and other caregivers to
pertinent policies and procedures governing psychotropic medication management,
psychoeducational materials about psychotropic medications, consent forms, adverse
effect rating forms, reports on prescription patterns for psychotropic medications, and links
to helpful, accurate, and ethical websites about child and adolescent psychiatric diagnoses
and psychotropic medications.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
28
Reasons for Issues
DCF did not have sufficient monitoring controls in place to ensure that children in its protective
custody have up-to-date and accurate health records in iFN and that its social workers prevent these
children from receiving medical care without approval.
Recommendation
DCF should establish sufficient monitoring controls to ensure that children in its protective custody
have up-to-date and accurate health records in iFN and that its social workers prevent these children
from receiving medical care without approval, including the following:
 DCF should review medical passports for children in its protective custody and update them
at least every six months or when there are changes to a child’s prescription, whichever comes
first (e.g., new prescriptions, dosage changes, or discontinued prescriptions).
 DCF should update iFN with all follow-up doctor appointments and psychosocial services for
children in its protective custody, including the type and frequency of these appointments
and services.
 DCF should document its consent for psychotropic medication for children in its protective
custody in iFN and store that consent in the same location in iFN for quick and accurate
reviews.
Auditee’s Response
DCF agrees with the recommendation for additional monitoring controls to ensure that the
Department is documenting a child’s health care in a timely and accurate way in iFN.
The iFN system is the Commonwealth’s SACWIS (statewide automated child welfare
information system), which supports the states’ child welfare business needs and is used
primarily to document the activities and services that DCF social worker staff provides to
the families and children it serves. DCF also utilizes iFN to document the health care a child
receives in conjunction with policy, but in no way does the iFN health care record serve as
a substitute for a child’s medical record kept by medical providers.
The OSA response combines both antipsychotic and non-antipsychotic medications under
the heading of “psychotropic medications”. Different consent procedures exist for both:
Antipsychotic medication consent is provided by the court and consent is indicated by the
Rogers order, which is uploaded into iFN.
Substitute caregivers are authorized to provide consent for medical treatment, including
non-antipsychotic medication, per DCF policy “Health Care Services to Children in
Placement”. . . When a child is placed in a foster home, the foster parent signs both a
Foster Parent Agreement and Child Placement agreement where they agree to manage the
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
29
child’s routine health care, dental care, and emergency medical treatment when necessary.
While the health care services policy requires foster parents to provide documentation of
health care events, it does not require DCF to document consent for routine health care,
which includes the administration of non-antipsychotic psychotropic medication.
DCF currently has multiple pathways for oversight to ensure safety and the judicious use
of psychotropic medications for youth in state custody including: DCF nurses and [its] child
psychiatrist are always available for consultation about the appropriate dosing and
effectiveness of medications. . . .
DCF entered into an information sharing agreement with MassHealth, which enabled the
documentation of psychotropic medication in the child’s electronic case record.
DCF has piloted and implemented the Antipsychotic Monitoring Program (AMP), overseen
by the DCF child psychiatrist. The purpose of AMP is to provide a medical review of initial
requests for antipsychotic medications for youth in state custody to help inform the court
and Rogers Process around the appropriateness of the use of the antipsychotic medication
in that child.
DCF collaborates with the MassHealth Pediatric Behavioral Health Medication Initiative
(PBHMI) to provide further expert review when there are concerns about appropriateness
of psychotropic medication for youth in state custody.
DCF will continue to collaborate with other child & family serving agencies around
continuous quality improvement efforts of the state’s current psychotropic oversight
program PBHMI as well as DCF’s internal oversight systems.
Auditor’s Reply
DCF states that it is not required to document consent for routine healthcare (for example,
psychotropic medication, not including antipsychotics). The “Authoritative Guidance” section of this
finding outlines best practices that, while not required, reflect best practices developed by healthcare
professionals and experts. The AACAP 2009 “Practice Parameter on the Use of Psychotropic
Medication in Children and Adolescents” goes into detail about “documenting the assent of the child
and consent of the parent.” In addition, iFN contains social worker notes and other references to
seeking or documenting consent.
We appreciate that DCF shared its use of multiple oversight pathways to ensure safety for the use of
psychotropic medications. However, we maintain our concern that DCF cannot provide proper
oversight without accurate documentation of these medications and will follow up with DCF in
approximately six months for our post-audit review.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
30
3. The Department of Children and Families did not ensure that children
received recommended psychosocial services in conjunction with their
prescriptions for psychotropic medications.
Children did not always receive recommended psychosocial services in conjunction with their
prescriptions for psychotropic medications. Specifically, 17 of the 118 (14%) children who were prescribed
at least one psychotropic medication in our FFS sample received no therapy during our audit period in
conjunction with their psychotropic medications, according to MMIS data. In addition, 8 of the 50 (16%)
children who were prescribed at least one psychotropic medication in our MCO sample receive no therapy
during the audit period in conjunction with their psychotropic medications, according to MMIS data.
We also found that an additional 24 of the 118 children who were prescribed at least one psychotropic
medication in our FFS sample and an additional 10 of the 50 children who were prescribed at least one
psychotropic medication in our MCO sample did not receive therapy in conjunction with their
psychotropic medications for over four months.
Additionally, we could not determine the frequency that children in residential facilities received therapy
based on MMIS data, because therapy is not billed separately in a residential facility, as it was with the
issues above.
Below is a table that summarizes the three issues we found related to children not receiving psychosocial
services.
Issue Number of Children from
FFS Sample
Number of Children from
MCO Sample
No therapy at all 17 out of 118 8 out of 50
Did not receive therapy for over four months 24 out of 118 10 out of 50
Unable to determine frequency of therapy visits 10 out of 118 3 out of 50
If children do not receive the recommended therapy and psychosocial services with psychotropic
medications, treatment effectiveness can be negatively affected. Further DCF cannot monitor the
effectiveness of these medications and cannot identify and mitigate any side effects that these children
may experience. For example, 28 children from both our samples had suicidal ideations.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
31
Authoritative Guidance
According to AACAP’s 2015 Recommendations about the Use of Psychotropic Medications for Children and
Adolescents Involved in Child-Serving Systems,
All youth with complex behavioral needs, including youth in foster care, should receive a
combination of evidence-based psychosocial interventions and psychotropic medication when
indicated, not just psychotropic medication alone.
While DCF is not required to follow these recommendations, we consider them a best practice.
Reasons for Issues
DCF does not have sufficient monitoring controls in place to ensure that children in its protective custody
who are prescribed psychotropic medications receive psychosocial services.
Recommendation
DCF should ensure that all children in its protective custody who are prescribed psychotropic medications
receive psychosocial services and DCF should implement sufficient monitoring controls to ensure that
these services are provided and that the efficacy of these services is evaluated.
Auditee’s Response
DCF agrees with this recommendation, in that we will continue to improve monitoring controls to
document psychosocial services are provided.
DCF agrees that we are not consistent with documenting services in iFN when children receive
psychosocial services at school and through Family Support Services. Many children receive services
via Family Support Services (“FSS”) which is covered by Title IV E and will not be captured in MMIS
claims. There are children who are stable on the ADHD medications, for example, who receive
school supports through an IEP or 504, which also may not be clear in iFN. That child may not
need additional outside support such as individual therapy.
Auditor’s Reply
We agree that DCF cannot rely solely on MassHealth claims to determine whether children in DCF custody
received recommended psychosocial services. As stated in our reply above, we did not recommend that
DCF do so. We instead recommend that DCF should ensure that all children in its protective custody who
are prescribed psychotropic medications receive psychosocial services by implementing sufficient
monitoring controls. Based on its response, DCF is addressing our concerns in this area.
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
32
4. The Department of Children and Families did not ensure that all employees
with access to COVID-19 funds received annual refresher cybersecurity
awareness training.
DCF was unable to provide evidence that 2 of its 10 employees who had access to COVID-19 funding
completed annual refresher cybersecurity awareness training for fiscal year 2020. Additionally, DCF was
unable to provide evidence that 1 out of 10 employees with access to COVID-19 funds completed annual
refresher cybersecurity awareness training for fiscal year 2021.
If DCF does not ensure that all its employees complete cybersecurity awareness training, then it is exposed
to a higher-than-acceptable risk of cyberattacks and financial and/or reputational losses.
Authoritative Guidance
Section 6.2.4 of the Executive Office of Technology Services and Security’s Information Security Risk
Management Standard IS.010, which went into effect October 15, 2018, states, “All personnel will be
required to complete Annual Security Awareness Training.”
Reasons for Issues
DCF stated that it encountered obstacles when retrieving certificates of completion of cybersecurity
awareness training associated with transitioning to a different cybersecurity awareness training provider.
Recommendation
DCF should develop and implement policies, procedures, and controls to ensure that all its employees
complete cybersecurity awareness training.
Auditee’s Response
Since the audit review period, the Department and the Executive Office of Technology Services and
Security (EOTSS) has developed and implemented additional procedures and controls to ensure
compliance with annual refresher cybersecurity awareness training requirements. The trainings are
offered through the Commonwealth’s Learning Management System, MassAchieve. From the data
in MassAchieve, DCF’s Office of Management Planning and Analysis has developed and distributes
monthly management reports which provide the status of individual employees’ completion of
cybersecurity awareness training prior to the established training deadline. Managers use these
reports to follow-up with employees and ensure they complete the annual refresher training by the
requisite deadline. In addition, the Department’s Deputy Commissioner for Administration and
Finance monthly reports out the status of the agency’s compliance with completing the annual
refresher cybersecurity awareness training to the agency’s leadership at the monthly Statewide
Audit No. 2022-1058-3H Department of Children and Families
Detailed Audit Findings with Auditee’s Response
33
Managers Meeting. Lastly, EOTSS has implemented a control which shuts down network access of
employees who have not completed the annual refresher cybersecurity awareness training by the
requisite deadline. Access can only be restored once the employee completes the training.
Auditor’s Reply
We commend DCF for implementing stronger monitoring controls to ensure that all employees complete
cybersecurity awareness training and believe DCF is taking steps to address this issue.
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
34
OTHER MATTERS
1. The Department of Children and Families should provide more oversight for
children in its custody receiving psychotropic medication in amounts and
dosages that exceed United States Food and Drug Administration
recommendations.
During our audit, we found that 8 out of the 40 (20%) children in our maximum dosage sample received
psychotropic medications in dosages that exceeded the US Food and Drug Administration’s (FDA’s)
recommendations. Amounts and dosages of medications that exceed the FDA’s recommended guidelines
may be appropriate under some circumstances, and while we are not questioning prescribers’ medical
expertise, there should be more state oversight for children in protective custody receiving these higher
amounts and dosages. The Department of Children and Families (DCF) currently does not have any
oversight policies or procedures for children prescribed psychotropic medications, especially for situations
when the dosages exceed the FDA’s recommendations.
According to the American Academy of Child and Adolescent Psychiatry’s 2005 “Position Statement on
Oversight of Psychotropic Medication Use for Children in State Custody: A Best Principles Guideline,”
State child welfare agencies, the juvenile court, or other state or county agencies empowered by
law to consent for treatment with psychotropic medications, in consultation with child and
adolescent psychiatrist, should design and implement effective oversight procedures that:
a) Establish guidelines for the use of psychotropic medications for youth in state custody. . . .
For example, the State of California has adopted the Los Angeles Department of Mental Health’s
“Parameters 3.8 For Use of Psychotropic Medication for Children and Adolescents” guidelines. These
guidelines state, “Treatment provided outside the parametric elements in this guide requires special
justification or consultation and subsequent documentation in medical record.”
We recommend that DCF implement maximum dosage guidelines for psychotropic medications and that
DCF provide additional oversight and authorization when children in its protective custody are prescribed
psychotropic medications that exceed the FDA’s recommended maximum dosage. For example, DCF
should contact the prescriber to ensure the safety and necessity of the dosage in question and then clearly
document the reasons given by the prescriber in iFamilyNet (iFN). Given the documentation weaknesses
we have described previously in this audit report, we believe this step is particularly appropriate, as the
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
35
absence of psychosocial treatment in many instances, and the lack of documentation regarding
medication and other treatments, may hinder treatment with these higher-than-recommended dosages.
As our audit was nearing completion, additional areas of concern that were outside the scope of our
objectives came to our attention. Given the high-risk nature of these areas, we looked into them, and the
results are documented below.
Auditee’s Response
[The Office of the State Auditor (OSA)] indicates that in 8 cases children received dosages that
were above FDA guidelines. DCF acknowledges cases exceeding FDA guidelines but emphasizes
that these guidelines are not the standard for most child psychiatrists. In Massachusetts and
throughout the country, nationally accepted literature-based recommended maximum dosages
prevail. Using these standards, there was one case that exceeded both FDA and literature-based
recommendations (16 year old on 40 mg escitalopram- Lexapro), which was an error.
These literature-based maximums are presented in the Texas Psychotropic Medication Utilization
Parameters and the [Los Angeles Department of Mental Health] Parameters for Psychotropic
Medication Use (cited by the OSA above), both of which provide guidance to the DCF medical team
and to pediatric mental health professionals nationally. In addition to these tools from other states
utilized by the DCF medical team, DCF also follows guidance from the MassHealth Pediatric
Behavioral Health Medication Initiative (“PBHMI”), that defines what is considered high risk
psychotropic prescribing in pediatrics in MA. Youth who are identified as high-risk and have
concerns for inappropriate use of a medication will require a Doc-to-Doc review with the DCF child
psychiatrist.
DCF disagrees that exceeding FDA maximums guidelines always necessitates prescriber follow-up.
Literature-based maximums, even when surpassing FDA recommendations, are accepted practice
with proper justification. We appreciate the OSA’s consideration of the national standard methods
being used, and note that DCF consults prescribers when concerns arise about inappropriate dosing
or when dosing falls outside literature-based parameters.
Auditor’s Reply
Based on DCF’s response, it believes that only 1 child was prescribed psychotropic medication over the
FDA-recommended maximum dosage. We verified that the number is 8 children out of the sample of 40
(see the “Maximum Dosages” section). We would still recommend that more oversight be outlined for
children who are prescribed psychotropic medications over the FDA-recommended maximum dosages,
even if only one child was involved.
We did not state in our audit that children cannot be prescribed psychotropic medications over the
FDA-recommended maximum dosages but rather that children who are prescribed these higher dosages
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
36
should also receive more oversight from DCF. We found that, of those 8 cases where children were
prescribed psychotropic medications over the FDA-recommended maximum dosages, there were no
notes related to the higher dosage, and iFN did not have up-to-date, accurate medication information.
We commend DCF for seeking consultation with psychotropic medication prescribers and participating in
other oversight programs. However, this oversight is not documented. We recommend that DCF
document all reasons/recommendations for children in its protective custody who are prescribed
psychotropic medications at dosages higher than FDA-recommended maximum dosages, or
literature-based maximums as mentioned in DCF’s reply, so that all individuals involved in the child’s
mental health treatment are informed and provide proper care.
2. The Department of Children and Families should coordinate with other
state agencies, law enforcement, and other child-serving agencies to
address how to detect and respond to human trafficking.
In July 2022, the US Department of Health and Human Services Office of Inspector General (HHS OIG)
issued a report17 that cited Massachusetts as one of five states where there was no evidence that children
in foster care were screened for human trafficking after they had gone missing from, and later returned
to, foster care. Massachusetts was selected because, in fiscal year 2018, it was one of five states that
reported the largest number of children who went missing from state custody. Specifically, in its report,
HHS OIG identified 949 children in Massachusetts who went missing from, and later returned to, foster
care. HHS OIG selected a sample of 88 out of the 949 identified children and reviewed their case files. HHS
OIG found that 72 out of the 88 sampled children were not screened for human trafficking after they
returned to foster care.
We reached out to DCF via email to ascertain what corrective measures the agency took, or is taking, to
address the findings and concerns of this HHS OIG report. Based on DCF’s responses, the agency does not
agree with the findings in the HHS OIG report but has taken measures to address those concerns.
Specifically, DCF stated in an email to us on January 5, 2024,
We also outlined subsequent actions and continuous improvement efforts that we continue to work
on. . . . Our ongoing quality improvement work to date has included our Missing or Absent Program
Manager presenting a series of trainings to congregate care providers. We have also created a
video for DCF staff, “Missing or Absent Children/Youth: DCF Screening Guidance for All Youth” on
17. This HHS OIG report is titled In Five States, There Was No Evidence That Many Children in Foster Care Had a Screening for Sex
Trafficking When They Returned After Going Missing.
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
37
best practices for screening youth who return from being Missing or Absent (MOA), in collaboration
with Support to End Exploitation Now (SEEN) at the Children’s Advocacy Center of Suffolk County,
My Life My Choice (MLMC), as well as the accompanying guide, “Human Trafficking: DCF Screening
Guidance for All Youth.” These supplementary tools and training aides have been communicated
to our staff and utilized through several platforms and meetings to date.
In addition, DCF has worked with [the Administration for Children and Families] National Human
Trafficking Training and Technical Assistance Center (NHTTTAC) to create a guide for the field
which is currently being vetted by executive staff for distribution and inclusion on our Human
Trafficking intranet page.
We also asked several related follow-up questions that addressed the detection and prevention of human
trafficking.
Illinois was one of the other states audited in the 2022 HHS OIG report. Specifically, the Illinois Department
of Children and Family Services agency developed a webpage, entitled Human Trafficking of Children, that
is dedicated to the awareness of human trafficking of minors. The webpage also has educational
brochures and posters in several languages available for download. Currently, DCF has one webpage,
entitled Definitions of abuse and neglect, that mentions this issue but DCF does not provide details on
how to detect and prevent human trafficking. We recommend that DCF create a webpage, or another
platform to easily reach the public, dedicated to recognizing signs and what to do if someone has
suspicions of human trafficking, like the webpage that the Illinois Department of Children and Family
Services developed.
Finally, we wish to share the model legislation, policies, and regulations put forth by the advocacy
organization, Shared Hope International,18 called “Report Cards on Child & Youth Sex Trafficking 2023
Toolkit.” Massachusetts received an overall F grade from this organization for its efforts to stop child and
youth sex trafficking. We recommend that DCF work with law enforcement and other child-serving
government agencies (e.g., the Office of the Child Advocate) to implement the model legislation, policies,
and regulations. DCF should also work to determine why Massachusetts has such a high rate of children
going missing from state care and address the issue.
18. According to its website, “Shared Hope is a member of the National Advisory Committee on the Sex Trafficking of Children
and Youth in the United States which publishes Best Practices and Recommendations for States to combat the sex trafficking
of children and youth in the United States.”
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
38
Auditee’s Response
The Department of Children and Families did not agree with the methodology used by the OIG
when the report was issued and subsequently did a comprehensive, parallel review and found
screenings were noted in the vast majority of the records the OIG reviewed. The OIG only accepted
evidence of screening documentation for 22 of the 89 sample cases it deemed eligible for the
review. After receiving this information, DCF conducted a thorough review of the other 67 cases in
which OIG determined that there was insufficient evidence of screenings. Based on this review, we
believe that there was evidence to support screenings of youth who returned to DCF placement in
82 of the 89 sample cases.
The screenings may have been missed by the OIG because [DCF] collected the data but had not
been consistent with entering the information in the case record or in structured data.
Since the OIG report, DCF has made [information technology] changes to capture the screening of
youth for human trafficking when they return from being missing or absent and to improve tracking
of the screenings that take place. This, and other system upgrades are scheduled to begin at the
end of August.
The Shared Hope report does not take into consideration the ongoing proactive, preventive
continuous improvement initiatives as mentioned above, and which also include:
 Extensive work with the Federal Administration for Children and Families and its
National Human Trafficking Training and Technical Assistance Center to update and
streamline information on the DCF Human trafficking employee intranet page.
 DCF’s Clinical Manager for Field Support becoming a member of the National Child
Welfare Anti-Trafficking Coalition. Participants include those creating and
implementing state-level child welfare policy and practice addressing human
trafficking.
 Collaborating with My Life My Choice to create a Commercial Sexual Exploitation of
Children (CSEC) prevention training program.
 [CSEC] is providing training congregate care providers over the next two years.
 Creating a 9-part video series for DCF social workers to use with foster parents and
other caregivers to increase their knowledge regarding Commercial Sexual Exploitation
of Children (CSEC).
 In collaboration with My Life My Choice (MLMC) and Support to End Exploitation Now
(SEEN), creating a training video for DCF social workers on best practices for screening
youth who return from being [Missing or Absent], as well an accompanying guide,
“Human Trafficking: DCF Screening Guidance for all Youth.”
Further, finalization of our negotiations with [the collective bargaining unit] will allow us to hire 5
full time supervisors and increase the Missing or Absent (MOA) unit from 10 social workers to 18.
MOA social workers are dedicated to engaging and locating children who are on-the-run. Additional
staff will allow for collaboration with community stakeholders as well as more prevention work with
youth, families and substitute care providers.
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
39
Massachusetts has multi-disciplinary teams (MDTs) in every county to respond to Child Trafficking.
These teams are based in the state’s Children’s Advocacy Centers (CAC) in partnership with their
DA offices, local police and DCF to provide a coordinated response to CSEC. MDTs are currently
operating in all CACs in the Commonwealth. DCF also collaborates with local police Departments,
including the Boston Police, on child trafficking concerns.
In its Annual Report, DCF tracks the number of 51A reports and supported 51Bs for human
trafficking labor and human trafficking sexually exploited child as well as the unique count of
children DCF has found to have been trafficked.
DCF will also develop a website on Mass.gov where the public can find information on human
trafficking prevention and detection in children.
Auditor’s Reply
We commend DCF on its efforts and current initiatives to prevent children from going missing and sex
trafficking in Massachusetts. Based on its responses, DCF appears to have taken steps to collaborate with
relevant organizations, coordinate additional training for care providers, and hire more staff members.
DCF also plans to develop a website to raise awareness of this issue. As part of our post-audit review
process, we look forward to revisiting this topic with DCF and seeing what progress has been made in
approximately six months from now.
3. The Department of Children and Families should collaborate with the
Massachusetts Commission on LGBTQ Youth to implement all
recommendations from its annual report.
The Massachusetts Commission on LGBTQ Youth’s goal is to make policy recommendations to the
Executive Office of Health and Human Services, the Department of Public Health, and other government
entities within the Commonwealth that support LGBTQ youth. The commission’s 2023 annual report19
included the following recommendations for DCF:
1. Ensure thorough and accurate [sexual orientation and gender identity (SOGI)] data
collection through implementation of the new mandatory data elements and staff training.
2. Create and follow a plan for additional phases of SOGI data collection.
3. Report SOGI data in detail, in annual and quarterly reports.
4. Ensure LGBTQ community representation in decisions regarding data collection and
reporting.
19. This Massachusetts Commission on LGBTQ Youth report is titled Report and Recommendations for Fiscal Year 2023.
Audit No. 2022-1058-3H Department of Children and Families
Other Matters
40
5. Release a comprehensive LGBTQ nondiscrimination policy
6. Continue implementation of the Gender Affirming Medication Consent Policy
7. Update the Family Resource Policy with LGBTQ-Inclusive Provisions
8. Clarify Policy Regarding Placement Consistent with Gender Identity
9. Continue Policy Collaborations with the Commission
10. Update the LGBTQ Guide and ensure that all staff, providers, youth, and families know
it exists and where to access a copy.
11. Expand and require LGBTQ cultural humility training.
12. Update and Improve the [Massachusetts Approach to Partnerships in Parenting]
Training
13. Create a statewide database listing LGBTQ-affirming homes.
14. Improve recruitment of LGBTQ affirming foster parents.
15. Create Positions for LGBTQ Regional Specialists and Add or Adjust Other Staff
Responsibilities to Promote LGBTQ Equity
16. Promote Youth Rights and Voices
We asked DCF whether it had implemented these recommendations and it told us in an email on June 18,
2024,
DCF has been continuing to work towards implementing the recommendations of the Commission.
As part of the recommendations, DCF has hired a Director of LGBTQIA+ Services and three
Regional LGBTQIA+ Specialists. In addition, DCF meets regularly with the Massachusetts
Commission on Lesbian, Gay, Bisexual, Transgender, Queer & Questioning Youth and requests
their feedback on DCF initiatives. DCF and [this commission] also participate in an [Executive Office
of Health and Human Services] workgroup that meets regularly to coordinate efforts on this topic.
Auditee’s Response
DCF appreciates the positive acknowledgement from the OSA.
Auditor’s Reply
Based on its response, DCF has taken steps to implement the commission’s recommendations.

Official Audit Report – Issued September 27, 2022
Department of Veterans’ Services
For the period July 1, 2019 through June 30, 2021
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
September 27, 2022
Ms. Cheryl Lussier Poppe, Secretary
Department of Veterans’ Services
600 Washington Street
Boston, MA 02111
Dear Secretary Poppe:
I am pleased to provide this performance audit of the Department of Veterans’ Services. This report
details the audit objectives, scope, methodology, findings, and recommendations for the audit period,
July 1, 2019 through June 30, 2021. My audit staff discussed the contents of this report with
management of the agency, whose comments are reflected in this report.
I would also like to express my appreciation to the Department of Veterans’ Services for the cooperation
and assistance provided to my staff during the audit.
Sincerely,
Suzanne M. Bump
Auditor of the Commonwealth
cc: Marylou Sudders, Secretary of the Executive Office of Health and Human Services
Audit No. 2021-0018-3S Department of Veterans’ Services
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 5
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE........................................................................................ 8
1. The Department of Veterans’ Services did not ensure that its Women Veterans’ Network achieved its
intended purpose. ...................................................................................................................................... 8
2. DVS did not have enough members appointed to its advisory committee on women veterans................ 15
Audit No. 2021-0018-3S Department of Veterans’ Services
List of Abbreviations
ii
LIST OF ABBREVIATIONS
DVS Department of Veterans’ Services
OSA Office of the State Auditor
VA United States Department of Veterans Affairs
VSO veterans’ services officer
WVN Women Veterans’ Network
Audit No. 2021-0018-3S Department of Veterans’ Services
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Department of Veterans’ Services
(DVS) for the period July 1, 2019 through June 30, 2021.
In this performance audit, we determined whether DVS had a process in place to ensure that its Women
Veterans’ Network (WVN) effectively achieved its purpose of identifying women veterans and informing
them about their potential eligibility for benefits they may have earned through their service. In
addition, we determined whether DVS had established an advisory committee on women veterans in
compliance with Section 2 of Chapter 115 of the General Laws.
Below is a summary of our findings and recommendations, with links to each page listed.
Finding 1
Page 8
DVS did not ensure that its WVN achieved its intended purpose.
Recommendation
Page 10
DVS should establish policies and procedures, including a monitoring component, for WVN’s
operations to ensure that all required activities are conducted and properly documented.
Finding 2
Page 15
DVS did not have enough members appointed to its advisory committee on women veterans.
Recommendation
Page 15
DVS should work with the Governor’s Office to ensure that the advisory committee has the
required 11 members.
Audit No. 2021-0018-3S Department of Veterans’ Services
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Department of Veterans’ Services (DVS) is an agency within the state’s Executive Office of Health
and Human Services that is authorized under Section 2 of Chapter 115 of the Massachusetts General
Laws to assist and advise veterans’ services officers (VSOs) regarding benefits and services for which
veterans may be eligible. According to its website,
The mission of the Department of Veterans’ Services is to advocate on behalf of all the
Commonwealth’s veterans and provide them with quality support services and to direct an
emergency financial assistance program for those veterans and their dependents who are in
need.
In addition, DVS represents state agencies and individual veterans before the United States Department
of Veterans Affairs (VA) in securing compensation and available benefits.
DVS also administers a need-based benefit program, pursuant to Chapter 115 of the General Laws,
through VSOs assigned to each municipality in the Commonwealth. Each city or town pays eligible
veterans, or their eligible dependents, their monthly benefits1 directly, by check, at the beginning of
each month and then requests reimbursement from the Commonwealth for 75% of the benefit amount
paid. DVS pays the municipalities quarterly for eligible benefits, a year in arrears. DVS reimbursed local
municipalities for Chapter 115 benefits of $40,149,878 in fiscal year 2020 and $39,442,237 in fiscal year
2021.
VSOs are appointed by the mayors or boards of selectmen in cities and towns to accept applications
from veterans and their dependents, determine their eligibility for benefits, and file requests for federal
benefits. In partnership with VSOs, DVS helps veterans and their families navigate available federal,
state, and local programs, benefits, and other resources.
During our audit period, DVS had approximately 60 employees, including managers and support
personnel. Its headquarters are at 600 Washington Street in Boston.

1. These need-based benefits, paid monthly to veterans or their eligible dependents, help recipients with their monthly living
expenses for food, shelter, fuel, and medical assistance.
Audit No. 2021-0018-3S Department of Veterans’ Services
Overview of Audited Entity
3
Women Veterans’ Network
According to the VA National Center for Veterans Analysis and Statistics, there were approximately
25,000 women veterans living in Massachusetts as of September 30, 2020. The Women Veterans’
Network (WVN), a program within DVS, was established in 1997. According to WVN’s website,
[WVN’s] purpose is to find women who served in the military, some of whom may not be aware
that they are veterans and eligible for benefits they have earned through their service. . . .
We provide information on federal, state and local benefits [to women veterans]. Our mission
also includes expanding awareness of the needs of women veterans and identifies available
resources to meet those needs.
WVN uses a database of information about women veterans in Massachusetts who have joined the
network to create a mailing list for its biannual newsletter, which contains information on benefits,
programs, resources, and events for women veterans. WVN also emails updates about events and
programs between issues of the newsletter. Each year, WVN hosts a Women Veterans Appreciation Day
ceremony, where it gives a woman veteran the Deborah Sampson Award.2 In addition, WVN hosts a
conference with keynote speakers, workshops, networking opportunities, and a panel on topics that are
relevant to women veterans.
WVN is administered by a director, who reports directly to the DVS chief of staff. According to the job
description, some of the WVN director’s duties and responsibilities are as follows:
Oversee and maintain a database of over 14,500 women veterans in the Commonwealth. . . .
Conduct aggressive research via informational fairs and workshops, public events and other
mediums to expand membership in the Women Veterans’ network.
Collaborate within DVS with other agencies at state, federal and non-profit provider levels that
serve women veterans to identify resources, improvements to programs and services to women
veterans, particularly in terms of healthcare, counseling, employment, education and
housing. . . .
Respond to direct inquiries from women veterans to provide information, referrals and establish
linkages with supportive services. . . .
Provide informational updates via email, website, and social network sites on services, programs,
and events between newsletters. . . .

2. According to the WVN website, “The Deborah Sampson Award proudly recognizes a female Veteran who has gone above
and beyond to serve her fellow Women Veterans throughout the year.”
Audit No. 2021-0018-3S Department of Veterans’ Services
Overview of Audited Entity
4
Prepare reports as required and requested by the Department of Veterans’ Services, Executive
Office of Health [and] Human Services, and the Legislature.
Maintain speaker’s bureau of Massachusetts Women Veterans to enhance awareness of women
veterans’ contributions to the Commonwealth and to the Nation.
Maintain library of historical resources on women veterans.
Continue education and awareness of current and emerging issues that impact women
veterans. . . .
Meet with [women veterans] in the community to assist with services and benefits.
Maintain WVN Twitter and Facebook sites and outreach programs through social media.
Governor’s Advisory Committee on Women Veterans
Section 2 of Chapter 115 of the General Laws requires DVS to “appoint an advisory committee on
women veterans to investigate, foster and promote the interests of women veterans.” The committee is
required to have at least 11 members, including the following:
 the Secretary of DVS or their designee
 an advisor on women’s issues, appointed by the Governor
 the chair of the Massachusetts Commission against Discrimination or their designee
 three members of veterans’ organizations, appointed by the Governor
 a VSO, appointed by the Governor
 four women veterans, appointed by the commissioner of DVS, at least two of whom participate
in the VA Vietnam Veterans Outreach Program.3
The advisory committee hosts an annual luncheon to recognize women veterans.

3. This program was established in 1977 by the Disabled American Veterans, a nonprofit organization that provides benefits to
disabled veterans and their families. The program later became part of VA. The benefits are offered through the VA
network of Vet Centers.
Audit No. 2021-0018-3S Department of Veterans’ Services
Audit Objectives, Scope, and Methodology
5
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Department of Veterans’ Services
(DVS) for the period July 1, 2019 through June 30, 2021.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer, the
conclusion we reached regarding each objective, and where each objective is discussed in the audit
findings.
Objective Conclusion
1. Did DVS have a process in place to ensure that its Women Veterans’ Network (WVN)
achieved its intended purpose of identifying women veterans and informing them
about their potential eligibility for benefits?
No; see Finding 1
2. Did DVS have an advisory committee on women veterans in accordance with Section
2 of Chapter 115 of the General Laws?
No; see Finding 2
To achieve our audit objectives, we gained an understanding of DVS’s internal control environment
related to the objectives by reviewing applicable policies and procedures, as well as conducting inquiries
with DVS staff members and management. To obtain sufficient, appropriate audit evidence to address
the objectives, we performed the procedures described below.
To determine whether DVS had a process in place to ensure that WVN achieved its intended purpose,
we performed the following procedures.
 We interviewed the WVN director and the DVS chief of staff to discuss the process of WVN’s
outreach to women veterans. We also asked these officials whether DVS management had
developed any performance measures to evaluate WVN’s effectiveness and, if WVN did not
achieve desired results, whether DVS management identified what needed to be done to
improve WVN’s performance.
Audit No. 2021-0018-3S Department of Veterans’ Services
Audit Objectives, Scope, and Methodology
6
 We requested from DVS a list (current as of the time of our fieldwork4
) of WVN members. DVS
gave us a list as of February 2022 that consisted of 1,545 records. Approximately 90% of the
records were email addresses only. From this list, we identified 878 records that either
contained a name (152 items), or did not contain a name but had information in the email
address that allowed for the extraction of a name (726 items). We selected and tested a
judgmental, nonstatistical sample of 60 members from these 878 records. We emailed a
questionnaire to each of the 60 members and evaluated all 14 responses that we received to
assess each member’s knowledge of WVN, their interaction with WVN, their awareness of the
types of assistance WVN provides, and their perspective on WVN’s overall effectiveness.
 Because there were no documented policies and procedures related to WVN, we used the job
description for the WVN director position to identify activities that WVN is expected to
complete. From this job description, we compiled a list of 16 outreach-related duties and
requested documentation to substantiate that the director had performed them during our
audit period.
To determine whether DVS had an advisory committee on women veterans in accordance with Section 2
of Chapter 115 of the General Laws, we performed the following procedures.
 We interviewed DVS’s chief of staff, general counsel, and other key staff members responsible
for monitoring and guiding the advisory committee’s activities to discuss the committee’s
operating procedures and how the committee is managed.
 We interviewed the Secretary of DVS to obtain knowledge and insight on committee history,
committee appointments, and the support and guidance provided to the committee by DVS
management during the audit period.
 We requested a list of committee members from the audit period, as well as any details related
to nominations, appointments, and membership terms. We reviewed a list of active advisory
committee members as of June 30, 2021. DVS could not provide details related to committee
members’ nominations, appointments, or membership terms.
 We interviewed two people who were members of the committee as of the end of the audit
period to determine each of these members’ level of involvement with the committee, how the
committee operates, how often it meets, and what its overall activities are.
Where nonstatistical sampling was used, we could not project the results of our testing to the overall
populations.
Data Reliability Assessment
The list of WVN members that DVS provided consisted of 1,545 email addresses. This list was the only
source of information that DVS could provide regarding WVN members. We interviewed the WVN

4. Because DVS does not maintain historical versions of WVN’s membership list, DVS could not provide a list of WVN members
as of the end of our audit period. Instead, DVS provided a then-current list of WVN members.
Audit No. 2021-0018-3S Department of Veterans’ Services
Audit Objectives, Scope, and Methodology
7
director, who is responsible for maintaining the list, and tested the list for duplicate records. In addition,
we scanned the list for records that did not contain names and did not have information in the email
address that allowed us to extract a name. We excluded all records without associated names. Because
the list was missing key identifying information, such as members’ names, addresses, and telephone
numbers, we could not determine whether some of the records were duplicates. Therefore, we could
not determine the exact number of WVN members. However, because we only used this list to create
the recipient list for our questionnaire, we determined that the data obtained for our audit period were
sufficiently reliable.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
8
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Department of Veterans’ Services did not ensure that its Women
Veterans’ Network achieved its intended purpose.
The Department of Veterans’ Services (DVS) did not ensure that its Women Veterans’ Network (WVN)
achieved its intended purpose of identifying women veterans and informing them about their potential
eligibility for benefits. We identified 16 tasks that DVS had designated for the WVN director to perform
to achieve its objectives. Key tasks included conducting outreach, collecting and maintaining complete
and current information on women veterans in Massachusetts, and storing this information in a
database. DVS uses the information in the database to communicate information to women veterans
about topics like veteran benefits, programs, events, and support services available through other state
agencies and/or the federal government. However, during our audit period, DVS could only provide
limited documentation—in some cases, no documentation—to substantiate the extent to which the
WVN director performed these 16 tasks. As a result, there is a higher-than-acceptable risk that WVN did
not complete many of DVS’s tasks, including outreach to women veterans.
For example, the WVN director stated that she had met with representatives from several other state
agencies, including the Office of the State Treasurer and Receiver General, Department of Transitional
Assistance, and Department of Mental Health, to discuss women veterans’ issues. However, she could
not provide us with any documentation of these meetings. She also stated that during our audit period
she had responded to inquiries from women veterans and provided them with general information and
referrals to support services. However, there was no documentation to substantiate this assertion.
Finally, the director stated that during our audit period she had met with DVS’s director of legislative
and media relations to assess the effectiveness of DVS’s outreach efforts completed via the WVN
website. However, we found that DVS did not collect any information that it could use to perform such
an assessment.
If WVN does not complete DVS’s tasks, including outreach to women veterans, women veterans may
not be aware of all of the resources available to them through DVS. For example, during our audit, DVS
gave us a list of 1,545 records of women veterans it had identified as the total population of women
veterans in WVN’s database. However, this number represents only about 6% of the approximately
25,000 women veterans living in Massachusetts as of September 30, 2020 according to the United
States Department of Veterans Affairs’ (VA’s) National Center for Veterans Analysis and Statistics.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
9
Authoritative Guidance
The WVN director’s job description lists 16 tasks that they should perform to support women veterans in
Massachusetts:
Oversee and maintain a database of over 14,500 women veterans in the Commonwealth.
Organize and lead regular meetings of the Women Veterans’ Network Steering Committee [a
committee of community leaders and women veterans that supports and guides WVN’s mission].
Conduct aggressive research via informational fairs and workshops, public events and other
mediums to expand membership in the Women Veterans’ network.
Collaborate within DVS with other agencies at state, federal and non-profit provider levels that
serve women veterans to identify resources, improvements to programs and services to women
veterans, particularly in terms of healthcare, counseling, employment, education and housing.
Plan and coordinate the annual Woman Veterans’ Appreciation Day at the State House each
November.
Coordinate nominations for the Deborah Sampson award (Outstanding Woman Veteran of the
Year).
Respond to direct inquiries from women veterans to provide information, referrals and establish
linkages with supportive services.
Publish both state wide print and online versions of the woman veterans’ newsletter.
Provide informational updates via email, website, and social network sites on services, programs,
and events between newsletters. . . .
Prepare reports as required and requested by the Department of Veterans’ Services, Executive
Office of Health [and] Human Services, and the Legislature.
Maintain speaker’s bureau of Massachusetts Women Veterans to enhance awareness of women
veterans’ contributions to the Commonwealth and to the Nation.
Maintain library of historical resources on women veterans.
Continue education and awareness of current and emerging issues that impact woman
veterans. . . .
Meet with woman [veterans] in the community to assist with services and benefits.
Receives and answers questions and inquiries from women veterans currently deployed overseas
via social networks requesting benefit information and care packages. . . .
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
10
Meets with DVS Director of Legislation and Media Relations to ensure outreach efforts via website
are effective and productive.
Reasons for Issue
DVS has not established policies and procedures, including a monitoring component, for WVN’s
operations to ensure that all required activities are conducted and properly documented.
Recommendation
DVS should establish policies and procedures, including a monitoring component, for WVN’s operations
to ensure that all required activities are conducted and properly documented.
Auditee’s Response
1. DVS has established policies and procedures for the Women Veterans’ Network (WVN)
Steering Committee.
a. The WVN Steering Committee is comprised of women veterans from the [United States
Veterans Administration, or VA], the military, the [Vet] Centers, and Veterans Services
Organizations from around the Commonwealth.
b. When joining the Steering Committee, members sign an agreement to complete the
volunteer roles they are assuming. This includes: Each member of the steering
committee is required to attend monthly meetings, and to participate in the WVN’s two
signature annual events: the Women Veterans’ Conference, and the Deborah Sampson
Award recognition.
c. The WVN Steering Committee meets monthly. During the audit period, most Steering
Committee meetings were held virtually due to distance and [2019 coronavirus]
precautions. As of the Fall of 2021, the Steering Committee has resumed in person
meetings, and meetings have taken place at the following locations: Winthrop Town Hall,
VA Bedford, VA Boston, Clear Path for Veterans New England, DVS Offices at 600
Washington Street.
2. DVS is in process of establishing standard policies and procedures for the outreach and
engagement work done by Women Veterans’ Network employees. This includes:
a. Standard operating procedures regarding planning the Women Veterans’ Conference.
b. Procedures regarding the nominations and awards process for the Deborah Sampson
Award.
c. Procedures regarding providing support to individual women veterans.
d. Procedures planning outreach activities in the community.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
11
3. DVS is nearing the final stages of re-platforming all its benefits processing systems and
databases to a newer system called OnBase. The Women Veterans’ Database is one of these
sets of data that is currently in the process of migration with additional functionality built out.
This process is aimed for completion Fall 2022.
a. DVS’ [information technology] partners have identified service data for nearly 20,000
women Veterans in the Commonwealth that is being re-verified for currency and validity
before being migrated to OnBase.
b. DVS also periodically receives discharge information for veterans returning to the
Commonwealth. This data is reviewed and analyzed for currency and validity through
Operations and Data Analysis teams, and Women Veterans receive a direct mailing
brochure on DVS programs including the Women Veterans Network. Each individual
municipality also receives information on veterans returning to their municipality and
reach out to them individually.
c. DVS has worked with its database administrator to build functionality for the newest
iteration of the database structure for the Women Veterans’ Network. . . . This additional
functionality will include:
i. Allowing referrals from [veterans’ services officers, or VSOs, and the Statewide
Advocacy for Veteran Empowerment Program, or SAVE] to the Women Veterans’
Network for follow up
ii. Logging and tracking outreach activities
iii. Logging and tracking support provided to individual women veterans
iv. Tracking subject matter interests of individual women veterans for outreach and
activities
v. Integration with e-newsletter distribution
4. During the audit period, DVS participated in outreach and engagement activities to recognize,
honor, and outreach to women veterans:
a. Events:
i. WVN hosted events:
1. November 8, 2019: WVN hosted 2019 Women Veterans’ Appreciation Day and
presented the Deborah Sampson Award.
2. October 3, 2020: Virtual Women Veterans’ Conference, special topics on
employment and mental health, benefits
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
12
3. 2020: Two (2) employment workshops cohosted by WVN and the [Executive
Office of Health and Human Services] Diversity Office to raise awareness of
hiring Veterans.
4. June 22, 2020: Managing Stress and Trauma During the Pandemic: A Forum for
Women Veterans, in partnership with Brookfield Institute
5. March 23, 2021: Virtual Forum: Parenting After Your Service
6. February 24, 2021: Virtual Forum: Heart Health for Women Veterans
7. June 11, 2021: Women [Veterans’] Appreciation Day & Deborah Sampson
recognition event in Sharon
ii. Community-hosted outreach events attended
1. July 19, 2019: WVN attended Run to Home Base
2. September 21, 2019: WVN Attended Veterans/Military Expo at VA Brockton
3. September 29, 2019: WVN attended Vettes to Vets Day
4. February 16, 2020: Outreach at Massachusetts All Nurses Medical Post 296 to
celebrate Black History month.
5. The [WVN] Director participated in the following Stand Downs [annual veteran
service events] during the period:
a. Worcester Stand Down on August 20, 2020
b. Worcester Stand Down on June 18, 2021
b. The WVN provides informational updates, connections to programs, and updates on
events and services through digital communication channels including its website, social
media, and via e-newsletter.
i. Sent 14 e-newsletters during the audit period
1. Overall, 20,189 emails were sent to e-newsletter subscribers, resulting in 3,800
email opens and 431 clicks on links and resources for women veterans
ii. In April, 2021, the WVN went live with its new website,
www.womenveteransnetworkmass.org, which has information, connections to the
network, and information on events for women veterans. Between April and June
2021, the site received 1250 visits.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
13
iii. Social media metrics
1. Facebook: During the audit period of July 1, 2019–June 30, 2021, the WVN had
3,898 followers and made 62 posts.
2. Twitter: During the audit period, WVN tweeted 225 times, earning 30,400
impressions, an average of 40 impressions per day, 30 link clicks, 40 retweets,
66 likes, and 6 replies.
c. The Network Director has provided direct assistance to women veterans in need. While
below are two examples, the re-platformed database will capture units of direct
assistance such as this. As an example:
i. In November of 2020 the Network Director assisted the mother of a service member
who died on active duty. We learned that she was an Air Force Veteran and the
Network Director connected her to her VSO and other services.
ii. The Network Director has collaborated with the Women’s Lunch Place and New
England Center to assist a homeless woman veteran with housing.
d. The WVN also collaborates with SAVE (Statewide Advocacy for Veteran Empowerment)
Team to crosswalk on issues.
i. The SAVE team takes referrals for veterans experiencing difficulties accessing
support and services. DVS has added one (1) full time employee who supports SAVE
and the WVN (hired in May 2022), which expands the reach of the Network, and the
supports DVS can provide for women veterans in the Commonwealth.
e. The WVN maintains a compendium of women veteran speakers and resources around
the Commonwealth.
i. The WVN locates women veteran speakers for engagements across the
Commonwealth. While the Speaker’s Bureau may not be in the same format, the
Network has been able to locate speakers when others request a woman veteran
speaker through its wide array of women veterans across the Commonwealth. We
can respond based on women’s service era, branch, or geographic location.
Additionally, with the significant additional female VSOs this provides more capacity
and opportunity for women veteran speakers. Going forward, we will expand this
bureau through social media recruitment.
ii. For example, the WVN has also begun a listing of Women’s Memorials around the
Commonwealth. The intent is to provide a historic trail so that women veterans may
visit these memorials when they are out in the Commonwealth. Once this is
complete, this will be shared online as a resource for Women Veterans to raise the
awareness of women veterans’ service and sacrifice.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
14
Auditor’s Reply
Contrary to what DVS asserts in its response, it has not established policies and procedures for WVN’s
operations and steering committee. DVS may request that steering committee members sign an
agreement that outlines the requirements of each volunteer role and that they participate in meetings
and attend certain other events. However, these written agreements do not constitute policies and
procedures that would establish how each member’s compliance with the terms and conditions of their
agreement would be evaluated, monitored, and documented. As noted above, the Office of the State
Auditor (OSA) identified 16 tasks that DVS had designated for the WVN director to perform to achieve
WVN’s program objectives; none of these were recorded in formal policies and procedures.
In its response, DVS delineates specific activities that WVN may have conducted during our audit period.
However, as noted above, when we requested it during the audit, DVS could not provide adequate
documentation to support the completion of most of these activities. For example, in its response, DVS
indicates that the WVN steering committee met monthly and that these meetings were virtual during
the audit period because of 2019 coronavirus pandemic precautions. However, when we requested
supporting documentation for these meetings during the audit, the WVN director told us that she did
not maintain agendas or meeting minutes and that there was no formal structure to these meetings.
We acknowledge that WVN uses its website, social media, and its newsletter to provide information on
benefits, programs, resources, and events to women veterans. However, as noted above, as of the time
of our fieldwork, there were only 1,545 records of women veterans in WVN’s database. This number
represented only about 6% of the approximately 25,000 women veterans living in Massachusetts as of
September 30, 2020. In OSA’s opinion, DVS needs to improve its outreach efforts to women veterans.
We believe that developing written policies and procedures that establish how WVN outreach activities
should be conducted, monitored, documented, and evaluated should improve the effectiveness of these
efforts.
According to its response, DVS will establish policies and procedures for WVN’s outreach and
engagement activities. In addition, DVS will implement improvements to WVN’s database, which will
allow DVS to document and track WVN outreach activities and support provided to women veterans.
We believe that these measures are prudent, but we again urge DVS to establish policies and
procedures, including a monitoring component, for all WVN operations to ensure that all required
activities are conducted and properly documented.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
15
2. DVS did not have enough members appointed to its advisory committee
on women veterans.
DVS did not have an 11-member advisory committee on women veterans during the audit period. DVS
gave us a list of active advisory committee members as of June 30, 2021; there were only five
commissioners on the list, and one of them had not been a member of the committee since 2010.
As a result of this issue, DVS does not benefit from the unique knowledge, skills, and expertise of a full
committee. Moreover, because the composition and structure of the committee are intended to
enhance the effectiveness of DVS’s work with women veterans, that effectiveness may be at risk
without a full committee.
Authoritative Guidance
Section 2 of Chapter 115 of the Massachusetts General Laws requires DVS to “appoint an advisory
committee on women veterans to investigate, foster and promote the interests of women veterans.”
The committee is required to have at least 11 members:
 the Secretary of DVS or their designee
 an advisor on women’s issues, appointed by the Governor
 the chair of the Massachusetts Commission against Discrimination or their designee
 three members of veterans’ organizations, appointed by the Governor
 a VSO, appointed by the Governor
 four women veterans (at least two of whom participate in the VA Vietnam Veterans Outreach
Program), appointed by the commissioner of DVS.
Reasons for Issue
DVS officials did not explain why the committee did not have the required 11 members.
Recommendation
DVS should work with the Governor’s Office to ensure that the advisory committee has the required 11
members.
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
16
Auditee’s Response
DVS has continued to work on identifying and appointing members to the Advisory Committee on
Women Veterans through the audit period. DVS has worked with the Governor’s Office and the
Executive Office of Health and Human Services to identify appropriate candidates for these
important positions. As of June 4, 2022, the following appointments to the Committee have been
made:
1
Secretary of DVS or
Designee Secretary of DVS
Susan
McDonough
2
Appointed by the
Governor
Governor (has
been appointed)
Stephanie
Landry
3
Chairman of the
[Massachusetts
Commission against
Discrimination, or
MCAD] or Designee
Commissioner of
MCAD (has
been appointed)
Deirdre Ann
Hosler, Esq.
4
Member of Veteran
Organization
Governor (has been
appointed) Deb Freed
5
Member of Veteran
Organization
Governor (awaiting
appointment letter)
Caitlynn
Almy
6
Member of Veteran
Organization
Governor (awaiting
appointment letter)
Catherine
Corkery
7 Veterans Agent
Governor (has
been appointed) Karen Tyler
8
Women veteran
(Participant in Vietnam
Veterans Outreach
Program of the [United
States Department of
Veterans Affairs, or VA])
Secretary of DVS
(awaiting being
sworn in)
Tiffany
Lever
9
Women veteran
(Participant in Vietnam
Veterans Outreach
Program of the [VA])
Secretary of DVS
(awaiting being
sworn in) Paula Smith
10 Women veterans
Secretary of DVS (has
been appointed)
Dr. Shakti
Sabharwal
11 Women Veterans
Secretary of DVS
(awaiting being
sworn in)
June
Newman
Non-statutory
members
Secretary of DVS
(awaiting being
sworn in)
Carolyn
Mason
Wholley
Secretary of DVS
(awaiting being
sworn in) Liseth Velez
Audit No. 2021-0018-3S Department of Veterans’ Services
Detailed Audit Findings with Auditee’s Response
17
Auditor’s Reply
Based on its response, DVS is taking measures to address our concerns in this area.

Official Audit Report – Issued June 16, 2021
Disabled Persons Protection Commission
For the period July 1, 2017 through June 30, 2019
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
erely,
nne M. Bump
June 16, 2021
Ms. Nancy Alterio, Executive Director
Disabled Persons Protection Commission
300 Granite Street, Suite 404
Braintree, MA 02184
Dear Ms. Alterio:
I am pleased to provide this performance audit of the Disabled Persons Protection Commission. This
report details the audit objectives, scope, methodology, findings, and recommendations for the audit
period, July 1, 2017 through June 30, 2019. My audit staff discussed the contents of this report with
management of the agency, whose comments are reflected in this report.
I would also like to express my appreciation to the Disabled Persons Protection Commission for the
cooperation and assistance provided to my staff during the audit.
Sinc
Suza
Auditor of the Commonwealth
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 6
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE ....................................................................................12
1. The Disabled Persons Protection Commission did not ensure that alleged abusers were always advised
of their rights. ........................................................................................................................................12
2. DPPC does not always complete its investigations within the required timeframes or document the
reasons for not doing so. ........................................................................................................................15
3. DPPC did not ensure that it consistently received final PSPs from providers for victims of alleged
abuse. ....................................................................................................................................................19
4. DPPC did not always identify and properly document individuals who had been identified as alleged
abusers in multiple reports.....................................................................................................................21
Audit No. 2020-0046-3S Disabled Persons Protection Commission
List of Abbreviations
ii
LIST OF ABBREVIATIONS
APS adult protective service
CMR Code of Massachusetts Regulations
DDS Department of Developmental Services
DMH Department of Mental Health
DPPC Disabled Persons Protection Commission
IR Initial Response
MRC Massachusetts Rehabilitation Commission
PSP protective service plan
SPDU State Police Detective Unit
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Disabled Persons Protection Commission (DPPC) for
the period July 1, 2017 through June 30, 2019. The objective of our audit was to follow up on the issues
identified in our prior audit report (No. 2015-0046-3S) to determine what measures, if any, DPPC had
taken to address them, as well as the adequacy of those measures. In addition, we reviewed DPPC’s
ability to ensure that protective service plans (PSPs) were submitted in a timely manner, its compliance
with regulatory requirements for abuse investigations, and its investigation of individuals with a history
of abuse allegations.
Below is a summary of our findings and recommendations, with links to each page listed.
Finding 1
Page 12
DPPC did not ensure that alleged abusers were always advised of their rights.
Recommendation
Page 13
DPPC should enhance its policies and procedures by implementing effective monitoring
controls to ensure that alleged abusers are made aware of their rights before being
interviewed for abuse investigations.
Finding 2
Page 15
DPPC does not always complete its investigations within the required timeframes or
document the reasons for not doing so.
Recommendations
Page 16
1. DPPC should enhance its policies and procedures by implementing effective monitoring
controls to ensure that investigators complete Initial Responses (IRs) and 19C reports
within the required timeframes and that when filing deadlines are not met, evidence of
the reasons for the delay is documented, verified, and retained in case files.
2. DPPC should continue to work with the Department of Developmental Services, the
Department of Mental Health, and the Massachusetts Rehabilitation Commission to
complete IRs for emergency cases within 24 hours, IRs for non-emergency cases within
10 days, and 19C reports within 30 days.
Finding 3
Page 19
DPPC did not ensure that it consistently received final PSPs from providers for victims of
alleged abuse.
Recommendation
Page 19
DPPC should implement effective monitoring controls within its policies and procedures to
ensure that providers complete PSPs within required timeframes.
Finding 4
Page 21
DPPC did not always identify and properly document individuals who had been identified as
alleged abusers in multiple reports.
Recommendation
Page 22
DPPC should establish formal policies and procedures, and develop monitoring controls, to
ensure that all staff members identify alleged abusers who have been involved with three or
more reports of abuse and document this information in the “Other Pertinent Information”
field of the Intake Abuse Form.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Disabled Persons Protection Commission (DPPC), established in 1987 by Chapter 19C of the
Massachusetts General Laws, is an independent state agency responsible for the investigation and
remediation of abuse of people with disabilities in the Commonwealth. According to its website, DPPC’s
mission is “to protect adults with disabilities from the abusive acts or omissions of their caregivers
through investigation, oversight, public awareness and prevention.” To carry out its mission, DPPC
performs its own investigations and oversees and directs investigations conducted on its behalf by the
Department of Developmental Services (DDS), the Department of Mental Health (DMH), and the
Massachusetts Rehabilitation Commission (MRC). DPPC received 11,900 and 13,102 abuse reports in
fiscal years 2018 and 2019, respectively.
DPPC’s website states,
The jurisdiction of DPPC includes adults with disabilities between the ages of 18 and 59, who are
within the Commonwealth whether in state care or in a private setting and who suffer serious
physical and/or emotional injury through the act and/or omission of their caregivers. The DPPC
enabling statute fills the gap between the Department of Children and Families (DCF) (through
the age of 17) and the Executive Office of Elder Affairs (EOEA) (age 60 and over) statutes.
DPPC’s three Governor-appointed commissioners report to the Governor and the Legislature. The
commissioners submit an annual report to the Governor and the Legislature outlining actions they have
taken; names, salaries, and duties of all employees; money DPPC has disbursed; and other matters
related to DPPC’s jurisdiction that they deem necessary. The executive director, who reports to the
commissioners, takes care of DPPC’s day-to-day operations and the oversight of its staff members.
During our audit period, DPPC was located at 300 Granite Street, Suite 404, in Braintree. It had
approximately 50 full-time employees as of November 2019, including managers, intake specialists,
oversight officers, investigators, and support employees. DPPC had state appropriations of $3.64 million
in fiscal year 2018 and $4.92 million in fiscal year 2019.
FileMaker Pro
DPPC’s case management system is a customized product called FileMaker Pro. It contains confidential
information about alleged abusers’ background information, abuse allegations, and investigations.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
3
Screening of Abuse Reports Submitted through DPPC’s Hotline
Citizens and mandated reporters1 can submit abuse reports involving adults with disabilities through a
24-hour phone hotline that is operated by DPPC’s Intake Unit from 9:00 a.m. to 3:30 p.m. and by a
DPPC-trained independent contractor after hours (from 3:30 p.m. to 9:00 a.m.). Although most abuse
reports are submitted through the hotline, they can also be submitted via email, by fax, or in person.
The Intake Unit receives, documents, and evaluates the information provided by each reporter
regarding the alleged victim, the alleged abuser, and the nature of the incident.
The hotline operator ensures that the alleged victim is in a safe environment and completes an Intake
Abuse Form in FileMaker Pro, where the form is assigned a unique case number and will ultimately be
screened in or out for investigation. An intake or oversight manager then reviews the completed Intake
Abuse Form within a day, evaluates whether the case is within DPPC’s jurisdiction, and makes the final
screening decision. If the case is within DPPC’s jurisdiction, it is screened in and assigned to DDS, DMH,
or MRC (depending on the alleged victim’s type of disability) to investigate on DPPC’s behalf. If it is not,
the case is screened out and sent to another state agency, such as the Department of Public Health or
the Executive Office of Elder Affairs, depending on the alleged victim’s type of disability, age, and
location.
In addition, the Massachusetts State Police Detective Unit (SPDU) reviews all abuse reports submitted to
DPPC’s hotline for indication of criminal activity. If SPDU determines that a possible criminal act has
occurred, the report is forwarded to the appropriate district attorney’s office for review.
Abuse Investigations and Reports
Intake managers review screened-in cases to determine how best to protect the alleged victims. Each
case is assigned to an adult protective service (APS) investigator, as well as a DPPC oversight officer who
is responsible for monitoring the progress of the investigation and reviewing the resulting report for
compliance with Section 5 of Title 118 of the Code of Massachusetts Regulations (CMR). The APS
investigator may be from DPPC’s Investigations Unit or may be a DDS, DMH, or MRC investigator
investigating on DPPC’s behalf. Factors for assigning staff members for investigation and oversight
include, but are not limited to, the alleged victim’s disability; the type of allegation; DPPC investigator

1. Mandated reporters are obligated to report suspected abuse and/or neglect. They include medical doctors, teachers, police
officers, school administrators, and guidance counselors. For a full definition, see Section 1 of Chapter 19C of the General
Laws and Section 3.03 of Title 118 of the Code of Massachusetts Regulations.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
4
availability; and whether there have been multiple prior reports regarding the same victim, abuser, or
program.
The investigator is required to file a report with DPPC that has two parts: an Initial Response (IR) and an
Investigation Report (referred to herein as a 19C report). The IR and 19C report, when fully completed,
are designed to ensure that each abuse investigation has met the minimum requirements of 118 CMR
5.02. IRs are used by investigators to document the results of their initial risk assessments and their
preliminary investigation findings. Investigators must submit IRs to the DPPC Oversight Unit within 24
hours of assignment for cases determined to be emergencies2 and within 10 calendar days for nonemergency3 cases. Cases are administratively closed when facts gathered during the IR phase indicate
that abuse did not occur. During our audit period, investigators completed 3,451 IRs.
The 19C report includes information intended to ensure that the investigator used evidence gained
through interviews, documentation, and site visits to conclude whether it is likely that abuse occurred.
Abuse is substantiated when there is enough evidence to conclude that an act, or neglect, by the alleged
abuser resulted in serious physical or emotional injury to the alleged victim. Abuse is unsubstantiated
when there is not enough evidence to conclude that the alleged abuser caused serious injury to the
victim. The investigator must submit the 19C report, including recommendations about protective
services required in order to address the situation and mitigate further risk when abuse is substantiated,
to DPPC within 30 calendar days from the date the case was assigned. If the investigator cannot submit
a 19C report within this timeframe and can provide a good cause (i.e., an explanation), s/he may request
an extension to be approved by DPPC. DPPC’s Oversight Unit emails notices to investigative units (once a
week to DPPC’s Investigations Unit and once a month to external agencies), alerting them to overdue
reports. During our audit period, DPPC completed 3,291 19C reports.
If SPDU finds any indication of criminal activity during its review, it refers the case to a district attorney’s
office. This may result in a criminal investigation; the district attorney’s office can request that DPPC
delay its investigation until the criminal investigation ends. During that time, DPPC monitors the
progress of the criminal investigation, consulting with SPDU and/or the district attorney’s office to
determine when or whether DPPC’s abuse investigation can be initiated or resumed.

2. According to 118 CMR 2.02, an emergency is “a situation involving an allegation of the presence of imminent Serious
Physical Injury or Serious Emotional Injury, or both, to a Person with a Disability that requires an immediate response to
protect the Person with a Disability from such Serious Physical Injury or Serious Emotional Injury.”
3. According to 118 CMR 2.02, a non-emergency is “a situation of alleged Abuse that is not an Emergency.”
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
5
If the investigator substantiates abuse, the assigned protective agency, such as DDS or DMH, must
submit a protective service plan (PSP) to DPPC within 30 days after the 19C report is completed. A PSP
includes an assessment of the abuse incident and recommendations to protect the victim from being
abused again. PSPs submitted after the due date are classified as overdue by FileMaker Pro. During our
audit period, DPPC completed 640 cases with findings of substantiated abuse.
Alleged Abuser Rights
Under 118 CMR 5.02(2), alleged abusers of people with disabilities have certain rights when being
interviewed as part of DPPC investigations. Recognizing its obligation to communicate these rights to
alleged abusers and to document that it has done so, DPPC has developed a form called the Notice of
Alleged Abusers’ Rights. Before beginning an interview, the alleged abuser is allowed time to review the
form, and the investigator answers any questions the individual has about its contents.
The investigator enters a check in the box on the 19C report, documenting that the form has been
provided to the alleged abuser. The investigator is allowed to mail a copy of the form or to advise the
alleged abuser of his/her rights over the phone if the investigator cannot do so in person. If the
investigator cannot provide notice of the rights to the alleged abuser, the investigator must record the
reason in the 19C report. However, the investigator can still conduct the interview. If the investigator
cannot interview the alleged abuser, s/he must record the reason in the 19C report.
After the completion of a 19C report containing a finding of substantiated abuse, DPPC mails a redacted
copy of the report and a written notice to the person identified as the abuser, informing the abuser of
his/her right to respond in writing to DPPC and contest the findings of the report, provided that notifying
the abuser will not place the victim at risk of further harm. If the investigator believes notifying the
abuser may present further risk to the victim, the investigator may recommend that the abuser not be
notified of the case’s outcome. In this instance, the investigator must submit a Recommendation to
Withhold Abuser Notification Form to a DPPC oversight officer. Upon receipt of the form, the DPPC
oversight officer makes a copy and submits it to DPPC’s Legal Unit for approval.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
6
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Disabled Persons Protection
Commission (DPPC) for the period July 1, 2017 through June 30, 2019.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Did DPPC advise alleged abusers of certain rights during abuse investigations in
accordance with Section 5.02 of Title 118 of the Code of Massachusetts Regulations
(CMR), Section 5 of Chapter 19C of the General Laws, and DPPC’s investigation policy
“Invest-137”?
No; see Finding 1
2. Did DPPC properly determine jurisdiction for abuse investigations during screening in
accordance with 118 CMR 4.03?
Yes
3. Did DPPC ensure that 19C reports and protective service plans (PSPs) were submitted
within the timeframes mandated by 118 CMR 5.02(3)(a) and 7.03(3)?
No; see Findings 2
and 3
4. Did DPPC’s abuse investigations meet the requirements of 118 CMR 5.02(1)? Yes
5. Did DPPC conduct investigations of alleged abusers with three reports of suspected
abuse as required by its intake, investigation, and oversight policies?
No; see Finding 4
DPPC has made some improvements in the areas reviewed since our prior audit (No. 2015-0046-3S).
Specifically, DPPC has assessed its operating procedures related to how alleged abusers are advised of
their rights based on its regulatory requirements and operational needs. It has also established and
implemented policies and procedures to help ensure that when required filing deadlines are not met,
evidence of the reasons for the delays is documented and retained in case files.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
7
To achieve our audit objectives, we gained an understanding of the internal controls we deemed
significant to the objectives by reviewing agency policies and procedures and conducting inquiries with
DPPC’s staff members, management, and shareholders. We tested the controls’ operating effectiveness
over the following areas: advisement of alleged abuser rights, manager approval of screening decisions
during intake, extension requests and notices sent for overdue reports, and 19C report completeness.
We identified an issue with DPPC oversight officers’ review and approval of investigator responses in
19C reports as an internal control for ensuring that investigators provided an explanation when alleged
abusers were not advised of their interview rights in accordance with DPPC’s investigation policy
“Invest-137” (Finding 1).
We performed the following procedures to obtain sufficient, appropriate audit evidence to address the
audit objectives.
Abuser Interview and Petition Rights
To determine whether investigators informed alleged abusers of their interview rights, we selected a
statistical random sample of 60 cases from a population of 3,291 cases completed during the audit
period, with a 95% confidence level, 5% tolerable error rate, and 0% expected error rate. We reviewed
the corresponding 19C reports for evidence that each investigator had noted that the alleged abuser
was given a copy of the Notice of Alleged Abuser’s Rights or was informed of his/her rights by phone or
mail. In instances where notice of rights was not provided, we determined whether an explanation for
this was noted in the 19C report.
To determine whether abusers were notified of their right to contest the findings of abuse
investigations, we selected a nonstatistical random sample of 50 cases out of a population of 640 cases
with substantiated abuse that were completed during the audit period. We verified that notification
letters were mailed to abusers. We also verified that DPPC maintained a Recommendation to Withhold
Abuser Notification Form approved by DPPC’s Legal Unit for instances where DPPC determined that
notifying the abuser might pose a risk to the victim. Since we used a nonstatistical approach, we did not
project our results to the entire population.
Determination of Jurisdiction for Abuse Investigations
To assess DPPC’s determination of jurisdiction for abuse reports screened in for investigation, we
selected a random statistical sample of 30 cases from a population of 3,445 cases screened in during the
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
8
audit period, with a 95% confidence level, 10% tolerable error rate, and 0% expected error rate. These
cases were designated in FileMaker Pro as “screening decision 4B” (cases where the alleged abuser was
employed by a state agency, which would require DPPC to conduct the abuse investigation) or
“screening decision 4C” (cases where the alleged abuser was not employed by a state agency) based on
the relevant sections of Chapter 19C of the General Laws. We reviewed each case’s intake data to
determine whether the alleged victim was a person with a disability, whether the alleged abuser was a
caretaker of the alleged victim, and whether Intake Department personnel noted that an act or omission
of the caretaker resulted in the alleged victim’s injury in accordance with 118 CMR 4.03.
To assess DPPC’s determination of jurisdiction for abuse reports screened out at intake for not meeting
the requirements of 118 CMR 4.03, we selected a random statistical sample of 30 cases from a
population of 19,342 cases screened out during the audit period, with a 95% confidence level, 10%
tolerable error rate, and 0% expected error rate. These cases were designated in FileMaker Pro as
“screening decision 4A” or “OUT.” Screening decision 4A includes reports involving a hospital or
nursing/long-term-care facility,4 referred to the Department of Public Health; reports involving a victim
under the age of 18, referred to the Department of Children and Families; and reports involving a victim
over the age of 59, referred to the Executive Office of Elder Affairs. We reviewed each case’s intake data
to confirm that the reported abuse did not meet the criteria to be screened in for DPPC investigation.
DPPC received 25,002 abuse reports during the audit period.
Initial Response and 19C Report Statutory Timeframes
To determine whether the DPPC Oversight Unit received Initial Responses (IRs) for non-emergency cases
within 10 days of reported abuse, we selected a random statistical sample (with a 95% confidence level,
5% tolerable error rate, and 0% expected error rate) of 60 of the 2,925 cases that had not been referred
to law enforcement,5 within a total population of 3,291 cases completed during the audit period. We
compared the IR due date of each case to the IR received date to determine whether any IRs were
submitted late.

4. According to 118 CMR 2.02, a long-term-care facility is “a convalescent home, nursing home, rest home, or charitable home
for the aged licensed by the Department of Public Health.”
5. We determined that deadlines were not updated in FileMaker Pro for cases delayed because of ongoing criminal
investigations. An analysis of the case data showed that 98% of the 366 IRs (out of 3,291 IRs) for cases completed during
the audit period that were referred to law enforcement were submitted after their due dates. Therefore, we exercised
auditor judgment to conclude that focusing on cases that were not referred to law enforcement would be a better
representation of investigators’ ability to meet statutory timeframes.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
9
To determine whether 19C reports were received within 30 days of reported abuse, we used the
previously mentioned sample of 60 of 2,925 cases. We compared the 19C report due date of each case
to the 19C report received date to identify 19C reports that were not received with 30 days of the
reported abuse. We also calculated updated 19C report due dates for cases that were granted
extensions6 and compared them to the 19C report received dates to determine whether those reports
met the updated deadlines.
We performed an analysis comparing the screen-in dates of cases with substantiated abuse to the
completion date of each victim’s previous case with substantiated abuse, if present, to produce a subset
of 44 cases with substantiated abuse that occurred during ongoing investigations. Through a preliminary
review, we identified nine victims, associated with 27 of the 44 cases, that we believed had experienced
additional abuse during ongoing investigations.
To determine whether PSPs were received within 30 days of the completion of 19C reports, we selected
a random nonstatistical sample of 60 cases from a population of 640 cases with substantiated abuse
that were completed during the audit period. We compared each PSP’s received date to the due date to
determine whether it was received within 30 days of the completed abuse investigation. We performed
the same test for all 25 substantiated cases that included criminal prosecutions resulting in guilty
verdicts. (The process for submitting PSPs for these cases is the same for those that did not include a
criminal prosecution.) Since we used a nonstatistical approach, we did not project our results to the
entire population.
Minimum Regulatory Requirements for Abuse Investigations
To test for regulatory compliance, we selected a random statistical sample of 30 of the 3,291 cases
completed during the audit period, with a 95% confidence level, 10% tolerable error rate, and 0%
expected error rate. We reviewed the 19C reports for evidence that the minimum requirements for
abuse investigations had been completed. These requirements are outlined in 118 CMR 5.02(a)
through (o):
(a) an interview with the person with a disability who was allegedly abused. . . .
(b) a visit to and evaluation of the site of alleged abuse. . . .

6 DPPC does not update due dates in FileMaker Pro, so cases with extension requests show up as overdue.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
10
(c) a determination of the nature, extent, and cause or causes of the injuries, if possible; or
a determination of whether abuse per se exists;
(d) use of the preponderance of evidence standard to substantiate or un-substantiate the
existence of abuse leading the investigator to conclude that it is more likely than not
that abuse does or does not exist;
(e) a determination or confirmation, if possible, of the identity of the alleged abuser(s),
whether named or not named in the Intake. . . .
(f) a determination of the identity of the person(s) who was/were responsible for the
health and welfare of the alleged victim(s) when the alleged incident occurred . . .
(g) an initial assessment of the immediate protective services needs of the person with a
disability who is the alleged victim of abuse. . . .
(h) an interview with all available witnesses to the abuse. . . .
(i) an interview with the alleged abuser(s), unless such an interview would create
additional risk of harm to the person with a disability;
(j) a determination that all relevant physical evidence of the alleged abuse has been
preserved . . .
(k) the review and obtaining of copies of all documents which are not plainly irrelevant to
the matter under investigation . . .
(l) an interview with the reporter;
(m) a determination in cases in which abuse is not substantiated as to whether the
allegation reported to the Commission constitutes a false report . . .
(n) any other tasks that, in the discretion of the Commission, are deemed appropriate and
are not plainly irrelevant to the investigation; and
(o) if an investigator does not perform one or more of the requirements in 118 CMR
5.02(1)(a) through (n), the investigator shall detail in the Investigation Report why the
requirement was not met and the Commission shall determine whether said
requirement(s) is material to the investigation.
We also determined whether investigators provided an explanation, in accordance with the regulation,
for any requirements not met.
Alleged Abusers with Three or More Reports of Abuse
We performed an analysis of all 3,291 cases completed during the audit period to identify alleged
abusers investigated for three or more incidents during that period that were not referred to DPPC. We
reviewed the associated Intake Abuse Forms to determine whether, for cases designated as “screening
decision 4B” (indicating that the caretaker is employed by a state agency) in FileMaker Pro, DPPC had
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
11
been assigned as the investigating agency based on the history of multiple reports, in accordance with
its intake, investigation, and oversight policies.
Data Reliability
We assessed the reliability of the data obtained from FileMaker Pro by interviewing knowledgeable
DPPC personnel about the system. We tested FileMaker Pro’s system controls, which included security
management, access control, and segregation of duties, and determined whether configuration
management and contingency planning policies were in place during the audit period.
We observed DPPC management extracting case data from FileMaker Pro. We then traced a sample of
20 completed cases, out of our dataset of 25,521 cases, to their source documents (Intake Abuse Form,
IR, 19C report, Notice of Alleged Abusers’ Rights, Investigation Extension Request Form, letter of referral
to law enforcement, and/or letter of referral to a district attorney’s office) provided by DPPC to assess
the accuracy of the data. We performed additional validity and integrity tests, including comparing the
total number of completed cases in the dataset to the totals in DPPC’s annual reports for fiscal years
2018 and 2019, verifying that there were no missing values in key fields, summarizing abuse reports to
ensure that there were no duplicates, testing for report dates outside the audit period, and validating
data values for screening decisions.
Based on the results of our assessment, we determined that the data were sufficiently reliable for the
purposes of our audit work.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
12
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Disabled Persons Protection Commission did not ensure that alleged
abusers were always advised of their rights.
In our prior audit, we found that the Disabled Persons Protection Commission (DPPC) did not ensure
that required forms were signed to document that alleged abusers were advised of their rights before
being interviewed.
In our current audit, we found that DPPC did not always ensure that alleged abusers were advised of
their rights before being interviewed for abuse investigations. In 7 of the 94 cases in our sample, the
alleged abusers were not advised of their rights, and the investigators did not document in the 19C
report why this requirement was not met. Two additional cases did not contain justification for not
advising the alleged abuser of the interview rights. Without such documentation, there is inadequate
assurance that the alleged abusers were made aware of, and understood, their rights and legal
obligations regarding the investigations.
Authoritative Guidance
Section 5.02(2) of Title 118 of the Code of Massachusetts Regulations (CMR) states that alleged abusers
of people with disabilities have the following rights when being interviewed as part of DPPC abuse
investigations:
a. to be accompanied during an interview by a person of his or her own choice who is 18
years of age or older; provided, that said companion shall not be a supervisor or
administrator employed by the alleged abuser’s employer, unless requested by the
alleged abuser and agreed to by the investigator; or is not otherwise involved or has an
interest in the matter under investigation. Other than being present during the interview,
said companion shall not participate in the actual conduct of the interview;
b. to be informed of the existence of the complaint and the general nature of the
allegations. Under no circumstances shall the identity of the reporter be disclosed;
c. to be informed that if he or she falls within the category of a mandated reporter, he or
she has an obligation pursuant to M.G.L. c. 19C to cooperate in the investigation and
truthfully provide to the investigator all information he or she may possess that is
relevant to the matter under investigation; and
d. to be informed that his or her refusal to be interviewed or to otherwise cooperate in the
investigation will be made a part of the report, and that if the person is an employee of a
state agency, unless his or her response to a question could be used against him or her
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
13
in a criminal proceeding, his or her failure to cooperate shall be reported to the person’s
supervisor and to the appropriate public agency for possible disciplinary action under that
agency’s regulations or pursuant to the provisions of any relevant collective bargaining
agreement or any other contract.
Accordingly, DPPC’s investigation policy “Invest-137” states,
2. Prior to commencing an interview of an individual being considered as an [alleged
abuser], the 19C investigator will provide the individual with a copy of the DPPC Notice of
Alleged Abuser’s Rights During DPPC Investigation Interview. . . .
4. The investigator will enter a check in the box on the 19C Investigation Report Form
documenting that the DPPC Notice of Alleged Abuser’s Rights During DPPC Investigation
Interview was provided to the [alleged abuser].
According to 118 CMR 5.02(1)(i), each investigation should include an interview with the alleged abuser.
According to 118 CMR 5.02(1)(o), if an investigator cannot fulfill one or more of the requirements in 118
CMR 5.02, the investigator must provide an explanation in the 19C report for a DPPC oversight officer to
review and approve.
Reasons for Noncompliance
DPPC had not implemented effective monitoring controls to ensure that alleged abusers were advised of
their rights before being interviewed for abuse investigations.
Recommendation
DPPC should enhance its policies and procedures by implementing effective monitoring controls to
ensure that alleged abusers are made aware of their rights before being interviewed for abuse
investigations.
Auditee’s Response
First, DPPC reiterates from the previous audit (2015-0046- 3S) its disagreement with the audit’s
characterization of the requirement of 118 CMR 5.02 (2). Pursuant to 118 CMR 5.02 (2), an
alleged abuser has certain rights available during an investigatory interview. That regulation does
not require the alleged abuser receive written notification of these rights, nor does it require the
alleged abuser sign a form acknowledging he/she has been advised of these rights.
Second, DPPC did in fact implement effective monitoring and controls to ensure alleged abusers
were advised of their rights. In response to a recommendation made in the previous audit, DPPC
created policy and procedure Invest-137. Pursuant to Invest-137, the DPPC added a checkbox to
the 19C Investigation Report Form where the investigator documents that the “DPPC Notice of
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
14
Alleged Abuser’s Rights During DPPC Investigation Interview” was provided to an alleged abuser
and also added a field to its database to track this information. Because there is no statutory or
regulatory requirement that the alleged abuser receive written notification of their rights or that
the alleged abuser signs an acknowledgment form, a verbal advisement of these rights to the
alleged abuser by the investigator is sufficient. . . .
DPPC’s new database is a customized relational database that went live on July 1, 2020, and
DPPC continues to work to tailor the new database to best meet the needs of DPPC operations.
Regarding the notice of alleged abuser rights, DPPC developed a uniform web-based
investigations form which forces the investigator to indicate for each named alleged abuser
whether he/she was advised of rights during a DPPC investigation interview, and if not, a
required field to explain why that did not occur. Therefore, DPPC has already updated its
operations to improve the clarity of the notification of rights process.
Auditor’s Reply
We recognize that DPPC regulations do not require an alleged abuser to sign a Notice of Alleged
Abuser’s Rights. However, DPPC’s investigation policy “Invest-137” states,
2. Prior to commencing an interview of an individual being considered as an [alleged
abuser], the 19C investigator will provide the individual with a copy of the DPPC Notice of
Alleged Abuser’s Rights During DPPC Investigation Interview.
This policy further requires 19C investigators to document that notification of the alleged abuser’s rights
has been completed by checking the associated box on the 19C Investigation Report Form. Our testing in
this area did not involve reviewing 19C files to determine whether there were signed Notices of Alleged
Abuser’s Rights in the sampled files. Rather, we deemed 19C investigators checking the appropriate box
on the 19C Investigation Report Form sufficient to indicate that they had provided the Notice of Alleged
Abuser’s Rights to the alleged abuser. As previously stated, we determined compliance using DPPC’s
investigation policy “Invest-137.” As discussed in a meeting with DPPC, we also accepted an
investigator’s notation in the 19C report of verbal advisement via phone call to be sufficient evidence
that alleged abusers had been advised of their interview rights.
DPPC asserts that it has implemented effective monitoring controls, but as noted above, we found nine
instances where a 19C investigator neither indicated that the alleged abuser had been advised of his/her
interview rights nor provided an explanation of the reason for this for the DPPC Oversight Unit to
review. Had the monitoring controls been effective, the issues discussed in this finding would have been
identified.
Based on its response, DPPC is taking measures to address our concerns on this matter.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
15
2. DPPC does not always complete its investigations within the required
timeframes or document the reasons for not doing so.
Our prior audit revealed that, contrary to state regulations, DPPC did not always complete Initial
Responses (IRs) and 19C reports within established timeframes. Also, for investigations where IRs and/or
19C reports did not meet deadlines, the investigators did not document explanations for the unmet
deadlines in the case files.
During our current audit, we found that 38 (63%) of the 60 IRs we randomly selected for review, out of a
population of 2,925 cases completed and not referred to law enforcement during the audit period, were
completed after the 10 days allowed by regulation. On average, DPPC completed these 60 IRs in 19 days.
When IRs are not completed within required timeframes, DPPC cannot ensure prompt implementation
of remedial action plans to address potential abuse.
Also, DPPC did not consistently complete 19C reports within required timeframes. Specifically, 51 (85%)
of the 60 19C reports for cases we statistically sampled for testing, out of a population of 2,925 cases
completed and not referred to law enforcement during the audit period, were not completed within
required timeframes. Based on the results of our testing, we project—with a 95% confidence level—that
at least 2,152 of the 2,925 completed 19C reports were not completed within the 30 days allowed by
regulation. On average, DPPC completed the 60 19C reports in 70 days.
Finally, 1,874 (56%) of the total 3,291 abuse investigations that were completed after their deadlines,
and were not referred to law enforcement, during the audit period, did not have documented
explanations for the unmet deadlines in the case files. Delays can put victims at risk of further abuse; in
fact, we identified six individuals whose abuse investigations (10 investigations in total) were completed
after their due dates and who experienced further substantiated abuse during their ongoing abuse
investigations.
Authoritative Guidance
The requirements for abuse investigation reports are detailed in 118 CMR 5.02(3)(a):
The . . . “Initial Response” . . . shall be submitted to the Commission by the investigator . . .
within ten calendar days for non-emergency reports of abuse. . . .
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
16
The . . . “Investigation Report” . . . shall be submitted to the Commission by the investigator
within 30 calendar days from the date the report of abuse was referred by the Commission for
investigation.
DPPC’s investigation policy “Invest-109” states that if an investigation cannot be completed within the
30 days allowed by regulation,
The investigator must complete and submit a DPPC Investigation Extension Request Form . . . to
the DPPC oversight manager. . . . This form must provide information that establishes good
cause for the need of an extension.
Reasons for Noncompliance
DPPC had not implemented effective monitoring controls to ensure that investigators completed IRs and
19C reports within the required timeframes or that, when required filing deadlines were not met,
evidence of the reasons for the delay was documented, verified, and retained in case files.
DPPC also stated that approximately 94% of abuse investigations were conducted by the Department of
Developmental Services (DDS), the Department of Mental Health (DMH), or the Massachusetts
Rehabilitation Commission (MRC). DPPC officials told us that DPPC worked closely with these agencies to
resolve investigations and develop protective service plans (PSPs). However, they stated that the
necessity of coordinating and recording information from more than one agency, along with DPPC’s
limited resources, continued to cause delays in meeting deadlines. They also stated that the complexity
and number of investigations had increased during the audit period.
Recommendations
1. DPPC should enhance its policies and procedures by implementing effective monitoring controls to
ensure that investigators complete IRs and 19C reports within the required timeframes and that
when filing deadlines are not met, evidence of the reasons for the delay is documented, verified,
and retained in case files.
2. DPPC should continue to work with DDS, DMH, and MRC to complete IRs for emergency cases within
24 hours, IRs for non-emergency cases within 10 days, and 19C reports within 30 days.
Auditee’s Response
DPPC accepts that it does not always complete its investigations within the required timeframes
but disagrees with the finding that DPPC does not document the reason for doing so. As stated in
the response to the previous audit, DPPC must first emphasize that tardiness of an Initial
Response (“IR”) or Investigation Report (“19C Report”) does not correlate to continued risk to an
alleged victim. The assessment of risk to the victim is the DPPC’s highest priority. It is a process
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
17
which begins at the intake of a report, continues through investigation, and does not conclude
until completion of post-investigation protective services monitoring by our oversight officers. The
DPPC also places the highest priority on emergency situations, and under no circumstances does
the DPPC tolerate delays in risk assessments in situations deemed emergencies.
Staffing and resources within the DPPC and Executive Office of Health and Human Services
referral agencies have long been contributing factors in DPPC’s ability to timely complete
investigations, as have the complexities of investigations and the need to prioritize high risk
investigations. In our effort to secure critically needed funds to support DPPC’s continuously
increasing caseload, DPPC has doggedly and finally successfully promoted its mission to the
Legislature and the Administration resulting in a long needed increase in appropriation in the last
budgetary session. However, with this increase in appropriation came a massive undertaking of
designing and implementing an Abuser Registry. To that end, DPPC has filed bills to amend the
DPPC’s statute as well as noticed for comment proposed amendments to its regulations, which
include a proposed regulatory amendment to extend the investigation timeline from 30 to 45
days—a still aggressive but more realistic goal. Also, DPPC will continue to work with its own
investigations unit as well the investigations units at the Department of Developmental Services
(DDS), Department of Mental Health (DMH) and the Massachusetts Rehabilitation Commission
(MRC), to ensure that investigations are completed as timely as possible.
To more specifically address Finding Two, following recommendations in the previous audit,
DPPC created a new Extension Request process whereby documentation of the reason for an
extension of a timeframe for an IR or 19C Report was required to be provided by the
investigator. Although this process proved ineffective due to the difficulty of relying upon referral
agencies to adhere to a new administrative process, DPPC’s Quality Assurance Unit categorized
and documented the reason for delays based upon available information. As previously
mentioned and in conjunction with the design of a new database, DPPC created web-based
investigations forms and coordinated with its three partner agencies to work on the same report
form in the web portal. These forms are automatically directed to DPPC when submitted by the
investigations manager and as a result the entire process has been digitized and streamlined.
There is a built-in workflow where oversight officers can communicate directly with the
investigators and supervisors.
With its database enhancements, DPPC has increased monitoring of timeframes by focusing on
systemic issues, including ad hoc outreach and notices to all four investigative agencies, following
identification of patterns of overdue IRs or 19C Reports; and assessed and continued with the
process of oversight officers routinely documenting the reason for investigation delays in the
DPPC’s database. Additionally, DPPC updated its Investigations Timeframes policy, Invest-109, in
July 2020, and instituted a periodic case status request process. Throughout the investigative
process, the status and location of an investigation report is tracked in DPPC’s database in a
portal designed specifically for this process where it is monitored by oversight officers.
Most troublesome with regard to the finding related to timeframes is the audit’s assertion that
through a preliminary review, nine victims were identified as believed by the auditor to have
experienced additional abuse during an ongoing investigation. First, as we understand the
government standards of an audit review process pursuant to M.G.L. c. 11, sec 12, findings are
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
18
not based upon preliminary reviews but rather upon an analysis of specific data. DPPC met with
the audit team on November 4, 2020 and discussed what we believe to be the nine cases to
which the audit report refers. DPPC staff reviewed with the audit team each case and determined
that although these alleged victims were involved in simultaneous investigations, in each instance
the specific risk issues were immediately identified and addressed throughout the entire DPPC
process. The statement in the audit report that nine victims were believed by the auditor to have
experienced additional abuse during an ongoing investigation is misleading as it suggests fault on
the part of the DPPC which is absolutely incorrect. The Commission is also charged with ensuring
that protective services are provided to persons with a disability in the least restrictive and most
appropriate manner possible, balancing protection with freedom. Unfortunately, persons with
disabilities are extremely vulnerable to abuse from a variety of sources for a variety of different
reasons. Some may experience a separate incident of abuse, despite the fact that protective
services related to an ongoing investigation were in place—that was the situation in each of these
nine cases as discussed during the November 4, 2020 meeting.
Auditor’s Reply
DPPC contends that it documents the reasons for delays in 19C investigations. However, our testing
determined that this was not the always the case. According to DPPC’s investigation policy “Invest-109,”
the investigator must submit an Investigation Extension Request Form when an investigation cannot be
completed within 30 days. This form must establish good cause for the need of an extension. As stated
previously, approximately half the cases completed during the audit period were submitted late and did
not have Investigation Extension Request Forms. Although DPPC did state that it would no longer use
this form, our testing involved assessing compliance with the requirements that were in place during our
audit period, and we found significant issues in this area.
We do not dispute DPPC’s assertion that “DPPC also places the highest priority on emergency situations,
and under no circumstances does the DPPC tolerate delays in risk assessments in situations deemed
emergencies.” However, during our audit, we asked DPPC officials to show us examples of cases it
deemed emergencies so that we could assess the timeliness of its IRs and the included risk assessments.
According to the data extract DPPC provided to us, of the 3,291 cases completed during the audit
period, there was only 1 with an “Urgency” designation of “Emergency,” and its IR and 19C report were
completed after their due dates.
Regarding the victims of additional abuse, our comment that delays in processing of abuse
investigations could put victims at risk of further abuse is accurate and is not disputed by DPPC. As
previously noted, some individuals did experience additional abuse while previous investigations were
ongoing and overdue.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
19
Based on its response, DPPC is taking measures to address our concerns on this matter.
3. DPPC did not ensure that it consistently received final PSPs from
providers for victims of alleged abuse.
During our audit period, DPPC did not ensure that it consistently received final PSPs from its providers
within 30 days of the completion of abuse investigations. Specifically, 41 of the 60 PSPs tested, from a
population of 640 substantiated abuse investigations completed during the audit period, were not
received within the required timeframes. On average, the 60 PSPs were submitted in 65 days. As a
result, victims might have experienced delays in receiving recommended social services, counseling,
and/or psychiatric services. Plans for victims of criminal abuse were sometimes delayed as well; PSPs
were not submitted on time for 13 (52%) of 25 cases with substantiated abuse that included a criminal
prosecution and guilty disposition result.
Authoritative Guidance
Section II(f) of DPPC’s oversight policy “Oversight-308” states, “Protective service plans not received
within 30 days of the completion of the 19C investigation will be considered overdue.”
Reasons for Late PSPs
DPPC had not implemented effective monitoring controls within its policies and procedures to ensure
that providers submitted PSPs within regulatory timeframes and in compliance with DPPC’s own policies
and procedures. DPPC management told us in an interview that protective action can be, and
sometimes is, taken at the time of intake or IR. Although we acknowledge this, PSPs are required
following the substantiation of abuse in completed abuse investigations.
Recommendation
DPPC should implement effective monitoring controls within its policies and procedures to ensure that
providers submit PSPs within required timeframes.
Auditee’s Response
Although PSPs are required in substantiated cases of abuse, the DPPC is dependent upon the
[protective service, or PS] agencies to file the final plans with the DPPC and as a result, cannot
and does not wait to ensure the safety of alleged victims in the interim. PSPs typically merely
serve to document those services already put in place during the investigation and those services
that are underway but not yet completed. Therefore, the tardiness or absence of receipt of a PSP
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
20
document from the PS agency does not correlate to continued risk to an alleged victim. As
detailed above, the assessment of risk to the victim is the DPPC’s highest priority. Protective
services are put in place as soon as they are determined to be necessary, most often prior to the
receipt of a PSP. Frequently this occurs during investigations. . . .
Furthermore, DPPC disagrees with the audit’s pure speculation that . . . victims might have
experienced delays in receiving recommended social services, counseling and/or psychiatric
services and that plans for victims of criminal abuse were ultimately delayed as well. There exists
no data to support this conclusion. Protecting alleged victims is the heart of what the DPPC does
which is why the specious suggestion that the DPPC is failing in this area suggests a complete
misunderstanding of the operation of the DPPC. However, with recent increase in staffing, DPPC’s
Oversight Unit is now able to be more aggressive in its follow up with the PS agencies with
regard to verification of protective services.
Particularly confusing and misleading is the audit’s statement that PSPs were not submitted on
time for 13 of 25 cases with substantiated abuse that included criminal prosecution and “guilty
disposition result.” As explained during the course of the audit, the [adult protective service, or
APS] and criminal processes are distinctly different. Although joint investigations between APS
and law enforcement are encouraged, each process has its own distinct standards and burdens
of proof. Therefore, it is unclear the correlation being made between protective services rendered
by an APS agency and a guilty disposition in a criminal court.
Auditor’s Reply
We acknowledge that DPPC takes measures to ensure the safety of alleged victims during 19C
investigations. Our concern is that DPPC has not implemented effective monitoring controls to ensure
that providers submit PSPs to it within regulatory timeframes and in compliance with its policies and
procedures.
In its response, DPPC asserts, “PSPs typically merely serve to document those services already put in
place during the investigation and those services that are underway but not yet completed.” However,
this description is not consistent with the description in DPPC’s “Oversight-308” policy:
Upon receipt, the oversight officer will review the protective service plan to determine if the
intended or completed actions adequately protect the victim. It is important to assess not only
the actions, but also the timeframe for completion when determining the adequacy of the plan.
This suggests that services provided, as well as services yet to be provided, need to be approved by
DPPC for adequacy. A delay in a provider’s submission of a PSP delays DPPC’s review and approval of
services, regardless of actions taken during the active investigation. In the opinion of the Office of the
State Auditor, victims could have experienced delays in receiving recommended social services,
counseling, and/or psychiatric services.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
21
We are aware that criminal investigations and DPPC’s civil investigations are different. Our office takes a
risk-based approach to developing audit topics. We chose to look at PSP timeliness for substantiated
abuse cases with criminal activity including assault, battery, and rape. Our intent was to determine
whether PSPs for victims of these types of abuse were submitted on time. We did not state or imply that
services had not been rendered. In fact, results for our testing of compliance with investigation
requirements were positive, as previously noted.
4. DPPC did not always identify and properly document individuals who had
been identified as alleged abusers in multiple reports.
DPPC did not always identify individuals who had been identified as alleged abusers in three or more
reports of suspected abuse and document this information in the corresponding case files. In our testing
of the 3,291 cases DPPC completed during our audit period, we found seven individuals who each had
three or more reports of suspected abuse, but whose corresponding case files did not reflect this
information: DPPC staff members had not documented it in the “Other Pertinent Information” field of
the Intake Abuse Form. All seven cases had been referred by DPPC to other state agencies for
investigation; however, the only information that DPPC provides to the agencies conducting this type of
investigation is a copy of the individual’s Intake Abuse Form. As a result, the agencies conducting
investigations of these seven individuals may not have had all the information necessary to conduct
thorough and effective investigations.
Authoritative Guidance
Section B(3)(I)(7) of DPPC’s intake policy “Intake-204” describes the criteria for assigning abuse reports
for investigations involving alleged abusers with multiple reports:
After two (2) previous complaints have been investigated and a third report is made, the DPPC
should conduct the 19C investigation of the third report.
However, it also indicates that DPPC can refer the matter to other agencies for investigation if it does
not have the resources to conduct the investigation.
DPPC told us that to comply with this policy, it had informal procedures in place requiring staff members
to manually search for alleged abusers by name and determine whether they had prior cases. According
to these informal procedures, if any prior history of abuse is found, it should be noted in the “Other
Pertinent Information” field of the Intake Abuse Form.
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
22
Reasons for Issue
DPPC did not have formal policies and procedures in place that required staff members to identify
alleged abusers who were involved with three or more reports of abuse and document this information
in the “Other Pertinent Information” field of the Intake Abuse Form. In addition, there were no
monitoring controls in place to ensure that DPPC’s informal process was consistently followed.
Recommendation
DPPC should establish formal policies and procedures, and develop monitoring controls, to ensure that
all staff members identify alleged abusers who have been involved with three or more reports of abuse
and document this information in the “Other Pertinent Information” field of the Intake Abuse Form.
Auditee’s Response
The DPPC would first like to clarify the process. The screening process is typically a multi-layered
review with each reviewer responsible for identifying risk, jurisdictional criteria, and checking the
database for any relevant history for the alleged abuser or alleged victim. The Intake Unit strives
to be cautious in documenting only relevant information on an Intake Form so as not to bias an
investigation. This process relies heavily on judgment of the screener, with a safety net being
that investigators, oversight officers, and sometimes investigative and oversight supervisors will
also review any related cases as part of the investigatory process. Based upon confidentiality
considerations and integrity of the investigatory process, DPPC has made a conscious decision to
rely upon training and judgment of its screeners. To that end, DPPC appreciates the audit’s
recommendation and will assess in what way we might more effectively ensure that this process
is deeply embedded in the job function.
To the extent that the audit suggests that the DPPC is not adhering to its own policy of assigning
a DPPC investigator to cases where two previous complaints against an alleged abuser have been
investigated, the clear caveat in the policy is that this is resource dependent. As explained to the
audit teams and simply stated here, inadequate resources were the reason for not assigning such
cases to the DPPC. . . .
However, as explained above, the DPPC’s new relational database has enhanced features such as
unique People Records which house all data elements and related information about individuals
involved in DPPC investigations. This feature will assist screeners and oversight officers in
conducting historical review of information. DPPC reserves the right to determine what
information to include on an Intake Form to best meet its mission requirements of protecting
persons with disabilities from abuse and neglect [through] objective . . . and unbiased
investigations.
Finally, [the DPPC executive director] understand[s] and appreciate[s] the value and importance
of an external review process regarding State agency operations and will work diligently to
Audit No. 2020-0046-3S Disabled Persons Protection Commission
Detailed Audit Findings with Auditee’s Response
23
continue to enhance the timeliness and efficiency of the DPPC investigation process as we strive
to protect persons with disabilities throughout the Commonwealth.
Auditor’s Reply
Our testing of individuals with two previous complaints was based on DPPC’s intake policy “Intake-204,”
which states,
After two (2) previous complaints have been investigated and the third report is made, the DPPC
should conduct the 19C investigation of the third report.
We are aware that case assignment depends on DPPC’s available resources and noted this in the
“Authoritative Guidance” section of the finding. DPPC states that reviewers check its database for any
relevant history. However, we determined that alleged abusers’ histories of previous complaints were
not mentioned on Intake Abuse Forms. Without documentation substantiating that a search of an
alleged abuser’s history was performed, it is unclear whether DPPC personnel were aware of the
previous abuse investigations. Therefore, we could not determine whether abuse history was
considered when cases were referred to external agencies instead of DPPC’s own Investigation Unit.
When we discussed the matter with DPPC management, they indicated that DPPC did not feel it was
important to require its intake staff to note that a history search had been performed. We respectfully
disagree, since there is no guarantee that the agency to which DPPC has assigned an investigation will
properly identify an alleged abuser’s previous substantiated abuse investigations.
We acknowledge that DPPC is taking measures to address our concerns on this matter

Official Audit Report – Issued June 30, 2025
Disabled Persons Protection Commission
For the period July 1, 2021 through June 30, 2023
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
June 30, 2025
Nancy Alterio, Executive Director
Disabled Persons Protection Commission
300 Granite Street, Suite 404
Braintree, MA 02184
Dear Executive Director Alterio:
I am pleased to provide to you the results of the enclosed performance audit of the Disabled Persons
Protection Commission. This report details the audit objectives, scope, methodology, and other matters,
for the audit period, July 1, 2021 through June 30, 2023. As you know, my audit team discussed the
contents of this report with agency managers. This report reflects those comments.
I appreciate you and all your efforts at the Disabled Persons Protection Commission. The cooperation
and assistance provided to my staff during the audit went a long way toward a smooth process. Thank
you for encouraging and making available your team. I am available to discuss this audit if you or your
team has any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 7
OTHER MATTERS ................................................................................................................................................. 14
Audit No. 2024-0046-3S Disabled Persons Protection Commission
List of Abbreviations
ii
LIST OF ABBREVIATIONS
APS Adult Protective Services
CMR Code of Massachusetts Regulations
CMS case management system
DDS Department of Developmental Services
DMH Department of Mental Health
DPPC Disabled Persons Protection Commission
IR Initial Response
MRC Massachusetts Rehabilitation Commission
PSP Protective Service Plan
RI Retaliation Investigation
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Disabled Persons Protection Commission (DPPC) for
the period July 1, 2021 through June 30, 2023.
The purpose of our audit was to determine the following:
• Regarding Initial Responses (IRs), Investigation Reports, and Retaliation Investigation (RI) Reports,
did DPPC either (1) ensure that these documents were filed by an Adult Protective Services (APS)
investigator in a timely manner or (2) record the reasons as to why these documents were not
filed in a timely manner, in accordance with Sections 5.02(4)(a)(1)(a), 5.02(4)(a)(2)(a), and 5.03(2)
of Title 118 of the Code of Massachusetts Regulations (CMR); Section II(E) of DPPC’s Retaliation
Investigations Procedure; and Section III(E) of DPPC’s Investigation Assignment, Monitoring and
Timeframes Procedure?
• To what extent did DPPC monitor the delivery of Protective Service Plans (PSPs), as required by
118 CMR 7.03(3), 7.05(1), and 7.05(2)?
• To what extent did DPPC maintain the Registry of Abusive Care Providers to ensure that entries
were accurate, current, and in compliance with Sections 15(b) and 15(f) of Chapter 19C of the
General Laws?
Our audit revealed no significant issues that must be reported under generally accepted government
auditing standards. However, during the course of our audit, we identified issues with the filing of
investigative reports. See Other Matters.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Disabled Persons Protection Commission (DPPC) was established in 1987 by Chapter 19C of the
Massachusetts General Laws. According to DPPC’s website, DPPC is “an independent state agency
responsible for investigation and remediation of instances of abuse committed against persons with
disabilities in the Commonwealth.” DPPC goes on to talk about its enabling statute on its website:
Pursuant to its enabling statute [Chapter 19C of the General Laws], the jurisdiction of DPPC includes
adults with disabilities between the ages of 18 and 59, who are within the Commonwealth whether
in state care or in a private setting and who suffer serious physical and/or emotional injury through
the act and/or omission of their caregivers. The DPPC enabling statute fills the gap between the
Department of Children and Families (DCF) (through the age of 17) and the Executive Office of
Elder Affairs (EOEA) (age 60 and over) statutes.
DPPC oversees investigations conducted on its behalf by the Department of Developmental Services (DDS),
the Department of Mental Health (DMH), and the Massachusetts Rehabilitation Commission (MRC).1 DPPC
also performs its own investigations to accomplish its mission “to protect adults with disabilities from the
abusive acts or omissions of their caregivers through investigation oversight, public awareness and
prevention.” The agency received 13,310 and 16,043 abuse reports requiring investigation in fiscal years
2022 and 2023, respectively.
DPPC consists of two commissioners and a chair, each of whom is appointed by the Governor. Reporting
directly to the commissioners and the chair is the executive director, who oversees DPPC’s daily
operations.
The commissioners and the chair must report directly to both the Governor and the Legislature by
submitting audit summary reports to them annually. These annual reports include details about the
actions DPPC has taken; the names, salaries, and duties of all employees; the funds disbursed; and any
other pertinent matters relevant to DPPC’s jurisdiction.
DPPC is located at 300 Granite Street in Braintree. During the audit period, DPPC had 132 employees,
which included, but was not limited to, directors, managers, intake specialists, oversight officers, and
Adult Protective Services (APS) investigators. DPPC had state appropriations of $9.74 million and $11.70
million in fiscal years 2022 and 2023, respectively.
1. MRC changed its name to MassAbilty after the audit period.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
3
Abuse Reporting Procedures
According to DPPC’s chief of quality assurance and audit officer, DPPC operates a 24-hour hotline that
allows people to report alleged abuse involving adults with disabilities. The majority of abuse reports that
DPPC receives are submitted through this hotline. DPPC also provides the option to file reports by email,
fax, or in-person means. Personnel members working on the hotline are responsible for receiving,
documenting, and evaluating information about the alleged victim, the alleged abuser, and the nature of
the incident.
When personnel members2 begin filing an abuse report in the Case Management System (CMS), the
system automatically generates a report number and records the date and time. The risk of harm to an
alleged victim is evaluated by the personnel member filing the report in the CMS, and based on this risk
determination, the report is classified as either an emergency or nonemergency. After the personnel
member makes the risk determination, they generate an intake number and the abuse report is assigned
to both a DPPC oversight officer and an APS investigator from either DPPC, DDS, DMH, or MRC for further
investigation. The CMS automatically calculates report submission deadlines based on the report’s
screening date3 and emergency status. In addition, a member of the State Police Detective Unit reviews
every abuse report to determine whether there is any indication of criminal activity in the allegation. If
the State Police Detective Unit detects any such activity, then it reports this activity to the appropriate
district attorney’s office for review.
Evaluation and Investigation Reports
According to DPPC’s chief of quality assurance and audit officer, if an abuse report filed in the CMS
requires investigation, the assigned APS investigator conducts an investigation and then must file an
Evaluation and Investigation Report. This report is composed of two parts: the first part is known as the
Initial Response (IR) and the second part is known as the Investigation Report.
IRs
APS investigators use IRs to document preliminary facts that they gather from interviews with the
alleged victim and/or the reporter and from visits to the site of alleged abuse. During the course of
2. For the purposes of this audit report, we use the term personnel members to refer to both DPPC employees and contracted
vendors trained by DPPC to operate the hotline after business hours (unless stated otherwise).
3. The screening date is the date a report of abuse was screened in by DPPC and assigned to an APS investigator.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
4
their investigations, the assigned APS investigator assesses further risks and documents both
protective service recommendations and any further actions needed (if any). APS investigators must
file IRs to DPPC’s Oversight Unit within the following timeframes after the abuse report screening
date: 10 calendar days for nonemergency cases or 24 hours for emergency cases.
According to DPPC’s chief of quality assurance and audit officer, during the audit period, DPPC filed a
total of 4,721 nonemergency IRs. There were zero IRs that needed to be filed as emergency cases.
Investigation Reports
The second part of the Evaluation and Investigation Report is known as the Investigation Report. This
second part contains information that the assigned APS investigator gathers as evidence from
additional interviews, additional documentation corroborating alleged abuse, and additional site
visits, when necessary, to determine whether the alleged abuse occurred. The occurrence of alleged
abuse is considered substantiated by an APS investigator when the evidence collected is sufficient to
confirm that either an act or neglect by the alleged abuser resulted in serious physical or emotional
injury to the alleged victim. Conversely, alleged abuse is deemed unsubstantiated by an APS
investigator when it cannot be established by the preponderance of the evidence. Investigation
Reports contain findings and appropriate protective service recommendations from the case’s APS
investigator. APS investigators must file an Investigation Report with DPPC within 45 calendar days
from the abuse report’s screening date. After the case’s APS investigator files the Investigation Report,
a DPPC oversight supervisor determines whether the Investigation Report is complete and can be
closed out in the system, warranting no further investigation.
According to DPPC’s chief of quality assurance and audit officer during the audit period, DPPC filed a
total of 2,645 Investigation Reports.
Both IRs and Investigation Reports, when fully completed, are designed to ensure that each abuse
investigation meets the minimum requirements of Section 5.02(1) of Title 118 of the Code of
Massachusetts Regulations.
Retaliation Investigation Reports
According to DPPC’s chief of quality assurance and audit officer, an APS investigator examines claims of
retaliation against people who file alleged abuse reports or participate in any DPPC investigations. At the
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
5
completion of an investigation, the involved APS investigator files a Retaliation Investigation (RI) Report
in DPPC’s CMS. Each RI Report contains relevant information that was obtained during an investigation,
such as evidence from interviews with the alleged retaliation victim, alleged retaliator, and witnesses of
the alleged retaliation. Documenting this information ensures that the involved APS investigator reviewed
all evidence relevant to the investigation and helps substantiate any claims of alleged retaliation. The
investigation and the initial RI Report draft must be filed with DPPC’s director of investigations by the
involved APS investigator within 60 business days from its screening date.
According to DPPC’s chief of quality assurance and audit officer during the audit period, DPPC filed a total
of 10 RI Reports.
Protective Service Plans
As part of the investigation process, DDS, DMH, or MRC may recommend a Protective Service Plan (PSP)
when alleged abuse is substantiated through their investigations. The PSP outlines protective service
recommendations, identifies individuals responsible for implementing those recommendations, states
the proposed or actual start date of the recommendation implementation, and includes any additional
information needed for DPPC to effectively monitor the implementation of recommendations.
According to DPPC’s chief of quality assurance and audit officer, if an APS investigator recommends a PSP,
then the plan must be filed with DPPC no later than 30 calendar days from the date that the corresponding
Investigation Report was filed with DPPC. The PSP can be filed through a variety of methods, such as
including it as a note in the corresponding IR or Investigation Reports, sending it as an email or physical
letter, or making it in a telephone call.
According to DPPC’s chief of quality assurance and audit officer, during the audit period, there were 341
filed PSPs that corresponded to substantiated reports of abuse.
Monitoring Policy
During our prior audit (Audit No. 2020-0046-3S), we found that DPPC did not always complete its
investigations within the required timeframes. In response to one of our recommendations from that
audit, DPPC implemented corrective actions. DPPC enhanced its monitoring policy that requires it to run
monthly monitoring reports and send monthly notices to external protective service agencies that alert
them of outstanding requirements. The policy also requires that, when a PSP is needed, it is documented
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Overview of Audited Entity
6
in the IR part of the Evaluation and Investigation Report. In addition, DPPC also documents in each IR and
Investigation Report the reasons for any filing delays.
Registry of Abusive Care Providers
DPPC established its Registry of Abusive Care Providers on July 31, 2021. According to Section 2(b) of
Chapter 19 of the Acts of 2020, DPPC is to “establish and maintain a registry of care providers against
whom the commission has made a substantiated finding of registrable abuse.” According to DPPC’s
website,
The DPPC Abuser Registry . . . is intended to protect individuals with intellectual or developmental
disabilities (“I/DD”) by barring care providers who have a substantiated finding of registrable abuse
from working with other persons with intellectual or developmental disabilities. . . .
The DPPC Abuser Registry is not a public registry. This information is not considered a “public
record” for purposes of [Chapter 66 of the General Laws] and all information regarding care
providers listed on the DPPC Abuser Registry is confidential.
No later than October 31, DPPC must annually submit reports that update and summarize the contents of
the registry to the clerks of the House of Representatives and the Senate, the House and Senate
Committees on Ways and Means, and the Joint Committee on Children, Families and Persons with
Disabilities. For more information on what details these reports must include, see the list of audit
summary report requirements here.
According to DPPC’s chief of quality assurance and audit officer, during the audit period, DPPC added 69
individuals to the Registry of Abusive Care Providers.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
7
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Disabled Persons Protection
Commission (DPPC) for the period July 1, 2021 through June 30, 2023.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Regarding Initial Responses (IRs), Investigation Reports, and Retaliation Investigation
(RI) Reports, did DPPC either (1) ensure that these documents were filed by an Adult
Protective Services (APS) investigator in a timely manner or (2) record the reasons as
to why these documents were not filed in a timely manner, in accordance with
Sections 5.02(4)(a)(1)(a), 5.02(4)(a)(2)(a), and 5.03(2) of Title 118 of the Code of
Massachusetts Regulations (CMR); Section II(E) of DPPC’s Retaliation Investigations
Procedure; and Section III(E) of DPPC’s Investigation Assignment, Monitoring and
Timeframes Procedure?
Yes; see Other
Matters
2. To what extent did DPPC monitor the delivery of Protective Service Plans (PSPs), as
required by 118 CMR 7.03(3), 7.05(1), and 7.05(2)?
To a sufficient
extent
3. To what extent did DPPC maintain the Registry of Abusive Care Providers to ensure
that entries were accurate, current, and in compliance with Sections 15(b) and 15(f)
of Chapter 19C of the General Laws?
To a sufficient
extent
To accomplish our audit objectives, we gained an understanding of DPPC’s internal control environment
relevant to our objectives by reviewing applicable policies and procedures and by interviewing DPPC
management. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we
performed the procedures described below.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
8
IRs, Investigation Reports, and RI Reports
Regarding IRs, Investigation Reports, and RI Reports, to determine whether DPPC either (1) ensured that
these documents were filed by an APS investigator in a timely manner or (2) recorded the reasons as to
why these documents were not filed in a timely manner, in accordance with 118 CMR 5.02(4)(a)(1)(a),
5.02(4)(a)(2)(a), and 5.03(2); Section II(E) of DPPC’s Retaliation Investigations Procedure; and Section III(E)
of DPPC’s Investigation Assignment, Monitoring and Timeframes Procedure, we took the actions
described below.
IRs
We selected a random, statistical sample4 of 60 IRs from the population of 4,721 IRs that external
protective service agencies filed with DPPC during the audit period, using a 95% confidence level, 5 a
0% expected error rate,6 and a 5% tolerable error rate.7 By comparing each IR’s screening date to the
date that the APS investigator filed it with DPPC, we determined whether each IR was filed with DPPC
within 10 calendar days of the IR’s screening date.8
We met with DPPC officials and observed them retrieving IRs from the case management system
(CMS) that we initially calculated as being late. We took screenshots9 of each potentially late IR, along
with any documentation that supported the delayed submission. For each IR confirmed as being late,
we determined whether it was accurately listed on DPPC’s monthly monitoring reports and whether
notices were sent to the protective services agency, notifying it of the late IR.
4. Auditors use statistical sampling to select items for audit testing when a population is large (usually over 1,000) and contains
similar items. Auditors generally use a statistical software program to choose a random sample when statistical sampling is
used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make
conclusions or projections about entire populations.
5. Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are
representative of the population (parameter), expressed as a percentage. A 95% confidence level means that 95 out of 100
times, the statistics accurately represent the larger population.
6. Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the
auditor’s knowledge of factors such as prior audit results, the understanding of controls gained in planning, or a probe sample.
7. The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while
still using the sample to conclude that the results from the sample have achieved the objective.
8. Note that, because no emergency cases were filed during the audit period, the population of IRs only consisted of
nonemergency cases. Thus, we only tested that all IRs were filed within the timeframe for nonemergency cases.
9. To preserve confidentiality during the inspection of documents, we used software to redact any personally identifiable
information.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
9
Investigation Reports
We selected a random, statistical sample of 60 Investigation Reports from the population of 2,645
Investigation Reports that APS investigators filed with DPPC during the audit period, using a 95%
confidence level, a 0% expected error rate, and a 5% tolerable error rate. By comparing each
Investigation Report’s screening date to the date that the APS investigator filed the Investigation
Report with DPPC, we determined whether each Investigation Report was filed with DPPC within 45
calendar days of the Investigation Report’s screening date.
We met with DPPC officials and observed them retrieving Investigation Reports that we initially
calculated as being late from the CMS. We took screenshots10 of each potentially late Investigation
Report, along with any documentation that supported the delayed submission. For each Investigation
Report confirmed as being late, we determined whether it was accurately listed on DPPC’s monthly
monitoring reports and whether notices were sent to the protective services agency, notifying it of
the late Investigation Report.
We also determined whether any alleged abusers reported in each Investigation Report were
accurately listed in DPPC’s Registry of Abusive Care Providers.
RI Reports
We tested the entire population of 10 RI Reports that DPPC APS investigators filed with DPPC’s
director of investigations during the audit period. Using the full population, we determined whether
each RI Report was filed by the APS investigator with the director of investigations within 60 business
days of the RI Report’s screening date.
We met with DPPC officials and observed them retrieving RI Reports that we initially calculated as
being late from the CMS. We took screenshots11 of each RI Report, along with any documentation
that supported the delayed submission. For the one RI Report confirmed as being late, we reviewed
DPPC’s Investigation Planning Form and Investigation Case Activity Log.12 We reviewed these
10. See Footnote 9 for more information.
11. See Footnote 9 for more information.
12. Both the Investigation Planning Form and Investigation Case Activity Log document the case number, the date that the APS
investigator was assigned to the case, the date that the investigation manager was assigned to the case, the APS investigator’s
initials (indicating who is performing the investigation), the investigation manager’s initials (indicating who is providing
managerial oversight of the investigation), a description of the alleged retaliation incident, all investigation steps taken and
the dates they were taken, and the results of the investigation.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
10
documents to determine whether the DPPC APS investigator documented each step of the
investigation by updating the Investigation Case Activity Log and by providing investigation
information to their investigation manager during the investigation process.
See Other Matters for additional information.
PSPs
To determine to what extent DPPC monitored the delivery of PSPs, as required by 118 CMR 7.03(3),
7.05(1), and 7.05(2), we took the following actions. First, we selected a random, nonstatistical sample 13
of 10 months out of the population of the 24 months in the audit period. We obtained the monthly
monitoring report corresponding to each month in our sample. We then reviewed each monthly
monitoring report to determine whether DPPC provided those monthly monitoring reports to the
appropriate protective services agency, notifying them of any PSPs that were late.
In addition, we selected a random, nonstatistical sample of 40 Investigation Reports from the population
of 341 Investigation Reports that had substantiated cases of abuse for which a PSP should have been
recommended by an APS investigator. We inspected each Investigation Report in the CMS and noted any
documentation that showed the dates that the PSP recommendations were made. We compared the
dates that an APS investigator filed each Investigation Report with DPPC to the dates that the PSPs were
recommended (as recorded in DPPC’s CMS). Using this sample, we determined whether each PSP was
recommended within 30 calendar days of the date of DPPC’s acceptance of the Investigation Report. We
also determined whether alleged abusers listed in our sample were accurately placed on DPPC’s Registry
of Abusive Care Providers.
For this objective, we found no significant issues during our testing. Therefore, we concluded that, based
on our testing, DPPC met the relevant criteria regarding monitoring the delivery of PSPs.
Registry of Abusive Care Providers
To determine to what extent DPPC maintained the Registry of Abusive Care Providers to ensure that
entries were accurate, current, and in compliance with Sections 15(b) and 15(f) of Chapter 19C of the
General Laws, we took the following actions. First, we obtained all two audit summary reports for the
13. Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items
are not similar enough, or there are specific items in the population that the auditors want to review.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
11
audit period. We inspected each audit summary report to determine whether it was submitted no later
than October 31 of the corresponding year to the clerks of the House of Representatives and the Senate,
the House and Senate Committees on Ways and Means, and the Joint Committee on Children, Families
and Persons with Disabilities. We reviewed each audit summary report and determined whether each
report contained the required information listed in Section 15(f) of Chapter 19C of the General Laws,
which is as follows:
(i) the number of substantiated findings of abuse found or not found to have been
registrable;
(ii) the number of people on the registry;
(iii) the number of people who were added to the registry in the last fiscal year;
(iv) the number of substantiated findings of registrable abuse that were appealed in the last
fiscal year;
(v) the number of substantiated findings of registrable abuse that were overturned on appeal
in the last fiscal year;
(vi) the number of requests made by employers for information from the registry and the
number of such requests that were granted in the last fiscal year;
(vii) the total number of instances in the last fiscal year in which the commission failed to
notify the department or the last known employer of a care provider who was placed on
the registry and the reasons for such failures; and
(viii) the number of employers found to have failed to meet the requirements of subsection
(d) [of Section 15 of Chapter 19C of the General Laws] in the last fiscal year.
We then reconciled the number of people added to the Registry of Abusive Care Providers for each fiscal
year during the audit period to the number of people reported as newly added in the corresponding audit
summary report to determine whether the information in the report was accurate.
Additionally, we selected a random, nonstatistical sample of 20 names out of the population of 69 names
that were on the Registry of Abusive Care Providers list. We traced each of the 20 names in our sample to
their corresponding Investigation Reports in DPPC’s CMS and determined whether those names were
substantiated abusers, which warranted their names being listed on the Registry of Abusive Care
Providers. Further, we determined whether the Registry of Abusive Care Providers was current by
comparing it to the samples we selected during our testing of Objectives 1 and 2, which were indicative
of substantiated abuse cases.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
12
For this objective, we found no significant issues during our testing. Therefore, we concluded that, based
on our testing, DPPC met the relevant criteria regarding maintaining the Registry of Abusive Care
Providers.
We used a combination of statistical and nonstatistical sampling methods for testing, and we did not
project the results of our testing to the populations corresponding to IRs, Investigation Reports, RI
Reports, PSPs, or the Registry of Abusive Care Providers.
Data Reliability Assessment
To determine the reliability of the data from DPPC’s computer network system, we reviewed a System
and Organization Controls 2 Type 2 Report,14 and mapped it to the respective areas such as: DPPC’s
security management, configuration management, segregation of duties, contingency planning, and
access controls. We tested all 68 employees hired during the audit period to determine whether they had
completed initial cybersecurity training. We also selected a nonstatistical, random sample of 8 employees
out of 142 who had computer network access during the audit period and tested whether these 8
employees completed annual cybersecurity training, whether background checks were performed at their
time of hire, and whether their computer user access rights matched their titles and positions. We also
determined whether employees who were terminated during the audit period had their computer
network access removed. We also interviewed DPPC management who were knowledgeable about the
data. We observed DPPC officials query and extract the following information from the CMS:
• a total of 4,805 IRs (of which 4,721 were from within the audit period and thus constituted our
testing population;15
• a total of 3,012 Investigation Reports (of which 2,645 were from within the audit period and thus
constituted our testing population;16
• a total of 10 RI Reports;
• a total of 341 investigations with substantiated cases of abuse committed during the audit period
(and which constituted our testing population);
14. A System and Organization Control report is a report on controls about a service organization’s systems relevant to security,
availability, processing integrity, confidentiality, or privacy issued by an independent contractor.
15. The original number of 4,805 extracted records was due to limitations with DPPC’s CMS, which resulted in extracting records
that were outside the audit period. Our date range test (referenced later in this section) helped in narrowing down the records
to only ones relevant to our audit.
16. See Footnote 15 for more information.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Audit Objectives, Scope, and Methodology
13
• a total of 35217 CMS users (of whom 142 were specifically DPPC employees active during the audit
period and thus constituted our testing population); and
• a total of 69 registered abusers who were added to the Registry of Abusive Care Providers during
the audit period.
The chief of quality assurance and audit officer then provided these records to us in Microsoft Excel
spreadsheets. We ensured that the number of CMS records we observed for each document type listed
above matched the corresponding number of records in the Excel spreadsheets. We performed a date
range test to ensure that our testing only involved records from within the audit period. We tested the
data to ensure that it did not contain any spreadsheet issues (e.g., hidden objects such as names, rows,
columns, or workbooks; duplicate records; or missing values in necessary data fields).
Specifically for testing the CMS users list, we also traced all 142 DPPC CMS users from our population to
the Human Resources Compensation Management System, which is the Commonwealth’s official payroll
system, to ensure that each CMS user in our population was an active DPPC employee during the audit
period.
Based on the results of the data reliability assessment procedures described above, we determined that
the information we obtained was sufficiently reliable for the purposes of our audit.
17. Of the 352 CMS users, 210 users were non-DPPC employees, such as contracted vendors and other state employees from the
Department of Developmental Services, the Department of Mental Health, the Massachusetts Rehabilitation Commission,
and the State Police.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
14
OTHER MATTERS
Filing of Investigation Reports
The Disabled Persons Protection Commission (DPPC) made progress in applying the recommendations
from our previous audit (Audit No. 2020-0046-3S) for Finding 2, which stated, “DPPC does not always
complete its investigations within the required timeframes or document the reasons for not doing so.”
Specifically, since our last audit, DPPC enhanced its monitoring policy that requires it to run monthly
reports and send notices to external protective service agencies to alert them of outstanding
requirements. Additionally, DPPC now requires that all protective service plans and reasons for
investigation delays be documented within the case notes. Our previous audit report disclosed that 56%
of late abuse investigations during the prior audit period did not have any documented reasons for delays,
compared to our current audit, which revealed that all late Initial Responses (IRs), Investigation Reports,
and Retaliation Investigation (RI) Reports had documented reasons for delays. We commend DPPC for
making progress addressing findings from our previous audit; however, there is room for improvement in
order for DPPC to achieve full compliance with Section 5.02(4)(a) of Title 118 of the Code of Massachusetts
Regulations (CMR). Specifically, despite DPPC appropriately documenting the reasons for delays related
to the filing of Investigation Reports, we feel that enhanced collaboration with the Department of
Developmental Services, the Department of Mental Health, etc., along with enhanced policies,
procedures, and monitoring, could result in more timely submission of investigation reports in the first
place. The following paragraphs outline detailed results from our audit.
IRs
We inspected 60 IRs (all of which were for nonemergency cases) and found that 21 were not filed with
DPPC by external Adult Protective Services (APS) investigators within the required timeframe of 10
calendar days from the screening date for nonemergency cases. The delays ranged from 1 to 65 days
after the due date, with an average of 10 days late. The following are the details of our results:18
• There were 9 instances where the corresponding external protective services agency was
understaffed and had a large caseload, according to both case notes and explanations that
DPPC’s chief of quality assurance and audit officer provided to us.
18. Some of the 21 IRs in question had more than one instance of delay. Because of this, in total, there were 27 instances of
delays.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
15
• There were 3 instances where the corresponding external protective services agency received
multiple reports regarding the same alleged victim, which made repeated interviews with the
alleged victim, witnesses, and alleged abuser necessary.
• There were 4 instances that involved reports of alleged sexual assault, which required
additional investigation time.
• There were 4 instances that involved police investigations, which required additional
investigation time.
• There were 7 instances where the protective services agency had difficulty scheduling
interviews with the alleged victims and/or abusers. The following is a breakdown of these 7
instances:
• There were 3 instances where the protective services agency received multiple reports
for one alleged victim, requiring repeated interviews with the alleged victim, witnesses,
and the alleged abuser.
• There were 3 instances where the interviewee was hospitalized and unable to be
interviewed.
• There was 1 instance where the interviewee was initially unable to be found by an APS
investigator due to being incarcerated.
Investigation Reports
We inspected 60 Investigation Reports and found that 31 were not filed with DPPC by the APS
investigator within the required timeframe of 45 calendar days from the screening date. Of the 31
late Investigation Reports, 23 were filed with DPPC by an external APS investigator, and the remaining
8 by an internal DPPC APS investigator. The delays ranged from 1 to 166 days after the due date, with
an average of 46 days late. The following are details regarding our results:19
• There were 16 instances where the corresponding external protective services agency was
understaffed and had a large caseload, according to both case notes and explanations that
DPPC’s chief of quality assurance and audit officer provided to us.
• There was 1 instance that involved reports of alleged sexual assault, which required additional
investigation time.
• There were 3 instances where the external protective services agency required multiple
report revisions that caused the late submission.
19. Some of the 31 Investigation Reports in question had more than one instance of delay. Because of this, in total, there were
34 instances of delays.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
16
• There were 4 instances that were deemed substantiated abuse cases, requiring additional
investigatory steps.
• There were 5 instances where the protective services agency encountered difficulties
accessing records involving alleged victims and abusers, such as obtaining private video
recordings of abuse incidents.
• There were 5 instances where the APS investigators had difficulty scheduling interviews with
the alleged victims and/or abusers. The following is a breakdown of these 5 instances:
• There were 2 instances where an APS investigator encountered a delay in (1) obtaining
victims’ health records from the hospital and (2) scheduling interviews with victims’
doctors.
• There was 1 instance where an alleged abuser’s attorney refused interview access to their
client.
• There was 1 instance that required interviewing multiple witnesses, but scheduling some
of the witness interviews was unsuccessful.
• There was 1 instance where there was an undocumented scheduling delay.
RI Reports
We inspected 10 RI Reports and found that 1 was not filed with DPPC’s director of investigations by
an APS investigator within the required timeframe of 60 business days from the screening date. This
RI Report was late by 14 business days.
Authoritative Guidance
The submission timeframe requirements for abuse investigation reports are detailed in 118 CMR
5.02(4)(a), which states,
• Requirements of Initial Response.
a. . . . The “Initial Response” . . . shall be submitted to the Commission by the investigator
within . . . ten calendar days for nonemergency reports of abuse. . . .
• Requirements of Investigation Report.
a. . . . The “Investigation Report” . . . shall be submitted to the Commission by the
investigator within 45 calendar days from the date the report of abuse was referred by the
Commission for investigation.
The submission timeframe requirement for RI Reports is detailed in 118 CMR 5.03(2), which states,
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
17
The Investigation Report of the investigation . . . shall be submitted to the Commission’s Director
of Investigations within 60 business days from the date on which the allegation of retaliation was
assigned for investigation.
The documentation requirements for abuse Investigation Reports and RI Reports are detailed in
Section II(E) of DPPC’s Retaliation Investigations Procedure and Section III(E) of DPPC’s Investigation
Assignment, Monitoring and Timeframes Procedure.
Section II(E) of DPPC’s Retaliation Investigations Procedure states, “All steps and actions . . . shall be
documented in the Retaliation Investigation Report section of the database.”
Section III(E) of DPPC’s Investigation Assignment, Monitoring and Timeframes Procedure states,
Throughout the submission and approval process . . . the status and location of an IR or 19C
Investigation Report will be tracked in the [case management system (CMS)]. The Oversight Officer
or designee will send status requests on IR or 19C Investigation Reports that have not met the
established timeframes to the Investigator and Investigations Manager who must respond to the
request . . . with the . . . reason for delay. . . . The status of each Oversight Officer’s assigned
cases will be reviewed weekly and updated as needed in the CMS.
Reasons for Late Reports
IRs and Investigation Reports
DPPC stated that IRs for nonemergency cases and Investigation Reports can be delayed for various
reasons, many of which are beyond the control of DPPC or the protective services agency. These
reasons include potential difficulties with the following:
• locating and/or interviewing witnesses, alleged victims, and/or alleged abusers;
• accessing critical third-party records from medical examiners or facilities such as hospitals;
• coordinating with law enforcement or other agencies conducting parallel investigations;
• delays in receiving evidence such as DNA and sexual assault results; and
• prioritizing risky cases while balancing high caseloads and investigatory staffing shortages at
protective service agencies.
DPPC also stated that its regulations permit adjustments to timelines because, in certain instances,
delays are inevitable and even necessary to ensure thorough investigations.
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
18
RI Reports
DPPC stated that the one RI Report delay occurred because the alleged retaliation victim did not
identify a specific alleged retaliator, which required additional time for an in-depth investigation. This
investigation ultimately indicated that the claims were unsubstantiated.
Recommendation
DPPC should continue to work with the Department of Developmental Services, the Department of
Mental Health, the Massachusetts Rehabilitation Commission, and its internal APS investigators to meet
required timeframes by filing IRs for nonemergency cases within 10 calendar days, Investigation Reports
within 45 calendar days, and RI Reports within 60 business days.
Auditee’s Response
The DPPC has processes in place to ensure swift action on potential abuse, independent of timelines
for submitting initial responses or investigation reports. As observed and documented by the audit
team throughout our several sessions during the audit and the audit team’s review of numerous
agency Policies & Procedures, DPPC prioritizes immediate safety and risk mitigation through:
• Immediate Safety Measures: Protocols for swift actions upon identifying potential
abuse (e.g., removal of alleged abuser, alternative placement, or care). These occur before
formal reports are issued.
• Ongoing Risk Assessment and Mitigation: Continuous risk assessment occurs from
initial hotline contact throughout issuance of the formal report. Immediate remedial
protective service measures are implemented if risks are identified, regardless of report
status.
• Interim Actions: DPPC and partner agencies implement interim remedies during ongoing
investigations (e.g., changes in supervision or training based on preliminary findings).
The DPPC’s ability to act promptly is not dependent on the submission of documentation associated
with the investigation. Our existing protocols prioritize individual safety and wellbeing, ensuring
prompt action when necessary. However, the DPPC agrees with the Auditor’s Recommendation to
continue to work with existing partners to ensure adherence to timelines to the full extent possible.
DPPC is committed to improving efficiency and will take the following steps:
• Enhanced Collaboration: Streamlining information sharing and coordination with [the
Department of Developmental Services], [Department of Mental Health], and MassAbility
through system enhancements, joint meetings, and improved notifications.
• Internal Process Review : Identifying and addressing bottlenecks in investigation
workflows (case assignment, evidence gathering, notifications, report review).
Audit No. 2024-0046-3S Disabled Persons Protection Commission
Other Matters
19
• Resource Allocation Assessment: Ensuring adequate staffing and support for effective
caseload management.
• Improved Tracking and Monitoring: Implementing enhanced mechanisms (e.g.,
updates to our [case management system], progress reports) for proactive delay
identification and intervention.
• Reinforced Training and Accountability: Emphasizing adherence to timelines and
protocols for all investigators.
Auditor’s Reply
We commend DPPC for taking steps to continue to improve upon its policies and procedures in these
areas.

Official Audit Report – Issued November 28, 2023
Hampden County District Attorney’s Office
For the period July 1, 2019 through June 30, 2021
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
Novemeber 28, 2023
District Attorney Anthony D. Gulluni
Hampden County District Attorney’s Office
50 State Street
Springfield, MA 01103
Dear District Attorney Gulluni:
I am pleased to provide to you the results of the enclosed performance audit of the Hampden County
District Attorney’s Office. As is typically the case, this report details the audit objectives, scope,
methodology, findings, and recommendations for the audit period, July 1, 2019 through June 30, 2021. As
you know, my audit team discussed the contents of this report with agency managers. This report reflects
those comments.
I appreciate you and all your efforts at the Hampden County District Attorney’s Office. The cooperation
and assistance provided to my staff during the audit went a long way toward a smooth process. Thank
you for encouraging and making available your team. I am available to discuss this audit if you or your
team have any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 4
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE........................................................................................ 7
1. The Hampden County District Attorney’s Office did not provide cybersecurity awareness training to its
employees. ................................................................................................................................................. 7
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has performed an audit of the Hampden County District Attorney’s Office (HCDA) for the period
July 1, 2019 through June 30, 2021.
In this performance audit, we examined the following:
 whether HCDA made forfeiture trust fund expenditures in accordance with Section 47(d) of
Chapter 94C of the General Laws;
 whether HCDA ensured that forfeited assets from closed cases were collected and deposited in
accordance with Section 47(d) of Chapter 94C of the General Laws; and
 whether HCDA ensured that its employees completed cybersecurity awareness training in
accordance with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and
Security’s Information Security Risk Management Standard IS.010.
Below is a summary of our finding and recommendations, with links to each page listed.
Finding 1
Page 7
HCDA did not provide cybersecurity awareness training to its employees.
Recommendations
Page 7
1. HCDA should create a policy and procedure to train new and existing employees on
cybersecurity awareness.
2. HCDA should provide cybersecurity awareness training to its employees within 30 days
of orientation and annually thereafter.
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Hampden County District Attorney’s Office (HCDA) was established under Sections 12 and 13 of
Chapter 12 of the Massachusetts General Laws, which provide for the administration of criminal law and
the defense of civil actions brought against the Commonwealth in accordance with Chapter 258 of the
General Laws.
HCDA is one of 11 district attorneys’ offices in the Commonwealth and represents the Commonwealth in
the prosecution of criminal offenses that occur within its jurisdiction. HCDA serves 23 cities and towns
across southwestern Massachusetts and serves a population of about 460,000 citizens. HCDA had a
budget of $12,429,625 in fiscal year 2020 and $13,951,535 in fiscal year 2021. HCDA’s main office is in
Springfield, with satellite locations in Chicopee, Holyoke, Palmer, and Westfield.
According to its website, HCDA “is proud to serve the people of Hampden County by faithfully pursuing
criminal justice and ensuring public safety with ethics, integrity, and fairness as [its] guiding values.”
HCDA’s forfeited asset revenue was $327,446 during the audit period. HCDA’s forfeiture trust fund
expenditures totaled $497,913 during the audit period. Forfeited asset revenue remains in HCDA’s
forfeiture trust fund account with the Office of the State Treasurer and Receiver General until expended,
as required by Section 47(d) of Chapter 94C of the General Laws. The unexpended balance at the end of
a fiscal year in the forfeiture trust fund account is rolled forward for the next fiscal year.
Asset Forfeiture
To prevent individuals from profiting from illegal drug activity, Section 47 of Chapter 94C of the General
Laws authorizes law enforcement agencies to seize assets, such as any profits of drug distribution or any
property that was used, or was intended to be used, for illegal drug activity. Some examples of assets that
may be subject to forfeiture are money, cell phones, computers, motor vehicles, and real property.1
The local or state police department that performed the seizure maintains possession of the seized assets
until a judge determines whether these assets should be forfeited to the Commonwealth. If assets are
ultimately deemed forfeited by a court order, then these assets are divided equally between HCDA and
the police department that performed the seizure and are moved to and held in a forfeiture trust fund
1. Real property (as opposed to personal property) includes land and additional structures/items in or on that land, such as
buildings, sheds, or crops.
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Overview of Audited Entity
3
account. If more than one police department was involved in the seizure, then the police departments
split a 50% share equitably.
According to Section 47(d) of Chapter 94C of the General Laws, HCDA may expend money from the
forfeiture trust fund for the following purposes:
To defray the costs of protracted investigations, to provide additional technical equipment or
expertise, to provide matching funds to obtain federal grants, or such other law enforcement
purposes as the district attorney . . . deems appropriate. The district attorney . . . may expend up
to ten percent of the monies and proceeds for drug rehabilitation, drug education and other antidrug or neighborhood crime watch programs which further law enforcement purposes.
Cybersecurity Awareness Training
The Executive Office of Technology Services and Security has established policies and procedures that
apply to all Commonwealth agencies within the executive branch. EOTSS recommends, but does not
require, non-executive branch agencies to follow these policies and procedures. Section 6.2 of EOTSS’s
Information Security Risk Management Standard IS.010 states,
The objective of the Commonwealth information security training is to educate users on their
responsibility to help protect the confidentiality, availability and integrity of the Commonwealth’s
information assets. Commonwealth Offices and Agencies must ensure that all personnel are
trained on all relevant rules and regulations for cybersecurity.
To ensure that employees are clear on their responsibilities, all employees in state executive agencies
with access to a Commonwealth-provided email address are required to complete a cybersecurity
awareness course every year. All newly hired employees must complete an initial security awareness
training course within 30 days of their orientation.
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Audit Objectives, Scope, and Methodology
4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Hampden County District Attorney’s
Office (HCDA) for the period July 1, 2019 through June 30, 2021.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Were expenditures from HCDA’s forfeiture trust fund appropriate and in compliance
with Section 47(d) of Chapter 94C of the General Laws?
Yes
2. Did HCDA ensure that all forfeited assets were collected and deposited in accordance
with Section 47(d) of Chapter 94C of the General Laws?
Yes
3. Did HCDA ensure that its employees completed cybersecurity awareness training in
accordance with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services
and Security’s Information Security Risk Management Standard IS.010?
No; see Finding 1
To achieve our audit objectives, we gained an understanding of HCDA’s internal control environment
related to the objectives by reviewing HCDA’s policies and procedures and interviewing HCDA staff
members and management. We evaluated the design and tested the operating effectiveness of the
internal control (specifically, supervisory approval) for forfeited trust fund expenditures.
Forfeiture Trust Fund Expenditures
To determine whether expenditures from HCDA’sforfeited trust fund were appropriate and in compliance
with Section 47(d) of Chapter 94C of the General Laws, we obtained a list from HCDA of all forfeiture trust
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Audit Objectives, Scope, and Methodology
5
fund expenditures that were made during the audit period. Using TeamMate Analytics,2 we selected a
nonstatistical, random sample of 5 forfeited trust fund expenditures (totaling $7,504) out of a total
population of 49 forfeited trust fund expenditures (totaling $497,913) made during the audit period.
We examined supporting documentation (including invoices, bills, and purchase orders) to determine
whether each expenditure was supported by documentation and was allowable under Section 47(d) of
Chapter 94C of the General Laws.
We noted no exceptions in our testing; therefore, we conclude that HCDA’s expenditures from its
forfeiture trust fund account were allowable and in compliance with Section 47(d) of Chapter 94C of the
General Laws.
Forfeited Assets
To determine whether HCDA ensured that all forfeited assets were accurately collected and deposited in
accordance with Section 47(d) of Chapter 94C of the General Laws, we obtained a list of all forfeited assets
that HCDA received during the audit period. Using TeamMate Analytics, we selected a nonstatistical,
random sample of 33 forfeited assets HCDA received (totaling $16,863) from a population of 499 (totaling
$327,446) from the audit period. We examined supporting documentation (including forfeiture orders,
checks to and from police departments, deposit slips, bank statements, and forfeiture trust fund account
activity) to determine whether forfeited assets were accurately collected and deposited.
We noted no exceptions in our testing; therefore, we conclude that HCDA ensured that all forfeited assets
were accurately collected and deposited in accordance with Section 47(d) of Chapter 94C of the General
Laws.
We used nonstatistical sampling methods and therefore did not project the results of our testing to any
population.
Cybersecurity Awareness Training
To determine whether HCDA ensured that its employees completed cybersecurity awareness training in
accordance with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and Security’s
Information Security Risk Management Standard IS.010, we obtained a list of all employees who worked
2. This is a Microsoft Excel-based data analytics tool that allows auditors to execute advanced data analysis.
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Audit Objectives, Scope, and Methodology
6
for part or all of the audit period. The list contained 190 employees. We interviewed HCDA staff members
about cybersecurity awareness training at the agency during the audit period. See Finding 1 for an issue
we identified with HCDA’s cybersecurity awareness training.
Data Reliability Assessment
In 2018 and 2022, the Office of the State Auditor performed data reliability assessments of the
Massachusetts Management Accounting and Reporting System (MMARS). These assessments focused on
testing selected system controls, including access, cybersecurity awareness, audit and accountability,
configuration management, identification and authentication, and personnel security. In addition, as part
of our current audit, we tested the controls in place over HCDA’s personnel security.
For the list of forfeited trust fund expenditures, we selected a random sample of five invoices from HCDA’s
hardcopy files and determined whether the information on the invoices matched the data in MMARS. We
also selected a random sample of five forfeited trust fund expenditures from MMARS and traced the
information to the invoices. For the list of employees, we selected a random sample of 10 employees from
HCDA’s personnel files and determined whether the information in the personnel files matched the data
in MMARS. We also selected a judgmental sample of 10 employees from MMARS and traced the
information to personnel files.
To determine the reliability of the data from the list of all forfeited assets HCDA received for the period
July 1, 2019 through June 30, 2021, we traced a sample of 20 forfeited assets from the list to the source
documents and selected 20 hardcopy documents to trace back to the list. In addition, we conducted tests
to identify any duplicates to determine the integrity of the information in the list.
Based on the results of our data reliability assessment procedures detailed above, we determined that
the information obtained for our audit period was sufficiently reliable for the purposes of our audit.
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Detailed Audit Findings with Auditee’s Response
7
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Hampden County District Attorney’s Office did not provide
cybersecurity awareness training to its employees.
The Hampden County District Attorney’s Office (HCDA) did not provide cybersecurity awareness training
to its employees during the audit period.
Without educating its employees on their responsibility to protect the security of information assets,
HCDA is exposed to a higher risk of cybersecurity attacks and financial and/or reputational losses.
Authoritative Guidance
The Executive Office of Technology Services and Security’s Information Security Risk Management
Standard IS.010 states,
6.2.3 New Hire Security Awareness Training: All new personnel must complete an Initial Security
Awareness Training course. . . . The New Hire Security Awareness course must be
completed within 30 days of new hire orientation.
6.2.4 Annual Security Awareness Training: All personnel will be required to complete Annual
Security Awareness Training.
Although HCDA is not required to follow this standard, we consider it a best practice.
Reasons for Issue
HCDA did not have policies and procedures that require new employees to complete cybersecurity
awareness training within 30 days of their orientation or that require employees to receive annual
cybersecurity awareness training.
Recommendations
HCDA should create a policy and procedure to train new and existing employees on cybersecurity
awareness.
HCDA should provide cybersecurity awareness training to its employees within 30 days of orientation
and annually thereafter.
Auditee’s Response
During the audit period, the Hampden District Attorney's Office did not have a specific cybersecurity
training program in place. However, all employees were instructed regarding security measures
Audit No. 2022-1259-3J Hampden County District Attorney’s Office
Detailed Audit Findings with Auditee’s Response
8
and how to report breaches of security should they occur. Knowing the importance of having a
specific training regimen, this office was in the process of securing cybersecurity awareness training
during the audit period.
When the audit was begun in July of 2022, the Hampden District Attorney's Office had a policy and
procedure in place for all employees regarding cybersecurity awareness training. This consists of
periodic training sessions throughout the year as well as security awareness testing. Therefore, the
recommendations resulting from the finding have been implemented.
Auditor’s Reply
Based on its response, HCDA has taken measures to address our concerns on this matter.

Official Audit Report – Issued June 23, 2023
Massachusetts Developmental Disabilities Council
For the period July 1, 2020 through June 30, 2022
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
June 23, 2023
Dan Shannon, Executive Director
Massachusetts Developmental Disabilities Council
108 Myrtle Street, Suite 202
Quincy, MA 02171
Dear Mr. Shannon:
I am pleased to provide to you the results of the enclosed performance audit of the Massachusetts
Developmental Disabilities Council. As is typically the case, this report details the audit objectives, scope,
methodology, findings, and recommendations for the audit period, July 1, 2020 through June 30, 2022. As
you know, my audit team discussed the contents of this report with agency managers. This report reflects
those comments.
I appreciate you and all your efforts at the Massachusetts Developmental Disabilities Council. The
cooperation and assistance provided to my staff during the audit went a long way toward a smooth
process. Thank you for encouraging and making available your team. I am available to discuss this audit if
you or your team have any questions.
Sincerely,
Diana DiZoglio
Auditor of the Commonwealth
cc: Craig Hall, Chief Financial Officer of the Massachusetts Developmental Disabilities Council
Rebecca Fillmore, Acting Chairperson of the Massachusetts Developmental Disabilities Council
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 5
APPENDIX ............................................................................................................................................................ 10
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Massachusetts Developmental Disabilities Council
(MDDC) for the period July 1, 2020 through June 30, 2022. In this performance audit, we examined
whether MDDC developed a five-year state plan in accordance with Executive Order 512c and whether
MDDC reviewed and monitored the state plan’s goals and activities. In addition, we examined whether
MDDC developed programs and trainings for individuals with limited English proficiency in accordance
with Section 1385 of Title 45 of the Code of Federal Regulations.
We also examined whether MDDC updated its internal control plan to address risks related to the 2019
coronavirus (COVID-19) pandemic in compliance with the Office of the Comptroller of the
Commonwealth’s “COVID-19 Pandemic Response Internal Controls Guidance” and whether MDDC
personnel who had access to COVID-19 funds completed cybersecurity awareness training in accordance
with Sections 6.2.3 and 6.2.4 of the Executive Office of Technology Services and Security’s Information
Security Risk Management Standard IS.010.1
Our audit revealed no significant instances of noncompliance by MDDC that must be reported under
generally accepted government auditing standards.
1. While all state employees are required to take the cybersecurity awareness training annually, this audit only looked at MDDC’s
employees who were responsible for the management of COVID-19 funds.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Massachusetts Developmental Disabilities Council (MDDC) is an independent agency funded through
federal grant programs under Subtitle B of the Developmental Disabilities Assistance and Bill of Rights Act
of 2000 (Federal Public Law 106-402). MDDC was established in accordance with Executive Order 512,
signed by the Governor in 2009. It is made up of 14 members appointed by the Governor, as well as
representatives from state agencies and the Massachusetts Developmental Disabilities Network.2
According to its website,
MDDC is an independent agency, funded by the federal government, dedicated to empowering
people with developmental disabilities and their families to enjoy full productive lives by promoting
self-sufficiency, community inclusion & opportunity.
During the audit period, MDDC received federal funding of approximately $1,282,120 in 2020 and
$1,192,317 in 2021. MDDC, located at 108 Myrtle Street in Quincy, is led by an executive director, who
manages its 14 employees.
State Plan and Goals
Under Subtitle B of the Developmental Disabilities Assistance and Bill of Rights Act of 2000, MDDC is
responsible for developing a five-year state plan that outlines its goals and planned activities (e.g.,
trainings and meetings). MDDC submits its state plan to the Administration for Community Living within
the United States Department of Health and Human Services for review and approval. MDDC, in an effort
to accomplish its goals, contracts with external organizations to set up and provide trainings for individuals
with disabilities and their families. MDDC awards grants to organizations that apply and meet the
requirements outlined in its Notice of Funding Available.3
MDDC is required to monitor its state plan implementation and report its progress at least once per year.
To meet these reporting requirements, MDDC does the following:
 MDDC submits its own Annual Program Performance Report to the Administration for Community
Living. (See Appendix for examples of MDDC’s objectives and performance measures for fiscal
year 2021.) The Administration for Community Living then reviews the Annual Program
2. This network consists of MDDC, the Institute for Community Inclusion, the University of Massachusetts Chan Medical School
Shriver Center, and the Disability Law Center. According to its website, the network provides “advocacy, education, research
and dissemination of information.”
3. Notices of Funding Available are public announcements that MDDC has funding for, and is looking to enter into, agreements
with qualified organizations to provide trainings that align with MDDC’s state plan and/or goals.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Overview of Audited Entity
3
Performance Report, provides feedback about the strengths and weakness of MDDC’s trainings,
and approves the Annual Program Performance Report.
 MDDC requires its grantees and council members to submit quarterly reports regarding the
progress of their trainings and/or activities. MDDC’s Notices of Funding Available include a
stipulation about monitoring and reporting requirements. The following is an example of some
requirements included in MDDC’s Notice of Funding Available:
MDDC Grantees submit periodic and final reports. . . . Periodic program reports will
summarize project activities, project participant demographic data and performance
measures. Grantees will submit a final program report with an overview of the grant
project activities and accomplishments, output, outcomes and federal performance
measure data. Addenda to be submitted with the final report will include grant
products, deliverables, survey copies or summary information and evaluation
documents. . . . Grantees will distribute the MDDC participant surveys to project
participants with developmental disabilities and family members, provide survey copies
and report survey summary data to the MDDC. . . . Final drafts of any training
materials, publications, videos, websites or other products shall be reviewed and
approved by the MDDC prior to dissemination to the general public.
Individuals with Limited English Proficiency
MDDC’s 2020 Annual Report to the Citizens of Massachusetts highlighted significant barriers to reaching
individuals with developmental disabilities who also have limited English proficiency and these individuals’
families. MDDC is subject to Section 1385.3(1)(ii) of Title 45 of the Code of Federal Regulations, which
requires it to offer assistive aids to ensure that all individuals with disabilities, including those with limited
English proficiency, understand their rights and can access MDDC’s trainings and other resources.
The Office of the Comptroller of the Commonwealth’s Pandemic Response
Guidance
On September 30, 2020, the Office of the Comptroller of the Commonwealth provided guidance in
response to the 2019 coronavirus (COVID-19) pandemic for state agencies. The guidelines helped state
agencies experiencing significant changes to their business processes to identify their goals, objectives,
and risks associated with COVID-19. Objectives could include telework; return-to-office plans; a risk
assessment of the impact of COVID-19 on state agency operations; changes to the business process; safety
protocols for staff members and visitors; and tracking of COVID-19–related awards and expenditures,
which were tracked separately from other federal, state, and local expenditures. The guidance also stated
that Commonwealth agencies experiencing a significant impact should draft separate COVID-19 Pandemic
Response Plan Appendixes to their internal control plans.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Overview of Audited Entity
4
Cybersecurity Awareness Training
The Executive Office of Technology Services and Security (EOTSS) has established policies and procedures
that apply to all Commonwealth agencies using EOTSS-managed information technology infrastructure,
such as email, websites, etc. EOTSS’s Information Security Risk Management Standard IS.010 requires that
all Commonwealth personnel are trained annually in cybersecurity awareness. According to Section 6.2
of EOTSS’s Information Security Risk Management Standard IS.010, “The objective of the Commonwealth
information security training is to educate users on their responsibility to help protect the confidentiality,
availability and integrity of the Commonwealth’s information assets.”
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Audit Objectives, Scope, and Methodology
5
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts Developmental
Disabilities Council (MDDC) for the period July 1, 2020 through June 30, 2022.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer and the
conclusion we reached regarding each objective.
Objective Conclusion
1. Did MDDC develop a state plan as required by Section 124 of the Developmental
Disabilities Assistance and Bill of Rights Act of 2000?
Yes
2. Did MDDC review and monitor Objectives 1.1–1.4 of Goal 1 (Advocacy and Leadership),
as stated in its Five-Year State Plan for 2017–2021, as required by Executive Order
512c?
Yes
3. Did MDDC review and monitor Objective 3.2 of Goal 3 (Inclusive Communities), as
stated in its Five-Year State Plan for 2017–2021, as required by Executive Order 512c?
Yes
4. Did MDDC develop programs and trainings for individuals with limited English
proficiency in accordance with Section 1385.3(1)(ii) of Title 45 of the Code of Federal
Regulations?
Yes
5. Did MDDC update its internal control plan to address the 2019 coronavirus (COVID-19),
goals, objectives, and risks, as required by the Office of the Comptroller of the
Commonwealth’s “COVID-19 Pandemic Response Internal Controls Guidance”?
Yes
6. Did MDDC ensure that MDDC employees who were responsible for the management
of COVID-19 funds completed cybersecurity awareness training in accordance with the
Executive Office of Technology Services and Security’s (EOTSS’s) Information Security
Risk Management Standard IS.010?
Yes
To achieve our audit objectives, we gained an understanding of MDDC’s internal control environment
related to the objectives by both reviewing applicable policies and procedures and interviewing MDDC
employees and management.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Audit Objectives, Scope, and Methodology
6
To obtain sufficient, appropriate audit evidence to address our audit objectives, we performed the
following procedures.
State Plan
We inspected MDDC’s Five-Year State Plan for 2021–2026 to determine whether it included the following
criteria, which are listed in Section 124 of the Developmental Disabilities Assistance and Bill of Rights Act
of 2000:
(3) COMPREHENSIVE REVIEW AND ANALYSIS. . . .
(C) an analysis of the extent to which community services and opportunities related to the
areas of emphasis directly benefit individuals with developmental disabilities, especially
with regard to their ability to access and use services provided in their communities, to
participate in opportunities, activities, and events offered in their communities, and to
contribute to community life. . . .
(4) PLAN GOALS. . . .
(A) specifying 5-year goals, as developed through data driven strategic planning, for advocacy,
capacity building, and systemic change related to the areas of emphasis, to be undertaken
by the Council, that— . . .
(ii) include a goal, for each year of the grant, to—
(I) establish or strengthen a program for the direct funding of a State self-advocacy
organization led by individuals with developmental disabilities;
(II) support opportunities for individuals with developmental disabilities who are
considered leaders to provide leadership training to individuals with
developmental disabilities who may become leaders; and
(III) support and expand participation of individuals with developmental disabilities in
cross-disability and culturally diverse leadership coalitions; and
(B) for each year of the grant, describing—
(i) the goals to be achieved through the grant, which, beginning in fiscal year 2002, shall
be consistent with applicable indicators of progress described in section 104(a)(3);
(ii) the strategies to be used in achieving each goal; and
(iii) the method to be used to determine if each goal has been achieved. . . .
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Audit Objectives, Scope, and Methodology
7
(5) ASSURANCES. . . .
(D) CONFLICT OF INTEREST.—The plan shall provide an assurance that no member of such
Council will cast a vote on any matter that would provide direct financial benefit to the
member or otherwise give the appearance of a conflict of interest.
(E) URBAN AND RURAL POVERTY AREAS.—The plan shall provide assurances that special
financial and technical assistance will be given to organizations that provide community
services, individualized supports, and other forms of assistance to individuals with
developmental disabilities who live in areas designated as urban or rural poverty areas.
(F) PROGRAM ACCESSIBILITY STANDARDS.—The plan shall provide assurances that
programs, projects, and activities funded under the plan, and the buildings in which such
programs, projects, and activities are operated, will meet standards prescribed by the
Secretary in regulations and all applicable Federal and State accessibility standards,
including accessibility requirements of the Americans with Disabilities Act of 1990, . . . the
Rehabilitation Act of 1973, . . . and the Fair Housing Act. . . .
(G) INDIVIDUALIZED SERVICES.—The plan shall provide assurances that any direct services
provided to individuals with developmental disabilities and funded under the plan will be
provided in an individualized manner, consistent with the unique strengths, resources,
priorities, concerns, abilities, and capabilities of such individual.
(H) HUMAN RIGHTS.—The plan shall provide assurances that the human rights of the
individuals with developmental disabilities . . . who are receiving services under programs
assisted under this subtitle will be protected . . . .
(I) MINORITY PARTICIPATION.—The plan shall provide assurances that the State has taken
affirmative steps to assure that participation in programs funded under this subtitle is
geographically representative of the State, and reflects the diversity of the State with
respect to race and ethnicity.
Goals
We met with MDDC officials to learn how MDDC reviewed and monitored the progress on its goals in its
Five-Year State Plan for 2017–2021. Specifically, we looked at Objectives 1.1–1.4 of Goal 1 (Advocacy and
Leadership) and Objective 3.2 of Goal 3 (Inclusive Communities), listed below:
Goal #1: Advocacy and Leadership. . . .
Objective 1. A minimum of 100 youth with developmental disabilities will develop advocacy
skills and engage in disability advocacy by September 30, 2021.
Objective 2. A minimum of 200 adults with developmental disabilities will develop advocacy
skills and engage in disability advocacy by September 30, 2021.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Audit Objectives, Scope, and Methodology
8
Objective 3. A minimum of 100 minority students with developmental disabilities and family
members will develop advocacy skills and engage in disability advocacy by
September 30, 2021.
Objective 4. A minimum of 50 people with developmental disabilities and family members will
become leaders in disability advocacy by September 30, 2021. . . .
Goal #3: Inclusive Communities. . . .
Objective 2. A minimum of 100 people with developmental disabilities will develop the skills
needed to obtain competitive, integrated employment by September 30, 2021.
We determined whether each grantee and/or council member submitted quarterly reports and verified
that MDDC approved each report submitted by each grantee and/or council member.
Limited English Proficiency
We interviewed MDDC officials to gather evidence of how MDDC ensured that individuals with limited
English proficiency had access to resources and trainings. We reviewed MDDC’s Notices of Funding
Available, the grantees’ applications, and the grantees’ and/or council members’ quarterly reports. In
addition, we reviewed post-training surveys (which are filled out by attendees) from grantees’ trainings
to determine whether MDDC’s grantees provided the trainings they were contracted to provide and that
the trainings were related to MDDC’s goals.
Internal Control Plan
To determine whether MDDC updated its internal control plan to address the effects of COVID-19, as
required by the Office of the Comptroller of the Commonwealth’s “COVID-19 Pandemic Response Internal
Controls Guidance,” we obtained the fiscal year 2021 internal control plan from MDDC and reviewed it.
Cybersecurity Awareness Training Attendance
To determine whether MDDC employees responsible for managing COVID-19 funds completed
cybersecurity awareness training, in accordance with EOTSS’s Information Security Risk Management
Standard IS.010, we reviewed the list of all MDDC employees who had access to COVID-19 funds and
cross-referenced this list with the certificates of completion for all employees who completed
cybersecurity awareness training.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Audit Objectives, Scope, and Methodology
9
Data Reliability Assessment
We performed validity and integrity tests on the list of MDDC employees who managed COVID-19 funds.
These tests included checking for duplicates, checking for blank fields, and comparing the list’s data to
information available in the Massachusetts Management Accounting and Reporting System.
In 2018 and 2022, the Office of the State Auditor performed a data reliability assessment of the
Massachusetts Management Accounting and Reporting System by testing selected information system
general controls (access controls, application controls, configuration management, contingency planning,
and segregation of duties). Based on the results of our data reliability procedures as described above, we
determined the data to be sufficiently reliable for the purpose of this audit.
Conclusion
Our audit revealed no significant instances of noncompliance that must be reported under generally
accepted government auditing standards.
Audit No. 2022-1460-3H Massachusetts Developmental Disabilities Council
Appendix
10
APPENDIX
The following table summarizes some of MDDC’s objectives (specifically, the ones we reviewed during our
audit) for 2017–2021. MDDC determined whether it met these objectives by counting how many people
with developmental disabilities participated in its activities, which were mostly trainings, during this fiveyear period.
Objective Activity Participants
[1.1] A minimum of 100 youth with developmental disabilities will develop effective
skills and engage in self-advocacy by September 30, 2021. 235
[1.2] A minimum of 200 adults with developmental disabilities will improve interactive
skills and engage in self-advocacy by September 30, 2021. 218
[1.3] A minimum of 100 students with developmental disabilities and family members
from racially diverse communities will develop advocacy skills and engage in selfadvocacy by September 30, 2021.
119
[1.4] A minimum of 50 people with developmental disabilities and family members will
develop and demonstrate leadership skills by September 30, 2021. 165
[3.2] A minimum of 100 people with developmental disabilities will develop the skills
needed to obtain competitive, integrated employment by September 30, 2021. 108
Source: MDDC’s Annual Program Performance Report for Fiscal Year 2021

Official Audit Report – Issued November 3, 2025
Massachusetts State Employee Retirement Board
For the period July 1, 2023 through June 30, 2024
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
November 3, 2025
Kathryn Kougias, Executive Director
Massachusetts State Retirement Board
1 Winter Street, 8th Floor
Boston, MA 02136
Dear Executive Director Kougias:
I am pleased to provide to you the results of the enclosed performance audit of the Massachusetts State
Retirement Board. As is typically the case, this report details the audit objectives, scope, and
methodology for the audit period, July 1, 2023 through June 30, 2024. As you know, my audit team
discussed the contents of this report with agency managers. This report reflects those comments.
I appreciate you and all your efforts at the Massachusetts State Retirement Board. The cooperation and
assistance provided to my staff during the audit went a long way toward a smooth process. Thank you
for encouraging and making available your team. I am available to discuss this audit if you or your team
has any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 7
Audit No. 2025-0088-3I Massachusetts State Retirement Board
List of Abbreviations
ii
LIST OF ABBREVIATIONS
EOTSS Executive Office of Technology Services and Security
IT information technology
MSRB Massachusetts State Retirement Board
W3C World Wide Web Consortium
WCAG Web Content Accessibility Guidelines
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts State Retirement
Board (MSRB) for the period July 1, 2023 through June 30, 2024.
The purpose of our audit was to determine whether MSRB’s website adhered to the World Wide Web
Consortium’s Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard
accessibility, navigation accessibility, language accessibility, error identification, and color accessibility.
Adherence to WCAG helps ensure that all users, regardless of ability, can access the content and functions
of MSRB’s website.
Our audit revealed no significant instances of noncompliance by MSRB that must be reported under
generally accepted government auditing standards.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
In 1911, the Commonwealth of Massachusetts established the Massachusetts State Retirement Board
(MSRB) under Chapter 532 of the Acts of 1911 and charged it with administering the Massachusetts State
Employees’ Retirement System (MSERS), one of 104 public contributory retirement systems in the
Commonwealth. MSRB also administers the former Massachusetts Turnpike Authority Employees’
Retirement System (MTAERS) and benefits for certain other groups, including employees of the
Commonwealth’s judiciary.
MSRB processes retirement applications, ordinary and accidental disability retirements, refunds, service
purchases, and survivor benefits for members and beneficiaries. It also provides retirement information
to state employees. All of these functions are performed in accordance with Chapter 32 of the
Massachusetts General Laws and applicable regulations. Investments of MSERS and MTAERS assets are
overseen by the Pension Reserves Investment Management Board and are held in the Pension Reserves
Investment Trust.
MSRB is a department within the Office of the State Treasurer and Receiver General and is governed by a
five‑member board established by Section 18 of Chapter 10 of the General Laws. The board consists of
the State Treasurer and Receiver General (chair), one member appointed by the Treasurer (a retired
member), two members elected by current and active MSERS members, and one member chosen by the
other four board members. The board is required to meet at least once a month. MSRB maintains offices
at One Winter Street, 8th Floor, in Boston and 436 Dwight Street, Room 109A, in Springfield.
According to MSRB’s annual report, as of June 30, 2019, the agency employed 76 permanent full‑time,
four permanent part‑time, and three contract employees, serving more than 89,000 active MSERS and
MTAERS members and more than 64,000 retirees and survivors. In recent years, MSRB has expanded its
virtual and online member services, including webinars and remote offerings.
For fiscal years 2018 and 2019, MSERS disbursed approximately $2.23 billion and $2.34 billion,
respectively, and MTAERS disbursed approximately $15.59 million and $15.16 million, respectively, in
annuities and pensions.
MSRB maintains a web presence on Mass.gov, where users can find information about the board, office
locations, meeting schedules, and other resources for members and retirees.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Overview of Audited Entity
3
Massachusetts Requirements for Accessible Websites
In 1999, the World Wide Web Consortium (W3C), an international nongovernmental organization
responsible for internet standards, published the Web Content Accessibility Guidelines (WCAG) 1.0 to
provide guidance on how to make web content more accessible to people with disabilities.
In 2005, the Massachusetts Office of Information Technology,1 with the participation of state government
webpage developers, including developers with disabilities, created the Enterprise Web Accessibility
Standards. These standards required all executive branch state agencies to follow the guidelines in Section
508 of the Rehabilitation Act amendments of 1998. These amendments went into effect in 2001 and
established precise technical requirements to which electronic and information technology (IT) products
must adhere. This technology includes, but is not limited to, products such as software, websites,
multimedia products, and certain physical products, such as standalone terminals.
In 2008, W3C published WCAG 2.0. In 2014, the Massachusetts Office of Information Technology added a
reference to WCAG 2.0 in its Enterprise Information Technology Accessibility Standards.
In 2017, the Executive Office of Technology Services and Security (EOTSS) was designated as the
Commonwealth’s lead IT organization for executive branch state agencies. EOTSS is responsible for the
development and maintenance of the Enterprise Information Technology Accessibility Standards2 and the
implementation of state and federal laws and regulations related to accessibility. As the principal
executive agency responsible for coordinating the Commonwealth’s IT accessibility compliance efforts,
EOTSS supervises executive branch state agencies in their efforts to meet the Commonwealth’s
accessibility requirements.
In 2018, W3C published WCAG 2.1, which built on WCAG 2.0 to improve web accessibility on mobile
devices and to further improve web accessibility for people with visual impairments and cognitive
disabilities. EOTSS published the Enterprise Information Technology Accessibility Policy in 2021 to meet
Levels A and AA of WCAG 2.1.
1. The Massachusetts Office of Information Technology became the Executive Office of Technology Services and Security in
2017.
2. EOTSS has since changed the titles and numbers of at least some of its policies and standards between the end of the audit
period and the publication of this report. In this report, we reference the titles and numbers of EOTSS’s policies and/or
standards as they were during the audit period (unless stated otherwise).
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Overview of Audited Entity
4
Timeline of the Adoption of Website Accessibility Standards by the Federal
Government and Massachusetts
Executive branch state agencies, such as MSRB, must comply with EOTSS’s policies and standards when
using an EOTSS web domain,3 as established by EOTSS’s Website Domain Policy. Part of this policy states
that any government organization using an EOTSS web domain must comply with EOTSS’s Web Design
Guidelines, which were published in 2020 and were based on the federal 21st Century Integrated Digital
Experience Act. This law helps state government agencies evaluate their website design and
implementation decisions to meet state accessibility requirements.
3. EOTSS web domains, according to its Website Domain Policy, include Mass.gov, Massachusetts.gov, Ma.gov, State.ma.us,
related subdomains (e.g., example.mass.gov), and all domains that EOTSS owns and manages.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Overview of Audited Entity
5
Web Accessibility
Government websites are an important way for the general public to access government information and
services. Deloitte’s4 2023 Digital Citizen Survey found that 55% of respondents preferred to interact with
their state government services through a website instead of face-to-face interaction or a call center.
Commonwealth of Massachusetts websites have millions of webpage views each month.
However, people do not interact with the internet uniformly. The federal government and nongovernmental
organizations have established web accessibility standards intended to make websites more accessible to
people with disabilities, such as visual impairments, hearing impairments, and others. The impact of these
standards can be significant, as the federal Centers for Disease Control and Prevention estimates that
1,488,012 adults (26% of the adult population) in Massachusetts have a disability, as of 2022.5 Among the
estimated 26% of the adult population, 14% reported having serious difficulty with cognition, 10% reported
having serious difficulty with mobility, 6% reported having deafness or serious difficulty hearing, and 5%
reported having blindness or serious difficulty seeing (even when wearing glasses).6 Examples of web
accessibility measures include, but are not limited to, having captioning on videos to help people with
difficulty hearing understand the contents of the video, having form fields describe what data needs to be
input into them to help people who have cognitive difficulties, and ensuring that people can interact with a
webpage using keyboard commands alone to help people who have difficulty with mobility.
How People with Disabilities Use the Web
According to W3C, people with disabilities use assistive technologies and adaptive strategies specific to
their needs to navigate web content. Examples of assistive technologies include screen readers, which
read webpages aloud for people who cannot read text; screen magnifiers for individuals with low vision;
and voice recognition software for people who cannot (or do not) use a keyboard or mouse. Adaptive
strategies refer to techniques that people with disabilities employ to enhance their web interactions.7
These strategies might involve increasing text size, adjusting mouse speed, or enabling captions.
4. Deloitte is an international company that provides tax, accounting, and audit services to businesses and government agencies.
5. This data is collected from surveys conducted using both landline telephones and cellular telephones, and all responses are
self-reported.
6. The percentages do not add up to 26%, as estimated by the federal Centers for Disease Control, because of overlapping data.
The self-reported survey allows individuals to report having multiple disabilities.
7. Web interaction refers to the various actions that users can take while navigating and using the internet. It encompasses a
wide range of online activities, including, but not limited to, clicking on hyperlinks, submitting forms, posting comments on
webpages, and engaging with web content and services in other forms.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Overview of Audited Entity
6
To make web content accessible to people with disabilities, developers must ensure that various
components of web development and interaction work together. This includes text, images, and structural
code, users’ browsers and media players, and various assistive technologies.
Accessibility Features of a Website8
8. We resized this webpage to fit in this audit report. To see the unaltered webpage, visit https://www.mass.gov/newslettersand-publications-msrb.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Audit Objectives, Scope, and Methodology
7
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts State Retirement
Board (MSRB) for the period July 1, 2023 through June 30, 2024.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our conclusion based on our audit objective. We believe that
the evidence obtained provides a reasonable basis for our conclusion based on our audit objective.
Below is our audit objective, indicating the question we intended our audit to answer and the conclusion
we reached regarding the objective.
Objective Conclusion
1. Did MSRB’s website comply with the Executive Office of Technology Services and
Security’s Enterprise Information Technology Accessibility Policy and the Web Content
Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard accessibility,
navigation accessibility, language accessibility, error identification, and color accessibility?
Yes
To accomplish our audit objective, we gained an understanding of the MSRB internal control environment
relevant to our objective by reviewing applicable policies and procedures and by interviewing MSRB staff
members and management. In addition, to obtain sufficient, appropriate evidence to address our audit
objective, we performed the procedures described below.
Web Accessibility
To determine whether MSRB’s website adhered to WCAG 2.1, for user accessibility, keyboard accessibility,
navigation accessibility, language accessibility, error identification, and color accessibility during the audit
period, we performed the following procedures for a random, nonstatistical9 sample of 60 webpages from
a population of 701 MSRB webpages:
9. Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items
are not similar enough, or there are specific items in the population that the auditors want to review.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Audit Objectives, Scope, and Methodology
8
User Accessibility
• We determined whether content on each webpage could be viewed in both portrait and
landscape modes.
• We determined whether content on each webpage was undamaged and remained readable
when zoomed in to both 200% and 400%.
Keyboard Accessibility
• We determined whether all elements10 on each webpage could be navigated using only
keyboard commands.
• We determined whether any elements on each webpage prevented a user from moving to a
different element when using only keyboard commands to navigate the webpage in question.
• We determined whether the first focusable control11 on each webpage was a hyperlink that
would redirect users to the main content of the webpage.
Navigation Accessibility
• We determined whether each webpage contained a title that was relevant to the webpage’s
content.
• We determined whether there was a search function present to help users locate content
across the whole website.
• We determined whether hyperlinks correctly navigated to the intended webpages.
• We determined whether headings within webpages related to the content of the section
below the header.
Language Accessibility
• We determined whether any video content found on each webpage had all important sounds
and dialogue captioned.
• We determined whether the words that appeared on each webpage matched the language
attribute12 to which the webpage in question was set.
• We determined whether any webpage sections that contained language differing from that
to which the webpage was set contained their own specified language attribute.
Error Identification
• We determined whether mandatory form fields alerted users if they left these fields blank.
10. An element is a part of a webpage that contains data, text, or an image.
11. The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard. If
first focusable control also redirects users to the main content of a webpage, then it is known as a bypass block or a skip link.
12. A language attribute (also known as a language tag) identifies the native language of the content on the webpage or PDF
(e.g., a webpage in English should have an EN language attribute). The language attribute is listed in the webpage’s or PDF’s
properties. This, among other things, is used to help screen readers use the correct pronunciation for words.
Audit No. 2025-0088-3I Massachusetts State Retirement Board
Audit Objectives, Scope, and Methodology
9
• We determined, for form fields that required a limited set of input values, whether users were
alerted if invalid values were entered into these types of fields.
• We determined whether there were labels for any elements that required user input. We also
determined whether these labels were programmed correctly.
• We determined whether examples were presented to assist users in correcting mistakes (for
example, a warning when entering a letter in a field meant for numbers).
Color Accessibility
• We determined whether there was at least a 3:1 contrast in color and additional visual cues
to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other
visual impairments.
We used nonstatistical sampling methods for testing and, therefore, did not project the results of our
testing to any corresponding populations.
Data Reliability Assessment
To determine the reliability of the URL list that we received from MSRB management, we interviewed
knowledgeable MSRB staff members and checked that certain variable formats (e.g., dates, unique
identifiers, and abbreviations) were accurate. Additionally, we ensured that none of the following issues
affected the URL list: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells,
or absent records), and duplicate records. We also ensured that all values in the dataset corresponded
with expected values.
We selected a random sample of 20 URLs from the URL list and traced each to the corresponding
webpages on MSRB’s website, checking that each URL and webpage title from the URL list matched the
information on the MSRB website. We also selected a random sample of 20 URLs from MSRB’s website
and traced the URL and webpage title to the URL list to ensure that there was a complete and accurate
population of URLs on the URL list.
Based on the results of the data reliability procedures described above, we determined that the URL list
was sufficiently reliable for the purposes of our audit.
Conclusion
Our audit revealed no significant instances of noncompliance by MSRB that must be reported under
generally accepted government auditing standards; therefore, this report contains no findings.

Official Audit Report – Issued October 14, 2021
Massachusetts Teachers’ Retirement System
For the period July 1, 2017 through June 30, 2019
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
October 14, 2021
Ms. Erika M. Glaster, Executive Director
Massachusetts Teachers’ Retirement System
500 Rutherford Avenue
Charlestown, MA 02129
Dear Ms. Glaster:
I am pleased to provide this performance audit of the Massachusetts Teachers’ Retirement System. This
report details the audit objectives, scope, methodology, findings, and recommendations for the audit
period, July 1, 2017 through June 30, 2019. My audit staff discussed the contents of this report with
management of the agency, whose comments are reflected in this report.
I would also like to express my appreciation to the Massachusetts Teachers’ Retirement System for the
cooperation and assistance provided to my staff during the audit.
Sincerely,
Suzanne M. Bump
Auditor of the Commonwealth
cc: Dr. Ventura Rodriguez, Chair, Massachusetts Teachers’ Retirement Board
John W. Parsons, Esq., Executive Director, Public Employee Retirement Administration
Commission
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 3
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 8
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE ....................................................................................12
1. The Massachusetts Teachers’ Retirement System did not always make initial benefit payments within the
mandated timeframe. ............................................................................................................................12
2. MTRS made approximately $1,470 of monthly benefit payments to a member whose account contained
an invalid Social Security number. ..........................................................................................................15
APPENDIX .........................................................................................................................................................18
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
List of Abbreviations
ii
LIST OF ABBREVIATIONS
BAFU Benefit Adjustment and Finalization Unit
IRS Internal Revenue Service
MTRS Massachusetts Teachers’ Retirement System
OSA Office of the State Auditor
PERAC Public Employee Retirement Administration Commission
PRIT Pension Reserves Investment Trust
SSN Social Security number
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Executive Summary
1
EXECUTIVE SUMMARY
The Massachusetts Teachers’ Retirement System (MTRS) is the largest contributory retirement system in
the Commonwealth. According to its website, MTRS provides “retirement, disability, and survivor
benefits to Massachusetts teachers, administrators and their families.” MTRS is administered by the
Massachusetts Teachers’ Retirement Board, and it is overseen and regulated by the Public Employee
Retirement Administration Commission. In fiscal year 2019, MTRS issued more than $3 billion in benefit
payments to more than 67,000 retirees and survivors.
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of MTRS for the period July 1, 2017 through June 30, 2019.1
The purpose of our audit was to determine whether (1) MTRS ensured that members received their first
pension payments within the timeframe established by Section 13(1)(b) of Chapter 32 of the General
Laws, (2) MTRS ensured that adjustments made to monthly benefit payments after the issuance of the
first payments were completed accurately in accordance with Section 5(2) of Chapter 32 of the General
Laws, and (3) MTRS promptly identified deceased retirees and beneficiaries and subsequently adjusted
or terminated benefit payments accurately in accordance with Section 12(2) of Chapter 32 of the
General Laws.
Below is a summary of our findings and recommendations, with links to each page listed.
Finding 1
Page 12
MTRS did not always make initial benefit payments within the mandated timeframe.
Recommendation
Page 13
MTRS should examine the benefit of redeploying existing resources toward processing
retirement applications that are received 60 days or less before proposed retirement dates
in an effort to achieve a higher percentage of “compliant” retirements.2

1. Generally accepted government auditing standards require that organizations be free from organizational impairments to
independence with respect to the entities they audit. Under Section 16 of Chapter 15 of the General Laws, the
Massachusetts Teachers’ Retirement Board consists of seven members, including the State Auditor. This disclosure is made
for informational purposes only, and this circumstance did not interfere with our ability to perform our audit work and
report its results impartially.
2. For audit testing purposes, we defined “compliant” as “processed within 62 days.” By statute, the timeframe for
compliance can range from 28 to 62 days depending on the day of the month a retirement is effective, as outlined in the
“Overview of Audited Entity” section of this report under “Retirement Application Process.”
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Executive Summary
2
Finding 2
Page 15
MTRS made approximately $1,470 of monthly benefit payments to a member whose
account contained an invalid Social Security number (SSN).
Recommendations
Page 16
1. MTRS should correct the member’s SSN in MyTRS and issue a corrected Form 1099-R to
both the member and the Department of the Treasury’s Internal Revenue Service (IRS).
2. MTRS should ensure that all benefits are paid and reported to the IRS under the correct
SSN.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Overview of Audited Entity
3
OVERVIEW OF AUDITED ENTITY
The Massachusetts Teachers’ Retirement System (MTRS) was established on July 1, 1914. According to
its website,
The MTRS, which is the largest of the Commonwealth’s 104 contributory retirement systems,
provides retirement, disability and survivor benefits to Massachusetts teachers, administrators
and their families.
Chapter 32 of the Massachusetts General Laws establishes the system’s benefits, contribution
requirements, and accounting structure. Teachers and administrators in Massachusetts public schools
(except those employed by the City of Boston3
), educational collaboratives, and charter schools are
eligible for membership.
All members are required to enroll with MTRS and make mandatory pretax contributions through
payroll deductions. Members contribute a percentage of their earnings based on the date they were
hired and when they joined the public employee retirement system:
Date Hired Contribution Rate
Before January 1, 1975 5%
January 1, 1975 through December 31, 1978 7%
January 1, 1979 through December 31, 1983 7%, plus 2% on earnings over $30,000
January 1, 1984 through June 30, 1996 8%, plus 2% on earnings over $30,000
July 1, 1996 through June 30, 2001 9%, plus 2% on earnings over $30,000
July 1, 2001 through April 1, 2012 11%
April 2, 2012 through present 11% (reduced by 3% after 30 years of service)
Based on a member’s age, length of service, and average salary, retirement allowance benefits can be
up to 80% of the average of the member’s three highest-paid consecutive years of service (if the
member was hired before April 12, 2012) or the average of the five highest-paid consecutive years of
service (if the member was hired thereafter).

3. Teachers employed by the City of Boston are members of the Boston Retirement System.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Overview of Audited Entity
4
Governance
Section 50 of Chapter 7 of the General Laws governs how public employee retirement systems are
overseen and regulated by the Public Employee Retirement Administration Commission (PERAC).
Sections 1 through 5 of Title 840 of the Code of Massachusetts Regulations governs the administrative
procedures, financial operations, recordkeeping, and reports required of public employee retirement
systems.
As part of its oversight, PERAC performs periodic reviews of records of all retirement systems at least
once every three years. PERAC also provides training, as well as legal and technical assistance, to
retirement boards.
MTRS files an annual report with PERAC for each fiscal year (which ends on June 30) on or before
December 31 of the following fiscal year. The report shows MTRS’s assets and liabilities, as well as
statistical information regarding membership, audit findings, the most recent actuarial valuation,4 the
system’s investment portfolio, and any other pertinent information that PERAC deems appropriate.
MTRS’s disbursements for annuities and pensions for July 2017 through June 2019 were as follows.
July 2017–June 2018 July 2018–June 2019
MTRS Disbursements—Gross $ 3,123,875,940 $ 3,235,508,937
Less: Refunds, Transfers, and Reimbursements (187,167,100) (208,386,577)
MTRS Disbursements—Net $ 2,936,708,840 $ 3,027,122,360
The Pension Reserves Investment Management Board manages and invests MTRS member
contributions; these funds are held in a trust fund known as the Pension Reserves Investment Trust
(PRIT). MTRS’s annual reports for fiscal years 2018 and 2019 listed the following investment values as of
June 30, 2018 and June 30, 2019.
As of June 30, 2018 As of June 30, 2019
MTRS Investment in PRIT Capital Fund $ 28,559,010,612 $ 29,318,664,851
MTRS Investment in PRIT Cash Fund 29,048,393 46,828,674
Total MTRS Investments $ 28,588,059,005 $ 29,365,493,525

4. An actuarial valuation is a statement of future values of pension assets and liabilities based on certain assumptions,
including pensioner demographics.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Overview of Audited Entity
5
Retirement benefits that members will eventually receive have two parts: an annuity and a pension. The
annuity consists of contributions deducted during employment; the pension is the difference between
the total retirement allowance specified by law and the amount of a member’s contributions and
related investment earnings.
Massachusetts Teachers’ Retirement Board
In accordance with Section 16 of Chapter 15 of the General Laws, MTRS is administered by an unpaid
seven-member board made up of the following members:
 Commissioner of Education or his/her designee, chairperson
 State Treasurer and Receiver General or his/her designee
 State Auditor or his/her designee
 Governor’s designee, who must be a retired teacher
 two members elected by the active and retired members of MTRS
 one member elected by the other six members of the board
Each member serves a four-year term, except the Commissioner of Education, the State Treasurer, and
the State Auditor, who serve as long as they are in office. According to MTRS’s website,
The Board, which meets at least once a month,
 votes on every disability retirement allowance,
 investigates all claims for accidental and ordinary disabilities,
 establishes the rules and regulations of the agency, and
 oversees the dissemination of services and information to its membership of more than
93,000 active educators and over 66,000 retirees and survivors.
MTRS maintains offices in Charlestown and Springfield to administer and implement its policies.
According to its annual report, as of June 30, 2019 there were 90 permanent full-time and 12 permanent
part-time employees serving MTRS members.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Overview of Audited Entity
6
Retirement Application Process
For eligible members, the retirement process begins with the submission of a retirement application.
MTRS reviews submitted applications and groups them based on how many days before the retirement
dates they were received. If necessary, MTRS Member Services Department counselors meet or
correspond with applicants to ensure that application data are complete. When an application is
complete, a Member Services Department retirement analyst enters data, such as applicant date of
birth, retirement option selection (see Appendix), beneficiary data, salary, years of service, and final
contribution amount, in MTRS’s benefit administration system (which is called MyTRS) and verifies the
application as complete and accurate. Once MTRS has received and verified all required data, certain
applications are subject to a quality review audit and recalculation by a Member Services Department
manager based on specific risk criteria. For instance, such reviews are conducted for applications from
administrators whose pay is not determined by a collective bargaining agreement, members who may
be entitled to benefits from multiple retirement systems, and retirees who are entitled to enhanced
benefits because they have been members of the system for 30 or more years. Next, letters known as
notices of estimated retirement benefits, or first pay letters, are sent to new retirees; they list the
details of retirement, including service dates, average salary (average of the member’s three or five
highest-paid consecutive years of service), and initial payment amounts.
MTRS processes approximately 2,500 retirements annually. The retirement application recommends
planning for retirement at least six months in advance and then filing the application three to four
months before the retirement date. According to MTRS management, applications are processed in 75
to 120 days, and the earlier an application is submitted, the quicker first payments can be made after
the retirement date. Section 13(1)(b) of Chapter 32 of the General Laws states that first payment must
be made “on the last day of the month following the month in which . . . such . . . pension . . . becomes
effective.” Depending on the day of the month when a retirement occurs, payment can be due in 28 to
62 days.
Benefit Payment Adjustments
Each year some retirements are processed before the finalization of information needed to accurately
calculate monthly pension benefits. In many instances, the best information available at processing time
is used to calculate monthly benefits. Primary causes for using this information include the timing of the
finalized monthly deduction amounts that school districts report to MTRS and pending service
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Overview of Audited Entity
7
purchases5 where service times have not yet been verified. MTRS chooses to pay estimated benefits to
retirees when MTRS management deems it necessary to do so; once the necessary information is
finalized, MTRS amends the monthly pension benefit amount and makes adjustments in initial
calculations.
Certain types of retirements are more likely to be adjusted and are routinely reviewed to determine
what adjustments may be required. This can happen because of the complexity of calculations. It occurs
with Option B (annuity protection) retirements, where MTRS reduces the monthly benefit payments by
0.25% until accurate actuarial data are available. It also occurs with dual-membership retirements,
governed by Section 3(8)(c) of Chapter 32 of the General Laws; in these cases, amounts may be due
from other retirement systems but not readily determined. Adjustments to monthly benefits are
processed by the Benefit Adjustment and Finalization Unit (BAFU), a subgroup within the MTRS Finance
and Reporting Department.
As revised or new information for previously processed retirements is received or certain complex
retirements are designated for review, adjustments in MyTRS are assigned to BAFU service
representatives for processing. All adjustments are reviewed by the BAFU senior coordinator for
accuracy and completeness after they are processed.

5. Service purchases are purchases of creditable service time by members for previous employment where annuity
contributions have not been paid. Examples include municipal or Commonwealth employment, non-public school teaching,
out-of-state public school teaching, and repurchased service time that was previously refunded.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
8
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts Teachers’
Retirement System (MTRS) for the period July 1, 2017 through June 30, 2019.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Does MTRS ensure that members receive their first pension payments within the
timeframe established by Section 13(1)(b) of Chapter 32 of the General Laws?
No; see Finding 1
2. Does MTRS ensure that adjustments made to monthly benefit payments, as a result of
changes to wage, service time, or annuity deposit data received after the initial
payment of benefits, are completed accurately in accordance with Section 5(2) of
Chapter 32 of the General Laws?
Yes
3. Does MTRS identify deceased retirees and beneficiaries promptly and ensure that
benefit payments are subsequently adjusted or terminated accurately in accordance
with Section 12(2) of Chapter 32 of the General Laws?
Yes
In addition, Finding 2 discusses monthly benefit payments that we found had been made to a member
whose account contained an invalid Social Security number (SSN).
To achieve our audit objectives, we gained an understanding of MTRS’s internal control environment
related to the objectives by reviewing agency policies and procedures, as well as conducting inquiries
with MTRS’s staff and management. We reviewed and tested the operating effectiveness of internal
controls related to the processing of new retirees’ first pension payments, adjustments to monthly
benefit payments, and retiree and beneficiary deaths during the audit period.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
9
To obtain sufficient, appropriate audit evidence to address our audit objectives, we conducted further
audit testing as follows.
To determine whether MTRS processed first pension payments within the required timeframe, we
performed the following procedures:
 MTRS gave us a list of all 4,907 new retirements with effective dates during our audit period
from its benefit administration system, MyTRS. For each new retiree, we calculated the amount
of time between the effective date of retirement and the date of the first pension payment.
 We split the population of new retirements during our audit period into two categories based
on the number of days between the effective date of retirement and the date of the first
pension payment.
Category Days to Process First Payment Population Percentage of Total
Compliant* 62 days or less 3,689 75.2%
Noncompliant More than 62 days 1,218 24.8%
Total 4,907 100%
* For audit testing purposes, we defined “compliant” as “processed within 62 days.” By statute, the timeframe
for compliance can range from 28 to 62 days depending on the day of the month a retirement is effective, as
outlined in the “Overview of Audited Entity” section of this report under “Retirement Application Process.”
 We selected a statistical, random sample for testing, with a 95% confidence level, a 0% expected
error rate, and a 10% tolerable error rate, of 30 of the 3,689 “compliant” retirements. We
reviewed retirement applications for completeness and the content of first pay letters to
determine whether there was any evidence of common attributes, data trends, or potential best
practices that may have contributed to the favorable processing times.
 We selected a statistical, random sample for testing, with a 95% confidence level, a 0% expected
error rate, and a 10% tolerable error rate, of 30 of the 1,218 “noncompliant” retirements. We
evaluated the retirement application processing timeline for the selected cases to determine
whether there were any underlying reasons for the delays in processing first payments. We
reviewed supporting documentation (such as retirement applications, application checklists,
application receipt acknowledgment letters, salary request and release forms, benefit request
sheets, data and annuity sheets, first pay letters, and MyTRS workflow reports) to identify any
similar circumstances, common causes for delays, or other trends in the data that might have
contributed to the delays.
To determine whether MTRS accurately processed adjustments made to monthly benefit payments, we
performed the following procedures.
 MTRS gave us a list from MyTRS of all 1,068 retirements during the audit period that required
adjustment after the issuance of the first benefit payment. We segmented the population based
on the impact (increase, net zero, or decrease) and dollar value of the adjustments, as follows.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
10
Impact of Adjustments Dollar Range of Adjustments Population Percentage of Total
Increase in Benefit $0.04–$2,451.78 552 51.7%
Net Zero Change in Benefit* $0.00 502 47.0%
Decrease in Benefit $4.58–$1,531.32 14 1.3%
Total 1,068 100%
* Net zero adjustments are adjustments where the change in the monthly pension amount is completely offset by
an equal and opposite adjustment to the monthly annuity amount. The net impact of such changes is $0.
Size of Adjustments Dollar Range of Adjustments Population Percentage of Total
Small $0.01–$9.99 482 45.1%
Medium $10.00–$99.99 364 34.1%
Large $100.00 or more 222 20.8%
Total 1,068 100%
 We selected a statistical, random sample for testing, with a 95% confidence level, a 0% expected
error rate, and a 10% tolerable error rate, of 30 of the 1,068 adjustments. We also randomly
selected a nonstatistical sample of 3 of the 14 adjustments with a decrease in monthly benefit
payments for testing. Finally, we randomly selected a nonstatistical sample of 20 of the 222
adjustments with impacts of $100 or more for testing. For each sample, we determined the
reasons for the adjustments, calculated the lengths of time from dates of first pay to dates of
adjustment, and verified the accuracy of the revised monthly benefit payments.
To determine whether MTRS promptly identified deceased retirees and beneficiaries and accurately
adjusted or terminated benefit payments, we performed the following procedures:
 We obtained a system-generated list of all 3,105 retirees and Option C beneficiaries whose
recorded dates of death occurred during our audit period. We selected a statistical, random
sample, with a 95% confidence level, a 0% expected error rate, and a 10% tolerable error rate,
of 30 deceased benefit recipients and reviewed supporting documentation (such as retirement
applications, death certificates, death notices, obituaries, and correspondence) to determine
whether death certificates were on file; dates of death were promptly and accurately recorded
in MyTRS; and appropriate actions were taken, including accurate and timely adjustments to
benefit payments when necessary.
 From the population of 3,105 recorded deaths, we identified 7 that were not recorded in MyTRS
until six months (180 days) or more after the dates of death. We tested all 7 deaths to
determine whether any underlying reasons or irregularities existed that caused the delays.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
11
 MTRS gave us a list of 3,192 possible deaths6 of MTRS retirees and Option C beneficiaries
received during our audit period from its third-party vendor. We selected a statistical, random
sample of 30 possible deaths, with a 95% confidence level, a 0% expected error rate, and a 10%
tolerable error rate, and reviewed supporting documentation (such as payee lists provided to
the third-party vendor, retirement applications, death certificates, death notices, obituaries, and
correspondence) to determine whether the deaths were of MTRS members or beneficiaries;
whether death certificates were on file; whether dates of death were promptly and accurately
recorded in MyTRS; and whether appropriate actions were taken, including accurate and timely
adjustments to benefit payments when necessary.
We used a combination of nonstatistical and statistical sampling methods for our audit objectives and
did not project the sample results to any of the population.
Data Reliability
We assessed the reliability of the data obtained from MyTRS by interviewing agency officials who were
knowledgeable about the data and testing the data for duplicate records and dates outside our audit
period. We also verified the number of records in each data population (retirements, adjustments, and
deaths) by comparing the total number of records to those in other data sources, such as monthly
summary reports and pension warrants (lists of monthly benefit payments). In addition, we traced
samples of records from each data population to and from original source documents, such as
retirement applications, system-generated workflows, death certificates, death notices, and pension
warrants, for completeness and accuracy. Further, we tested one automated application control related
to the creation of workflows for certain adjustment activities in MyTRS. We also tested certain general
information system controls over MyTRS. This included testing of system access controls related to the
initial granting of access privileges to new hires and the revocation of access privileges for terminated
employees.
As part of our assessment of the reliability of data pertaining to deceased retirees and beneficiaries
(Objective 3), we found that monthly benefit payments were made to a member whose account
contained an invalid SSN; as previously mentioned, this is discussed in Finding 2.
We determined that the information obtained from MyTRS for our audit period was sufficiently reliable
for our audit work.

6. MTRS periodically provides a third-party vendor with a copy of its entire retiree payee file, including designated joint
survivors, to identify member and beneficiary deaths. The vendor cross-matches the information in this file with its death
data to determine whether any MTRS benefit recipients or designated beneficiaries have died. The vendor’s match
produces a file of deceased individuals whose names and Social Security numbers closely match those of individuals in
MTRS’s retiree payee file. MTRS conducts additional research to determine whether the decedents are MTRS payees.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
12
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Massachusetts Teachers’ Retirement System did not always make
initial benefit payments within the mandated timeframe.
The Massachusetts Teachers’ Retirement System (MTRS) was delayed in issuing initial monthly benefit
payments to retirees. Our analysis of the 4,907 retirees whose effective dates of retirement were during
our audit period showed that 24.8% (1,218) did not receive their first monthly benefit payments within
the required timeframe. The delays may have resulted in financial hardship for retirees.
For the 4,907 retirements included in our analysis and testing, retirement processing times ranged from
1 day to 1,064 days,7 as measured from dates of retirement to dates of initial payment. On average, the
first payment took 55 days, which is within the state-mandated 62-day8 timeframe. However, the
detailed substantive testing we performed on a random sample of 30 retirees with instances of
noncompliance supports the processing delays identified by our analytical procedures, which indicated
that the later an application was filed, the higher was the likelihood of noncompliance. Specifically, in
the sample of 30 retirees with instances of noncompliance, we noted that 20 (66.7%) of the applications
were received less than 31 days before the members’ retirement dates or received after the retirement
dates, and 10 (33.3%) of the applications were received 31 to 74 days before the retirement dates.
The detailed substantive testing examined documentation such as Part 1 of retirement applications
(applicant retirement data), internal control sheets (internal checklists), application receipt
acknowledgment letters, Part 2 of retirement applications (service, salary, and contribution data from
employing school districts), support for benefit calculations, notices of estimated retirement benefits,
first pay letters, and MyTRS workflow reports.
Authoritative Guidance
Section 13(1)(b) of Chapter 32 of the Massachusetts General Laws states,
The first . . . full payment [to a retiree] shall be due and payable on the last day of the month
following the month in which falls the date as of which such annuity, pension or retirement
allowance becomes effective. If such effective date is a day other than the last day of the month

7. The range of days is reduced from 1,064 days to 337 if disability retirements are excluded. Disability retirements usually
take longer to process because of the potential for delays related to medical evaluations and lengthy appeal procedures.
8. For audit testing purposes, we defined “compliant” as “processed within 62 days.” By statute, the timeframe for
compliance can range from 28 to 62 days depending on the day of the month a retirement is effective, as outlined in the
“Overview of Audited Entity” section of this report under “Retirement Application Process.”
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
13
in which it falls, a pro rata payment shall be allowed for the period following such date and
ending with such last day.
We believe this excerpt indicates that the first full payment is due and payable on the last day of the
month after the month of retirement. If the retirement date falls on the last day of a month, then
payment is due on the last day of the following month and payable in a minimum of 28 days. If the
retirement date falls on the first day of a month, then payment is due on the last day of the following
month and payable in a maximum of 62 days.
Reasons for Issue
According to MTRS management, the primary factor leading to delays in issuing first benefit payments
relates to the number of days in advance an application is submitted. For all 4,907 retirements during
the audit period, processing time averaged 55 days; however, when we reviewed how far in advance
applications were received, we noted the following.
Days in Advance Applications Received
Number of
Retirements
Average Number of
Days to First Pay
Received 0 to 60 days before retirement date 1,508 97
Received 61 or more days before retirement date 3,399 36
Total Retirements 4,907 55
MTRS management also indicated that delays in the issuance of first benefit payments could be caused
by retirees or employing school districts submitting inaccurate or incomplete information. Management
indicated that when this happens, application processing slows or stops until accurate and complete
information is received. MTRS management added that the agency often needs additional time to
conduct research, verify information, request additional or revised information, and confirm the
accuracy and completeness of the data used to calculate benefits.
Recommendation
MTRS should examine the benefit of redeploying existing resources toward processing retirement
applications that are received 60 days or less before proposed retirement dates in an effort to achieve a
higher percentage of “compliant” retirements.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
14
Auditee’s Response
The MTRS appreciates the Office of the State Auditor’s (OSA) review of our retirement application
processes and accepts the finding that 75% of our retirement applicants are paid within 62 days
of their date of retirement. We continually strive to enhance our members’ retirement experience,
and will work to improve our processes where possible to pay more new retirees within the 62-
day threshold while maintaining accuracy of benefit calculations. . . .
 Since we are the retirement system for teachers, we naturally have a spike in retirement
applications at the end of each school year, when approximately two-thirds of our
retirement applications are received. The staff who process retirement calculations must
be highly trained, so we cannot redeploy other staff or hire temporary staff to address
this seasonal high volume. Instead, as noted in your report, the MTRS retirement
application, educational materials, videos and in-person seminars encourage members to
apply for retirement three months in advance of their intended retirement date, so that
our trained staff can begin working on the high volume of cases in advance, and provide
our members with an accurate and timely benefit inception.
 As you may have noticed during your analysis of our data, over 90% of members who
file their applications three months in advance, receive their first benefit payment in their
first full month of retirement.
 The members who receive their first payment more than 62 days after their retirement
date are typically those who file their applications very close to, or after, their retirement
date. (Pursuant to M.G.L. c. 32, § 10[3], members are allowed to file their retirement
applications as late as 60 days after their effective date of retirement.) We do not share
your assumption that these members suffer a financial hardship. Most teachers receive
their summer pay as a lump sum in June in the year they retire, which carries them for
the months of July and August. As noted in your report, the average number of days to
pay late filers their retroactive benefits is 97 days, so most of those members receive
their first benefit payment in September, with the remainder in October. Finally, any
retirement applicant who expresses financial need as a result of a delay is offered a
pension advance payment, which is recouped from their first retroactive benefit payment.
Auditor’s Reply
The Office of the State Auditor (OSA) acknowledges that retirement application processing times can be
affected by when members submit retirement applications, which MTRS cannot control. However, as
noted above, during our audit period, 24% of all new retirees did not receive their first benefit payments
within the timeframe prescribed by Section 13(1)(b) of Chapter 32 of the General Laws, indicating
significant problems in this area that MTRS must address.
We recommended that MTRS, in an effort to achieve a higher percentage of “compliant” retirements,
examine the benefit of redeploying existing resources toward processing retirement applications that
are received 60 days or less in advance of proposed retirement dates. In its response, MTRS indicates
that it cannot redeploy other personnel because of the extensive training necessary for staff members
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
15
who process retirement calculations. Although we believe our recommendation offered a reasonable
means to minimize this problem, we acknowledge that it is ultimately up to MTRS management to
determine what measures it can and should take to improve this process with its available resources.
In its response, MTRS states that it does not agree with OSA’s assumption that delays in providing initial
pension payments to retirees may have resulted in financial hardship for these retirees. MTRS states
that most teachers receive their summer pay as a lump sum in June in the year they retire, which carries
them for the months of July and August. However, not all teachers retire in June or receive their
summer pay in a lump sum; most school districts allow teachers to choose how they will receive their
summer pay. In any case, MTRS still needs to ensure that teachers receive their initial pension payments
on time, as extended delays in providing initial pension payments can result in financial hardships for
retirees.
MTRS indicates that it offers an advance benefit payment program to help soon-to-be retirees meet
their financial needs while waiting to receive their retirement benefits. However, during our audit, we
analyzed participation in this program and found that only 66 (1.3%) of the 4,907 new retirees in our
population received advances.
Based on its response, MTRS is taking measures to address our concerns on this matter.
2. MTRS made approximately $1,470 of monthly benefit payments to a
member whose account contained an invalid Social Security number.
From August 2018 through May 2021, MTRS paid a total of approximately $1,470 in monthly benefit
payments (approximately $40 per month) under an invalid Social Security number (SSN) to an individual
who was entitled to receive a “minimum distribution”9 payment. As a result, the invalid SSN was
reported to the Department of the Treasury’s Internal Revenue Service (IRS) for tax purposes, and an
incorrect Form 1099-R10 was issued to the member.

9. If state service is terminated but funds are left on account with MTRS, federal and state laws require retirees to take a
mandatory minimum distribution of their retirement account when they turn 72 if they are not collecting a retirement
benefit and are not employed under MTRS.
10. A Form 1099-R is an Internal Revenue Service tax form used to report retirement income received and federal income tax
withheld.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
16
Authoritative Guidance
According to the Internal Revenue Code, annuity and pension payments from MTRS are subject to
federal income taxes. MTRS requires that payees complete a Form W-4P11 for the purpose of
determining tax withholding amounts. A Form W-4P requires a valid SSN.
Reasons for Issue
The incorrect SSN appears to have been entered in MyTRS as a placeholder or dummy SSN until a valid
SSN was received from the benefit recipient. Once the actual SSN was received, the necessary update to
the member’s account with the accurate information was not completed. MTRS management could not
explain why the account was not updated.
On July 30, 2021, MTRS gave us the following response as to why an incorrect SSN appears to have been
entered in MyTRS as a placeholder or dummy SSN:
 During the 1970s and early 1980s, MTRS’s recordkeeping systems were not computerized, and
MTRS did not consistently collect or maintain SSNs. Instead, MTRS assigned a unique number to
each member, stored it on an index card, and wrote it on all paper files pertaining to that
member.
 When MTRS first began using a computer system in the 1980s, SSNs were not required.
However, when MTRS updated to a new system in the 1990s, SSN became a required field.
 MTRS did not have SSNs in its records for members who still had funds on account with MTRS
when it converted to the new system in the 1990s and whose last active employment was in the
1970s. To satisfy the validation requirements of the new system, MTRS chose to enter
placeholder or dummy SSNs for these members, generated by combining an invalid SSN prefix
(998) with each retiree’s MTRS member number.
 When a member with a placeholder SSN, or such a member’s survivor, applies for benefits,
MTRS obtains the beneficiary’s actual SSN, updates the account, pays benefits, and reports the
payment to the IRS. MTRS retains the placeholder record for reference.
Recommendations
1. MTRS should correct the member’s SSN in MyTRS and issue a corrected Form 1099-R to both the
member and the IRS.
2. MTRS should ensure that all benefits are paid and reported to the IRS under the correct SSN.

11. A Form W-4P is an Internal Revenue Service tax form used by individuals who receive pensions, annuities, and certain other
deferred compensation to tell payers what amount of federal income tax to withhold from their payments.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
17
Auditee’s Response
The MTRS agrees with the facts of the case referred to in this finding by the OSA. We have
corrected the member’s SSN in our system and issued corrected 1099-Rs to both the member
and the IRS. Going forward, as a control on this manual process, the MTRS Application and
Database Services department has queried our data and determined that no other disbursements
have been made under an invalid SSN during or after the audit period. In addition, they have
added that query to the list of data and scenarios that are reviewed by our 1099-R preparation
team prior to the annual issuance of 1099-Rs.
Auditor’s Reply
Based on its response, MTRS is taking measures to address our concerns on this matter.
Audit No. 2020-0163-3S Massachusetts Teachers’ Retirement System
Appendix
18
APPENDIX
Retirement Allowance Options
At retirement, state employees choose one of three options that determine how retirement benefits are
paid. If no option is selected, Chapter 32 of the Massachusetts General Laws states that the member
defaults to Option B.
Retirement allowances are paid monthly. The payment amount depends on the option selected. The
option also determines what benefits, if any, will be paid to survivors. The options are as follows:
 Option A: Members receive their full retirement benefit in monthly payments during their
lifetime. All benefit payments cease upon their death, and no benefits are provided for their
survivors.
 Option B: Members receive a lifetime benefit that is approximately 1% to 3% less per month
than Option A. The annuity portion (member contributions) of their benefits is reduced to allow
for a potential benefit for their beneficiaries. Upon a member’s death, surviving beneficiaries
are paid the unexpended balance of the accumulated total contributions.
 Option C: Members receive a lifetime benefit that is approximately 9% to 11% less per month
than Option A. Upon a member’s death, the designated beneficiary is paid a monthly benefit for
the rest of his/her lifetime.

Official Audit Report – Issued October 16, 2025
Massachusetts Teachers’ Retirement System
For the period July 1, 2023 through June 30, 2024
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
October 16, 2025
Jonathan Osimo, Executive Director
Massachusetts Teachers’ Retirement System
500 Rutherford Avenue, Suite 210
Charleston, MA 02129
Dear Executive Director Osimo:
I am pleased to provide to you the results of the enclosed performance audit of the Massachusetts
Teachers’ Retirement System. As is typically the case, this report details the audit objective, scope,
methodology, finding, and recommendations for the audit period, July 1, 2023 through June 30, 2024.
As you know, my audit team discussed the contents of this report with agency managers. This report
reflects those comments.
I appreciate you and all your efforts at the Massachusetts Teachers’ Retirement System. The
cooperation and assistance provided to my staff during the audit went a long way toward a smooth
process. Thank you for encouraging and making available your team. I am available to discuss this audit
if you or your team has any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 3
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 8
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 11
1. The Massachusetts Teachers’ Retirement System’s website was not fully accessible to all website users.11
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
List of Abbreviations
ii
LIST OF ABBREVIATIONS
EOTSS Executive Office of Technology Services and Security
IT information technology
MTRS Massachusetts Teachers’ Retirement System
W3C World Wide Web Consortium
WCAG Web Content Accessibility Guidelines
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts Teachers’
Retirement System (MTRS) for the period July 1, 2023 through June 30, 2024.
The purpose of our audit was to determine whether MTRS’s website adhered to the World Wide Web
Consortium’s Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard accessibility,
navigation accessibility, language accessibility, error identification, and color accessibility. Adherence to WCAG
helps ensure that all users, regardless of ability, can access the content and functions of MTRS’s website.
Below is a summary of our finding, the effect of that finding, and our recommendations, with hyperlinks
to each page listed.
Finding 1
Page 11
MTRS’s website was not fully accessible to all website users.
Effect Noncompliance with WCAG 2.1 reduces accessibility for individuals with disabilities and
limits equitable access to MTRS information and services. Specific risks include the
following:
• A lack of keyboard navigability means that users with mobility impairments cannot
access certain content.
• Because MTRS’s website does not make the first focusable control on a webpage a
hyperlink that skips to the main content of the webpage, users who navigate
sequentially through webpages are forced to navigate through repeated content
every time a webpage loads. This increases the time needed to reach the desired
information on a webpage and may, for users who have motor impairments, make
navigating content cumbersome or difficult.
• Hyperlinks that do not have a 3:1 contrast ratio or a secondary identifiable
component may be imperceptible to users and will prevent them from interacting
with MTRS content.
• If MTRS’s website cannot reflow at 400%, some users will not be able to read
content if they zoom in to alleviate a visual impairment.
• Broken or faulty hyperlinks limit users from having access to critical information and
key online services offered by MTRS. Broken or faulty hyperlinks also increase the
likelihood that users may either access outdated or incorrect information or be
directed to webpages that no longer exist.
• Improper use of headings and labels makes it difficult for users to navigate and read
the website. Additionally, improper use of headers and labels negatively impacts
users who rely on screen readers to navigate the website.
• Entry fields that are improperly labeled or that do not provide users with warnings
about input errors or omissions prevent users from accessing MTRS content.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Executive Summary
2
Recommendations
Page 14
1. MTRS should work with its vendor to resolve the issues directly identified in this report.
2. MTRS should select a new theme for its website with more accessible design elements.
3. MTRS should train staff members on website accessibility requirements and provide
staff members with third-party website accessibility tools to monitor WCAG
compliance.
4. MTRS should implement preventative controls to ensure that content on its website is
posted in a WCAG-compliant manner.
5. MTRS should ensure that its third-party vendor monitors the website for instances of
WCAG noncompliance, or MTRS should acquire tools to monitor WCAG compliance on
its own.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Overview of Audited Entity
3
OVERVIEW OF AUDITED ENTITY
The Massachusetts Teachers’ Retirement System (MTRS) was established on July 1, 1914. According to its
website, MTRS, which is the largest of the Commonwealth’s 104 contributory retirement systems,
provides retirement, disability, and survivor benefits to Massachusetts teachers, administrators, and their
families.
Chapter 32 of the Massachusetts General Laws establishes the system’s benefits, contribution
requirements, and accounting structure. Teachers and administrators in Massachusetts public schools
(except those employed by the City of Boston), educational collaboratives, and charter schools are eligible
for membership.
During fiscal year 2024, MTRS received a funding appropriation of $2,352,500,000 and, as of January 1,
2024, managed assets with a market value of $38,228,453,000. As of June 30, 2024, MTRS had a total
pension liability of $65,779,000,000 and a net pension liability of $25,356,461,000.
As of December 2023, MTRS had 98 full-time and 17 part-time employees, and it has offices in
Charlestown and Springfield.
MTRS’s website provides several utilities to its users. It provides online enrollments, information about
benefits, the ability to update member information, access to funds, and general information about its
various programs.
Massachusetts Requirements for Accessible Websites
In 1999, the World Wide Web Consortium (W3C), an international nongovernmental organization
responsible for internet standards, published the Web Content Accessibility Guidelines (WCAG) 1.0 to
provide guidance on how to make web content more accessible to people with disabilities.
In 2005, the Massachusetts Office of Information Technology,1 with the participation of state government
webpage developers, including developers with disabilities, created the Enterprise Web Accessibility
Standards. These standards required all executive branch state agencies to follow the guidelines in Section
508 of the Rehabilitation Act amendments of 1998. These amendments went into effect in 2001 and
1. The Massachusetts Office of Information Technology became the Executive Office of Technology Services and Security in
2017.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Overview of Audited Entity
4
established precise technical requirements to which electronic and information technology (IT) products
must adhere. This technology includes, but is not limited to, products such as software, websites,
multimedia products, and certain physical products, such as standalone terminals.
In 2008, W3C published WCAG 2.0. In 2014, the Massachusetts Office of Information Technology added a
reference to WCAG 2.0 in its Enterprise Information Technology Accessibility Standards.
In 2017, the Executive Office of Technology Services and Security (EOTSS) was designated as the
Commonwealth’s lead IT organization for executive branch state agencies. EOTSS is responsible for the
development and maintenance of the Enterprise Information Technology Accessibility Standards2 and the
implementation of state and federal laws and regulations related to accessibility. As the principal
executive agency responsible for coordinating the Commonwealth’s IT accessibility compliance efforts,
EOTSS supervises executive branch state agencies in their efforts to meet the Commonwealth’s
accessibility requirements.
In 2018, W3C published WCAG 2.1, which built on WCAG 2.0 to improve web accessibility on mobile
devices and to further improve web accessibility for people with visual impairments and cognitive
disabilities. EOTSS published the Enterprise Information Technology Accessibility Policy in 2021 to meet
Levels A and AA of WCAG 2.1.
2. EOTSS has since changed the titles and numbers of at least some of its policies and standards between the end of the audit
period and the publication of this report. In this report, we reference the titles and numbers of EOTSS’s policies and/or
standards as they were during the audit period (unless stated otherwise).
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Overview of Audited Entity
5
Timeline of the Adoption of Website Accessibility Standards by the Federal
Government and Massachusetts
Executive branch state agencies, such as MTRS, must comply with EOTSS’s policies and standards when
using an EOTSS web domain,3 as established by EOTSS’s Website Domain Policy. Part of this policy states
that any government organization using an EOTSS web domain must comply with EOTSS’s Web Design
Guidelines, which were published in 2020 and were based on the federal 21st Century Integrated Digital
Experience Act. This law helps state government agencies evaluate their website design and
implementation decisions to meet state accessibility requirements.
3. EOTSS web domains, according to its Website Domain Policy, include Mass.gov, Massachusetts.gov, Ma.gov, State.ma.us,
related subdomains (e.g., example.mass.gov), and all domains that EOTSS owns and manages.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Overview of Audited Entity
6
Web Accessibility
Government websites are an important way for the general public to access government information and
services. Deloitte’s4 2023 Digital Citizen Survey found that 55% of respondents preferred to interact with
their state government services through a website instead of face-to-face interaction or a call center.
Commonwealth of Massachusetts websites have millions of webpage views each month.
However, people do not interact with the internet uniformly. The federal government and
nongovernmental organizations have established web accessibility standards intended to make websites
more accessible to people with disabilities, such as visual impairments, hearing impairments, and others.
The impact of these standards can be significant, as the federal Centers for Disease Control and Prevention
estimates that 1,488,012 adults (26% of the adult population) in Massachusetts have a disability, as of
2022.5 Among the estimated 26% of the adult population, 14% reported having serious difficulty with
cognition, 10% reported having serious difficulty with mobility, 6% reported having deafness or serious
difficulty hearing, and 5% reported having blindness or serious difficulty seeing (even when wearing
glasses).6 Examples of web accessibility measures include, but are not limited to, having captioning on
videos to help people with difficulty hearing understand the contents of the video, having form fields
describe what data needs to be input into them to help people who have cognitive difficulties, and
ensuring that people can interact with a webpage using keyboard commands alone to help people who
have difficulty with mobility.
How People with Disabilities Use the Web
According to W3C, people with disabilities use assistive technologies and adaptive strategies specific to
their needs to navigate web content. Examples of assistive technologies include screen readers, which
read webpages aloud for people who cannot read text; screen magnifiers for individuals with low vision;
and voice recognition software for people who cannot (or do not) use a keyboard or mouse. Adaptive
4. Deloitte is an international company that provides tax, accounting, and audit services to businesses and government agencies.
5. This data is collected from surveys conducted using both landline telephones and cellular telephones, and all responses are
self-reported.
6. The percentages do not add up to 26%, as estimated by the federal Centers for Disease Control, because of overlapping data.
The self-reported survey allows individuals to report having multiple disabilities.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Overview of Audited Entity
7
strategies refer to techniques that people with disabilities employ to enhance their web interactions.7
These strategies might involve increasing text size, adjusting mouse speed, or enabling captions.
To make web content accessible to people with disabilities, developers must ensure that various
components of web development and interaction work together. This includes text, images, and structural
code, users’ browsers and media players, and various assistive technologies.
Accessibility Features of a Website8
7. Web interaction refers to the various actions that users can take while navigating and using the internet. It encompasses a
wide range of online activities, including, but not limited to, clicking on hyperlinks, submitting forms, posting comments on
webpages, and engaging with web content and services in other forms.
8. We resized this webpage to fit in this audit report. To see the unaltered webpage, visit https://mtrs.state.ma.us/about/#generalinformation.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
8
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Massachusetts Teachers’
Retirement System (MTRS) for the period July 1, 2023 through June 30, 2024.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We
believe that the evidence obtained provides a reasonable basis for our finding and conclusion based on
our audit objective.
Below is our audit objective, indicating the question we intended our audit to answer, the conclusion we
reached regarding our objective, and where our objective is discussed in the audit finding.
Objective Conclusion
1. Did MTRS’s website (mtrs.state.ma.us) comply with the Executive Office of Technology
Services and Security’s Enterprise Information Technology Accessibility Policy and the
Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard
accessibility, navigation accessibility, language accessibility, error identification, and
color accessibility?
No; see Finding 1
To accomplish our audit objective, we gained an understanding of the MTRS internal control environment
relevant to our objective by reviewing applicable policies and procedures and by interviewing MTRS staff
members and management. In addition, to obtain sufficient, appropriate evidence to address our audit
objective, we performed the procedures described below.
Web Accessibility
To determine whether MTRS’s website adhered to WCAG 2.1, for user accessibility, keyboard accessibility,
navigation accessibility, language accessibility, error identification, and color accessibility during the audit
period, we performed the following procedures for a random, nonstatistical9 sample of 50 webpages from
a population of 511 MTRS webpages:
9. Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items
are not similar enough, or there are specific items in the population that the auditors want to review.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
9
User Accessibility
• We determined whether content on each webpage could be viewed in both portrait and
landscape modes.
• We determined whether content on each webpage was undamaged and remained readable
when zoomed in to both 200% and 400%.
Keyboard Accessibility
• We determined whether all elements10 on each webpage could be navigated using only
keyboard commands.
• We determined whether any elements on each webpage prevented a user from moving to a
different element when using only keyboard commands to navigate the webpage in question.
• We determined whether the first focusable control11 on each webpage was a hyperlink that
would redirect users to the main content of the webpage.
Navigation Accessibility
• We determined whether each webpage contained a title that was relevant to the webpage’s
content.
• We determined whether there was a search function present to help users locate content
across the whole website.
• We determined whether hyperlinks correctly navigated to the intended webpages.
• We determined whether headings within webpages related to the content of the section
below the header.
Language Accessibility
• We determined whether any video content found on each webpage had all important sounds
and dialogue captioned.
• We determined whether the words that appeared on each webpage matched the language
attribute12 to which the webpage in question was set.
• We determined whether any webpage sections that contained language differing from that
to which the webpage was set contained their own specified language attribute.
10. An element is a part of a webpage that contains data, text, or an image.
11. The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard. If the
first focusable control also redirects users to the main content of a webpage, then it is known as a bypass block or a skip link.
12. A language attribute (also known as a language tag) identifies the native language of the content on the webpage or PDF
(e.g., a webpage in English should have an EN language attribute). The language attribute is listed in the webpage’s or PDF’s
properties. This, among other things, is used to help screen readers use the correct pronunciation for words.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Audit Objectives, Scope, and Methodology
10
Error Identification
• We determined whether mandatory form fields alerted users if they left these fields blank.
• We determined, for form fields that required a limited set of input values, whether users were
alerted if invalid values were entered into these types of fields.
• We determined whether there were labels for any elements that required user input. We also
determined whether these labels were programmed correctly.
• We determined whether examples were presented to assist users in correcting mistakes (for
example, a warning when entering a letter in a field meant for numbers).
Color Accessibility
• We determined whether there was at least a 3:1 contrast in color and additional visual cues
to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other
visual impairments.
We used nonstatistical sampling methods for testing and therefore did not project the results of our
testing to any corresponding populations.
For our objective, we found certain issues during our testing of the accessibility of MTRS’s website. See
Finding 1 for more information.
Data Reliability Assessment
To determine the reliability of the URL list that we received from MTRS management, we interviewed
knowledgeable MTRS staff members and checked that certain variable formats (e.g., dates, unique identifiers,
and abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the
URL list: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, or absent records),
and duplicate records. We also ensured that all values in the dataset corresponded with expected values.
We selected a random sample of 20 URLs from the URL list and traced each to the corresponding
webpages on MTRS’s website, checking that each URL and webpage title from the URL list matched the
information on the MTRS website. We also selected a random sample of 20 URLs from MTRS’s website
and traced the URL and webpage title to the URL list to ensure that there was a complete and accurate
population of URLs on the URL list.
Based on the results of the data reliability procedures described above, we determined that the URL list
was sufficiently reliable for the purposes of our audit.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
11
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The Massachusetts Teachers’ Retirement System’s website was not fully
accessible to all website users.
The Massachusetts Teachers’ Retirement System’s (MTRS’s) website was not fully accessible to all website
users. We reviewed a sample of 50 webpages and found that none of these webpages were compliant
with Web Content Accessibility Guidelines (WCAG) 2.1. We determined the following issues within our
sample:
• Of these 50 webpages, all 50 had content that could not be navigated to via a keyboard;
• Of these 50 webpages, all 50 did not have the first focusable control be a hyperlink that skips to
the main content of the webpage;
• Of these 50 webpages, all 50 had hyperlinks that did not have any secondary identifiable
component to distinguish themselves;
• Of these 50 webpages, 24 contained content that was illegible when zoomed in to 400%;
• Of these 50 webpages, 5 had broken hyperlinks;
• Of these 50 webpages, 5 misused headers;
• Of these 50 webpages, 1 did not provide users with warnings when they left entry fields blank;
and
• Of these 50 webpages, 1 did not properly label input fields.
Noncompliance with WCAG 2.1 reduces accessibility for individuals with disabilities and limits equitable
access to MTRS information and services. Specific risks include the following:
• A lack of keyboard navigability means that users with mobility impairments cannot access certain
content.
• Because MTRS’s website does not make the first focusable control on a webpage a hyperlink that
skips to the main content of the webpage, users who navigate sequentially through webpages are
forced to navigate through repeated content every time a webpage loads. This increases the time
needed to reach the desired information on a webpage and may, for users who have motor
impairments, make navigating content cumbersome or difficult.
• Hyperlinks that do not have a 3:1 contrast ratio or a secondary identifiable component may be
imperceptible to users and will prevent them from interacting with MTRS content.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
12
• If MTRS’s website cannot reflow13 at 400%, some users will not be able to read content if they
zoom in to alleviate a visual impairment.
• Broken or faulty hyperlinks limit users from having access to critical information and key online
services offered by MTRS. Broken or faulty hyperlinks also increase the likelihood that users may
either access outdated or incorrect information or be directed to webpages that no longer exist.
• Improper use of headings and labels makes it difficult for users to navigate and read the website.
Additionally, improper use of headers and labels negatively impacts users who rely on screen
readers to navigate the website.
• Entry fields that are improperly labeled or that do not provide users with warnings about input
errors or omissions prevent users from accessing MTRS content.
Authoritative Guidance
The World Wide Web Consortium’s WCAG 2.1 states,
Success Criterion 1.4.1 Use of Color
(Level A)
Color is not used as the only visual means of conveying information, indicating an action, prompting
a response, or distinguishing a visual element. . . .
Success Criterion 1.4.10 Reflow
(Level AA)
Content can be presented without loss of information or functionality, and without requiring
scrolling in two dimensions for:
• Vertical scrolling content at a width equivalent to 320 [cascading style sheet (CSS)]
pixels;
• Horizontal scrolling content at a height equivalent to 256 CSS pixels.
Except for parts of the content which require two-dimensional layout for usage or meaning. . . .
Success Criterion 2.1.1 Keyboard
(Level A)
All functionality of the content is operable through a keyboard interface without requiring specific
timings for individual keystrokes, except where the underlying function requires input that depends
on the path of the user’s movement and not just the endpoints. . . .
13. Reflow is defined as the rearrangement of content when type size, line length, spacing, etc. changes.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
13
Success Criterion 2.4.1 Bypass Blocks
(Level A)
A mechanism is available to bypass blocks of content that are repeated on multiple web pages. . . .
Success Criterion 2.4.5 Multiple Ways
(Level AA)
More than one way is available to locate a web page within a set of web pages except where the
Webpage is the result of, or a step in, a process.
Success Criterion 2.4.6 Headings and Labels
(Level AA)
Headings and labels describe topic or purpose. . . .
Success Criterion 3.3.1 Error Identification
(Level A)
If an input error is automatically detected, the item that is in error is identified and the error is
described to the user in text.
Success Criterion 3.3.2 Labels or Instructions
(Level A)
Labels or instructions are provided when content requires user input.
Reasons for Issue
MTRS management provided us with the following reasons for the findings identified in this audit:
• MTRS management stated that they chose a theme14 for their website that created accessibility
challenges. MTRS was unable to alter website characteristics like text and hyperlink color and
continued to create webpages using this problematic theme.
• MTRS management stated that, in 2017, MTRS made the decision to host its own website and
contract with a third-party vendor to create it. MTRS management stated that the original thirdparty vendor contracted to create its website did not have the required expertise to create an
accessible website. Additionally, MTRS stated that subsequent contracts for website hosting and
maintenance services did not include provisions for assessing or improving website accessibility.
14. A website’s theme is the stylistic framework used to design the front-end appearance of the website for a user. It includes a
website’s colors, fonts, headers, footers, text boxes, layout, and the way graphics are displayed. Web content creators and
developers use themes so that webpages across a website appear uniform and can be created and launched faster.
Audit No. 2025-0163-3I Massachusetts Teachers’ Retirement System
Detailed Audit Findings with Auditee’s Response
14
• MTRS management stated that they did not provide website accessibility training to MTRS staff
members, and that those staff members were unable to identify accessibility shortcomings on the
website.
• MTRS management stated that their content management system15 does not have preventative
controls in place to ensure that content is posted in a WCAG-compliant fashion and that they do
not perform periodic accessibility reviews.
Recommendations
MTRS should work with its vendor to resolve the issues directly identified in this report.
MTRS should select a new theme for its website with more accessible design elements.
MTRS should train staff members on website accessibility requirements and provide staff members
with third-party website accessibility tools to monitor WCAG compliance.
MTRS should implement preventative controls to ensure that content on its website is posted in a
WCAG-compliant manner.
MTRS should ensure that its third-party vendor monitors the website for instances of WCAG
noncompliance, or MTRS should acquire tools to monitor WCAG compliance on its own.
Auditee’s Response
The MTRS acknowledges the audit findings and agrees that our website was not fully compliant
with the Web Content Accessibility Guidelines (WCAG) 2.1 at the time of review.
We recognize the importance of ensuring equitable access for all website visitors, including
individuals with disabilities, and we take these findings very seriously. Corrective actions are already
underway. These include immediately addressing critical issues such as broken links and mislabeled
input fields, improving keyboard navigability, and correcting header structures.
Most significantly, we plan to work with our website hosting vendor to implement a modern, fully
accessible WordPress theme. WordPress themes determine much of the website’s underlying code,
including accessibility functions. Selecting an “accessibility-ready” theme will provide a strong
foundation for achieving full compliance. This initiative, combined with staff training and regular
scheduled third-party accessibility scans, will help ensure ongoing compliance and equitable access
for all users.
Auditor’s Reply
Based on its response, MTRS has taken measures to address our concerns regarding this matter. As part
of our post-audit review process, we will follow up on this matter in approximately six months.
15. Organizations use content management systems to create, publish, and modify the content on their website.

Official Audit Report – Issued September 28, 2020
Office of the Comptroller of the Commonwealth
For the period July 1, 2017 through June 30, 2019
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
cerely,
anne M. Bump
September 28, 2020
Mr. William McNamara, Comptroller of the Commonwealth
Office of the Comptroller of the Commonwealth
One Ashburton Place, Ninth Floor
Boston, MA 02108
Dear Comptroller McNamara:
I am pleased to provide this performance audit of the Office of the Comptroller of the Commonwealth.
This report details the audit objectives, scope, and methodology for the audit period, July 1, 2017
through June 30, 2019. My audit staff discussed the contents of this report with management of the
agency.
I would also like to express my appreciation to the Office of the Comptroller of the Commonwealth for
the cooperation and assistance provided to my staff during the audit.
Sin
Suz
Auditor of the Commonwealth
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 6
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
List of Abbreviations
ii
LIST OF ABBREVIATIONS
CFO chief financial officer
CMR Code of Massachusetts Regulations
CTR Office of the Comptroller of the Commonwealth
DRA data reliability assessment
MMARS Massachusetts Management Accounting and Reporting System
OSA Office of the State Auditor
SSTA Self-Service Time and Attendance
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Office of the Comptroller of the
Commonwealth (CTR) for the period July 1, 2017 through June 30, 2019. The purpose of our audit was to
determine whether CTR had administered its employee payroll in accordance with the six-day
timeframe established in Section 148 of Chapter 149 of the General Laws and its employee expense
reimbursements in accordance with its Employee Benefits & Policies Manual. We also determined
whether it had administered its non-payroll expenditures in accordance with its Operations Guide and
Section 4.03(1)(e) of Title 815 of the Code of Massachusetts Regulations. This audit was performed at
the request of the Comptroller who had been appointed February 18, 2019, Andrew Maylor.
Comptroller Maylor resigned effective February 21, 2020. Subsequently, the Governor appointed
William McNamara to the position of Comptroller effective February 21, 2020.
Our audit revealed no significant instances of noncompliance by CTR that must be reported under
generally accepted government auditing standards.
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Office of the Comptroller of the Commonwealth (CTR) is an independent agency established by
Section 1 of Chapter 7A of the Massachusetts General Laws. The Comptroller is the administrative and
executive head of CTR and is appointed by the Governor for a term that runs concurrently with the
Governor’s.
Section 2 of Chapter 7A establishes an advisory board to the Comptroller:1
There shall be an advisory board to the comptroller which shall consist of the attorney general,
the treasurer, the secretary of administration and finance who shall be the chairman, the auditor,
the court administrator of the trial court, and two persons who have experience in accounting,
management, or public finance who shall be appointed by the governor. . . .
Said advisory board shall provide advice and counsel to the comptroller in the performance of his
duties. The advisory board shall be responsible for reviewing any rules or regulations
promulgated by the comptroller prior to their implementation. The advisory board shall also
review prior to publication the annual financial report of the commonwealth published by the
comptroller.
CTR’s website states,
We promote accountability, integrity, and clarity in Commonwealth business, fiscal, and
administrative enterprises. . . .
The Comptroller oversees the Commonwealth’s . . . major audit functions promoting security,
transparency, accountability, and service delivery across all branches of state government. . . .
We strive to be a model for good government and to protect public resources by mitigating the
risk of fraud, waste, and abuse while promoting transparency.
CTR oversees more than $60 billion in funding from governmental and other sources and 12 million
business transactions annually. It received direct appropriations of $8,834,952 and $9,044,996 for fiscal
years 2018 and 2019, respectively.2 Its offices are located at 1 Ashburton Place in Boston.

1. Generally accepted government auditing standards require that organizations be free from organizational impairments to
independence with respect to the entities they audit. Pursuant to Section 1 of Chapter 7A of the General Laws, State
Auditor Suzanne M. Bump serves on the seven-member advisory board to the Comptroller. This disclosure is made for
informational purposes only, and this circumstance did not interfere with our ability to perform our audit work and report
its results impartially.
2. These amounts represent appropriations for account 1000-0001, which funds the general administration of CTR, including
management of accounting, payroll, related financial systems, and annual financial reports.
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Overview of Audited Entity
3
Employee Payroll and Expense Reimbursement Processing
CTR employees use the Self-Service Time and Attendance (SSTA) system to report hours worked and are
paid six days later, on the Friday after the end of the biweekly payroll period. SSTA is a component of
the Human Resources Compensation Management System. This system captures payroll-related
accounting entries, which are recorded in the Massachusetts Management Accounting and Reporting
System (MMARS). Employees can also submit reimbursement requests for allowable job-related
expenses by completing an online reimbursement form. The form is submitted along with a Portable
Document Format file containing required supporting documentation (e.g., invoice, receipt, and
mileage).
Employees log in to SSTA with their employee identification numbers and unique passwords and enter
weekly hours worked by the required due date. Supervisors are responsible for reviewing and approving
time submitted for further processing by payroll. Additionally, immediate supervisors review employee
expense reimbursement requests, along with accompanying invoices and payment receipts, for
compliance with employee benefit policies. Payroll and reimbursement records are maintained in SSTA.
At the beginning of each week, a group of CTR Operations Department employees known as the Payroll
Administration Group runs preliminary management reports to identify missing or potentially inaccurate
time and expense reimbursement submissions. For submissions identified as missing or inaccurate, the
Payroll Administration Group contacts the employees and their supervisors to correct and resubmit the
information. On the Monday after the biweekly payroll period ends, preliminary management reports
are run again and provided to the CTR chief financial officer (CFO) to review. The CFO reviews the
reports and initials and dates them to indicate his/her approval. On Tuesday, the Payroll Administration
Group begins its preliminary payroll certification, compares the amounts reported for each employee in
the preceding week’s payroll records, and notes special circumstances (e.g., retirements or cost-of-living
wage adjustments). On Wednesday, the Payroll Department forwards its final payroll certification to the
CFO for approval. CTR employees are compensated on Friday via electronic funds transfer or,
occasionally, printed checks.
If an error occurs during payroll processing that cannot be resolved before the payroll is finalized, an
employee can be compensated using funds from a CTR advance checking account, DynaCash. CTR’s CFO
can use these funds by following policies from CTR and the State Treasurer that are in place for this
purpose.
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Overview of Audited Entity
4
Non-payroll Expenditure Processing
The purchase of goods and services is administered though the MMARS accounts payable module. The
process is initiated by the creation of a purchase order,3 which may be issued to preapproved
Commonwealth vendors or to vendors that have been chosen at the end of a competitive bidding
process. Approval for all purchases must be obtained in writing from the Comptroller, the First Deputy
Comptroller, CTR’s CFO, or another person with the requisite signatory authority. Once approved,
purchase orders for goods and services are sent to a CTR Operations Department employee, who enters
encumbrances4 for the purchases in MMARS and sets purchase orders’ status as “held.” The CFO or a
designee performs another review to process each purchase order and set its status as “final.” Vendors
send invoices to CTR for payment as goods and services are provided. Most invoices are received
through email, although some vendors mail hardcopy invoices. CTR stamps hardcopy invoices with the
date received. According to CTR’s Operations Guide,
Absent a prompt payment discount or avoidance of late penalties, or other factors that must be
contemporaneously documented, payments must be scheduled 45 days from the date the invoice
was received or the date performance was completed, whichever is later.
CTR’s “Prompt Payment Discounts” policy states,
The Commonwealth's goal is to pay its bills through Electronic Funds Transfer (EFT) while
maximizing the use of prompt payment discounts (PPD). EFT payments are required for all
contracts and any Contractor seeking accelerated payments must provide a PPD.
Electronic Funds Transfer (EFT) forms and Prompt Payment Discount (PPD) terms should be
initiated whenever a Contract is procured, and for current Contracts, whenever an amendment or
renewal is negotiated. During procurements Bidders submit Responses identifying PPDs with the
assumption that Departments will pay their bills more quickly in order to receive the PPD from
the total price of an invoice. Having prompt payment discount options in contracts is
advantageous to both Contractors and purchasing Departments.
Contractors benefit from PPD by increased, usable cash flow as a result of fast and efficient
payments for commodities delivered or services rendered. Contractors must accept payments
through Electronic Funds Transfer (EFT) which further maximizes this benefit by ensuring that
funds are paid directly to the Contractor’s designated accounts, eliminating the impact of check
clearance policies and traditional mail lead time or delays.

3. Purchase orders are documents that buyers use to document the intended purchase of goods and services from external
suppliers, indicating specifications including types, quantities, and agreed-upon prices.
4. Encumbrance is the process of reserving funds for specific goods or services to be purchased, ensuring the availability of
funds for the expenditure.
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Overview of Audited Entity
5
The Commonwealth benefits because the Department’s cost for products and services are
reduced by taking advantage of the Prompt Payment Discount (PPD) and EFT and the
Commonwealth saves the expense of processing, printing and mailing checks.
The designated CTR Operations Department employee distributes copies of invoices to departments for
verification that the vendors have provided the goods or services. The departments review the invoices
and acknowledge receipt by signing and dating the invoices and any related documentation (e.g.,
packing slips). They then send the invoices and any related documentation to a CTR Operations
Department employee who is assigned accounts payable responsibilities. After receiving a signed and
dated invoice, this employee processes the payment transaction in MMARS and sets the payment status
as “pending.” The CFO reviews all documentation and verifies that the goods and services have been
provided and that all associated transactions are accurately recorded in MMARS. When the CFO is
satisfied, s/he changes the payment status from “pending” to “final” to process the payment.
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Audit Objectives, Scope, and Methodology
6
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor (OSA) has conducted a performance audit of certain activities of the Office of the Comptroller of
the Commonwealth (CTR) for the period July 1, 2017 through June 30, 2019.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer and the
conclusion we reached regarding each objective.
Objective Conclusion
1. Does CTR administer its employee payroll in accordance with the six-day timeframe
established in Section 148 of Chapter 149 of the General Laws and its employee
expense reimbursements in accordance with its Employee Benefits & Policies Manual?
Yes
2. Does CTR administer its non-payroll expenditures in accordance with its Operations
Guide and Section 4.03(1)(e) of Title 815 of the Code of Massachusetts Regulations
(CMR)?
Yes
To achieve our objectives, we gained an understanding of CTR’s internal control environment related to
our audit objectives by reviewing agency policies and procedures, as well as conducting inquiries with
CTR’s staff and management. We reviewed and tested the operating effectiveness of internal controls
regarding CTR’s employee payroll, expense reimbursements, and non-payroll expenditures.
Additionally, we performed the procedures described below.
Employee Payroll and Expense Reimbursements
To determine whether CTR paid employees within six days of the end of the pay period in which wages
were earned, as required by Section 148 of Chapter 149 of the General Laws, we obtained a list of all
11,516 payroll transactions recorded in the Massachusetts Management Accounting and Reporting
System (MMARS) during the audit period, which totaled $14,332,851. For each transaction, using
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Audit Objectives, Scope, and Methodology
7
electronic spreadsheet functionality, we calculated the number of days from the end of the pay period
to the biweekly paycheck date to determine whether the paychecks were issued within the required sixday timeframe.
To determine whether CTR reimbursed its employees for allowable employee expenses in accordance
with Section 148 of Chapter 149 of the General Laws and CTR’s Employee Benefits & Policies Manual, we
obtained a list of all 214 weekly payroll expense reimbursement transactions recorded in MMARS during
the audit period, which totaled $22,548. We selected a judgmental, nonstatistical sample of 35
transactions and reviewed each one for receipts for employee out-of-pocket expenses, employee
affidavits of expenses (e.g., public transportation fares), mileage logs or destination calculations, and
travel authorization forms.
Non-payroll Expenditures
To determine whether CTR paid its vendors in accordance with its Operations Guide and 815 CMR
4.03(1)(e), we obtained a list of all 779 vendor payment transactions recorded in MMARS during the
audit period, which totaled $3,315,688. We selected a judgmental, nonstatistical sample of 40 (totaling
$887,715) of the 779 transactions and reviewed the vendor invoice for each one to verify that the
correct vendor was paid the correct amount and that the expenditure was appropriately classified.
To determine whether CTR paid its vendors within 45 days for all 779 of these transactions, we used
electronic spreadsheet functionality to calculate the number of days from the date CTR received the
invoice or the date it received the goods or services, whichever occurred later, to the date of payment.
To determine whether CTR used prompt payment discounts when available, we reperformed CTR’s
query that identifies full discounts taken, partial discounts taken, and missed discounts. We recalculated
full discounts taken, partial discounts taken, and missed discounts for all 779 of the transactions
discussed above.
When sampling, we used a nonstatistical sampling method, whose results we could not project to the
entire population.
Data Reliability
In 2018, OSA performed a data reliability assessment (DRA) for MMARS for the period April 1, 2017
through March 31, 2018. The DRA focused on reviewing selected system controls, including access,
Audit No. 2020-0028-3S Office of the Comptroller of the Commonwealth
Audit Objectives, Scope, and Methodology
8
security awareness, audit and accountability, configuration management, identification and
authentication, and personnel security.
During this audit, we made inquiries with CTR management to identify and assess changes that occurred
after our 2018 DRA. We requested confirmation that policies or procedures had not changed for areas
where no issues were noted. For areas where issues were noted, we reviewed CTR’s progress on
resolving the issues.
To determine the reliability of CTR’s list of MMARS payroll and employee expense reimbursement
transactions, we interviewed agency officials who were knowledgeable about the data and reconciled
the aggregate amount of CTR’s payroll and employee expense reimbursements during our audit period
to the Governor’s budget website. We determined that the list of payroll and employee expense
reimbursement transactions was sufficiently reliable for the purpose of this audit.
To determine the reliability of CTR’s list of MMARS vendor payment transactions, we interviewed
agency officials who were knowledgeable about the data and reconciled the aggregate amount of CTR’s
vendor payment transactions during our audit period to the Governor’s budget website. We determined
that the list of vendor payment transactions was sufficiently reliable for the purpose of this audit.
Conclusion
Our audit revealed no significant instances of noncompliance that must be reported under generally
accepted government auditing standards.

Official Audit Report – Issued September 23, 2021
Office of the Governor
For the period July 1, 2018 through June 30, 2020
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
cerely,
anne M. Bump
September 23, 2021
The Honorable Governor Charles D. Baker
Office of the Governor
State House, Room 280
Boston, MA 02133
Dear Governor Baker:
I am pleased to provide this performance audit of the Office of the Governor. This report details the
audit objectives, scope, methodology, findings, and recommendations for the audit period, July 1, 2018
through June 30, 2020. My audit staff discussed the contents of this report with management in your
office, whose comments are reflected in this report.
I would also like to express my appreciation to the Office of the Governor for the cooperation and
assistance provided to my staff during the audit.
Sin
Suz
Auditor of the Commonwealth
Audit No. 2021-0933-3S Office of the Governor
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 4
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE ..................................................................................... 7
1. Some state boards and commissions had vacancies that the Office of the Governor did not fill
immediately when members’ terms ended.............................................................................................. 7
OTHER MATTERS...............................................................................................................................................14
Audit No. 2021-0933-3S Office of the Governor
List of Abbreviations
ii
LIST OF ABBREVIATIONS
BCO Boards and Commissions Office
GOV Office of the Governor
IQ Intranet Quorum
OSA Office of the State Auditor
Audit No. 2021-0933-3S Office of the Governor
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Office of the Governor (GOV) for the period July 1,
2018 through June 30, 2020. The purpose of our audit was to determine whether GOV had filled all
vacant seats on the Commonwealth’s active boards and commissions as prescribed by state laws,
regulations, or executive orders.
Below is a summary of our finding and our recommendations, with links to each page listed.
Finding 1
Page 7
Some state boards and commissions had vacancies that GOV did not fill immediately when
members’ terms ended.
Recommendations
Page 9
1. GOV should enhance its process to monitor all required appointments by the Governor
to ensure that state boards and commissions are filled with appointments by the
Governor immediately when members’ terms end.
2. GOV should implement a review process for any new board or commission created to
ensure that it is not duplicative and does not have a mission similar to that of an existing
board or commission, and GOV should combine duplicate responsibilities to create a
single board or commission.
3. GOV should develop policies and procedures to ensure that its Intranet Quorum
database accurately contains all active state boards and commissions and the
appointments made to them.
Audit No. 2021-0933-3S Office of the Governor
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Office of the Governor (GOV) was established under Section I of Chapter II of the Constitution of the
Commonwealth. It consists of the Offices of the Governor and the Lieutenant Governor, both of whom
are elected by popular vote every four years. The Governor and Lieutenant Governor oversee a cabinet
consisting of the Secretaries of Administration and Finance, Education, Energy and Environmental
Affairs, Health and Human Services, Housing and Economic Development, Labor and Workforce
Development, Public Safety and Security, and Transportation. Each Secretary is appointed by the
Governor and is responsible for overseeing the activities of the executive departments and other
agencies within the secretariat. GOV sets policy for implementation by all cabinet secretariats, agencies,
offices, commissions, boards, and other entities within the state executive department to achieve GOV’s
mission.
According to its internal control plan,
The Office of the Governor is committed to making Massachusetts a truly great place for all
individuals to live, work, start a business, raise a family, and reach their full potential. It will work
toward a growing economy with family-sustaining jobs; ensure that schools across the
Commonwealth provide opportunity for every child regardless of zip code; improve the delivery of
state services; and make Beacon Hill a true partner with our local governments to create safer
and thriving communities across Massachusetts.
GOV makes appointments to more than 700 boards and commissions in the Commonwealth. Each state
board’s or commission’s enabling legislation describes the Governor’s responsibility and authority to
make appointments to it.
For fiscal years 2019 and 2020, GOV’s state appropriations were $5,251,345 and $5,751,345,
respectively.
Boards and Commissions Office
GOV’s Boards and Commissions Office (BCO) has existed since approximately 1991. It oversees
appointments of appropriate and qualified candidates to all Commonwealth executive branch boards
and commissions. BCO regularly interacts with citizens across the Commonwealth to assess and recruit
qualified and committed candidates to serve on state boards and commissions. Since approximately
2005, BCO has used the Intranet Quorum (IQ) database, a data tracking software product from Leidos
Audit No. 2021-0933-3S Office of the Governor
Overview of Audited Entity
3
Digital Solutions, Inc., to manage the appointments of each seat on each state board or commission.
During our audit period, BCO was staffed by a director and three other staff members.
Typically, BCO is notified of a citizen’s request to be appointed to a state board or commission by phone,
email, or the GOV website. Boards and commissions also notify BCO if the Governor needs to make an
appointment. According to BCO personnel, when the Governor appoints someone to a board or
commission, the following steps are taken to track the appointment:
 BCO notifies the Office of the Secretary of the Commonwealth and the Office of the Comptroller
of the Commonwealth of the appointment by sending both offices a copy of the appointment
letter signed by the Governor.
 BCO notifies the relevant board or commission by sending it a copy of the appointment letter
signed by the Governor.
 BCO creates a record of the appointment in IQ.
Through recruitment procedures implemented by BCO, GOV seeks to ensure that appointees to the
Commonwealth’s boards and commissions are qualified and diverse. On February 17, 2011, the thenGovernor signed Executive Order 526, which reaffirmed the Commonwealth’s commitment to civil rights
and provided for transgender people to have equal access to executive branch employment and
programs. The order, which covers state boards and commissions, states,
Non-discrimination, diversity, and equal opportunity shall be the policy of the Executive Branch of
the Commonwealth of Massachusetts in all aspects of state employment, programs, services,
activities, and decisions.
Through BCO, GOV also measures diversity in appointments to state boards and commissions. During
our audit period, BCO set internal goals for at least 50% of its appointees to be female and at least 20 to
25% of appointments to be made to diverse candidates.1

1. According to the Diversity Spreadsheet GOV uses to track the diversity of board and commission appointees, GOV considers
an appointment diverse if the appointee identifies as being “Black, Hispanic, Asian, or Native American”; being a veteran;
having a disability; or being “[lesbian, gay, bisexual, or transgender].”
Audit No. 2021-0933-3S Office of the Governor
Audit Objectives, Scope, and Methodology
4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Office of the Governor (GOV) for
the period July 1, 2018 through June 30, 2020.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is our audit objective, indicating the question we intended our audit to answer, the conclusion we
reached regarding the objective, and where the objective is discussed in this report.
Objective Conclusion
1. Does GOV ensure that it immediately fills vacancies on state boards and commissions
as prescribed by state laws, regulations, or executive orders?
No; see Finding 1
To achieve our audit objective, we gained an understanding of the internal controls we deemed relevant
to the objective by reviewing GOV’s mission statement, policies, and procedures, as well as conducting
interviews with key personnel. Additionally, we performed the procedures described below.
Testing of Governor-Appointed Seats for Vacancies
We conducted an assessment of the number of active boards and commissions by asking GOV whether
each board or commission listed in the Intranet Quorum (IQ) database was active or inactive. We
obtained all appointment data from IQ and then filtered the data to include only boards and
commissions GOV had identified as active.
After giving us a list of 611 active boards and commissions, GOV told us it had determined that 25 of
these boards and commissions should be considered inactive. This resulted in a reduction of 212
Audit No. 2021-0933-3S Office of the Governor
Audit Objectives, Scope, and Methodology
5
Governor-appointed seats, discussed below.2 From the remaining 586 active boards from IQ, we
identified each board or commission seat that the Governor was required to fill and determined
whether each seat was filled as of the end of the audit period (June 30, 2020) by examining the member
identification number, appointment date, and term end date (the date the appointment term is
completed). We accomplished this by performing the following procedures.
We filtered the list of 8,333 seats that GOV initially provided to us from its database, removing
appointments with a blank term end date to produce a list of 7,246 seats. We sorted that list to ensure
that the most recent term end date was listed first. We then summarized the list by seat to remove
duplicates.3 Finally, we removed the 212 seats from inactive boards that had not initially been identified
as inactive, as well as 19 ex officio seats that GOV officials told us had not been made by the Governor,
which resulted in a list of 2,341 seats.
From that list, we determined whether each seat was vacant by determining whether the term end date
was after the end of the audit period. If the term end date was before that date, we deemed the seat
vacant. We determined each vacancy’s duration by calculating the number of days between the term
end date and the end of the audit period.
Data Reliability
To determine the reliability of data in IQ, we interviewed GOV personnel who were responsible for the
source data. We performed a user access test to verify that only GOV employees made entries in IQ.
Further, we reviewed the data in IQ for duplicates and to determine whether all the appointment dates
were before the end of the audit period. To determine the accuracy of data in IQ, we selected a sample
of 20 appointments and verified that there was an appointment letter from the Governor that matched
each appointee’s name and appointment date in IQ. To determine completeness of the appointment
data, we compared it to data in the Diversity Spreadsheet4 and identified 24 appointments that should
have been entered in IQ (see Finding 1). Additionally, in an interim response to our finding, GOV
identified 72 instances where the appointment record in IQ did not reflect the current appointment

2. For each board and commission, the Governor may have multiple appointments to make. For example, the Governor may
have nine appointments to make to a single board or commission. This would result in nine seats.
3. There may be many duplicate seat appointments within the 7,246 seats because there had been previous appointments to
the same seat. We are reporting on whether a vacancy existed at the end of the audit period for each unique seat, so we
filtered our list to include only unique seats.
4. GOV uses the Diversity Spreadsheet to track the diversity of appointments to boards and commissions.
Audit No. 2021-0933-3S Office of the Governor
Audit Objectives, Scope, and Methodology
6
term. Based on the procedures above, we determined that the appointment data from IQ were
sufficiently reliable for the purpose of this audit.
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
7
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. Some state boards and commissions had vacancies that the Office of the
Governor did not fill immediately when members’ terms ended.
As of the end of our audit period, the term had ended for 248 (10.6%) of the 2,341 seats5 on active state
boards and commissions; these 248 seats should have been filled by the Office of the Governor (GOV)
according to the Intranet Quorum (IQ) database. Additionally, during our audit, we noted that GOV did
not always maintain complete and accurate information about appointments in IQ. When boards and
commissions are without a full complement of appointees, they may lack the knowledge and skills to
effectively meet their mission and may encounter difficulty securing a quorum to officially conduct
business.
Based on our initial analysis, as of the end of our audit period, 230 (92.7%) of the 248 vacancies we
identified had existed for 200 calendar days or longer:
Days Vacant as of June 30, 2020 Number of Board/Commission Seats Vacant
Less than 200 18
200–1,000 78
1,001–2,000 81
2,001–3,000 22
3,001–4,000 23
Over 4,000 26
Total 248
Additionally, of the 2,341 total active seats, there were 97 new seats from 22 boards and commissions
that were created during the audit period, and IQ indicated that 4 (4.1%) of these were vacant at the
end of the audit period. The existence of 586 boards and commissions, and continued creation of boards
and commissions, may result in duplicative work.

5. In its interim response to our audit testing, GOV identified 19 ex officio seats, from the 2,572 seats that we initially
identified as active seats appointed by the Governor, that should not be included in this population because they were not
gubernatorial appointments. Additionally, GOV identified 212 seats that should not be included in our analysis because the
board or commission was inactive. We subtracted these 231 seats from the population, resulting in a total of 2,341 seats.
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
8
During our analysis, we identified 24 instances where appointments by the Governor to boards and
commissions were not listed in IQ. According to GOV officials, these were omitted from IQ in error.
Additionally, in its interim response to our audit testing, GOV identified 212 instances where a seat on a
board or commission was reported as vacant in IQ, but the board or commission was actually inactive
and no longer meeting. GOV did not accurately document in IQ that these 212 seats were for boards and
commissions that were inactive.
GOV also identified 72 instances where IQ did not list a seat as filled, but GOV officials researched the
vacancy and determined that the seats were filled with an active appointment. GOV did not accurately
document in IQ that these 72 seats had an active board or commission member appointed.
Without ensuring that board and commission appointment information is complete and accurate in its
database, GOV cannot effectively manage the filling of vacancies.
Authoritative Guidance
Regarding the Governor’s responsibility to appoint seats to boards and commissions,6 GOV’s website
states,
Governor Baker appoints citizens to more than 700 boards and commissions dealing with virtually
every state department and public policy area.
Each state board’s or commission’s enabling legislation describes its composition and, if applicable,
GOV’s appointment responsibility and authority. For example, Section 42 of Chapter 13 of the
Massachusetts General Laws states,
There shall be a board of registration of cosmetology and barbering to consist of 9 members who
shall be appointed by the governor, 1 of whom shall be designated as chairperson by a majority
vote of the board. Members shall be appointed for terms of 3 years; provided, however, that any
person designated to fill a vacancy shall be appointed only for the remainder of the unexpired
term of the member so replaced. Upon the expiration of a term of office, a board member may
continue to serve until a successor has been appointed and qualified.

6. GOV officials confirmed only 586 active boards and commissions during the audit period.
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
9
Reasons for Vacancies
GOV’s Boards and Commissions Office (BCO) has not established a process that effectively monitors
upcoming board and commission vacancies so they can be filled immediately when members’ terms
end. Additionally, GOV does not have an adequate process in place to monitor the creation of new
boards and commissions. Finally, GOV has not established any policies and procedures to ensure that
the information entered in IQ is complete and accurate.
Recommendations
1. GOV should enhance its process to monitor all required appointments by the Governor to ensure
that state boards and commissions are filled with appointments by the Governor immediately when
members’ terms end.
2. GOV should implement a review process for any new board or commission created to ensure that it
is not duplicative and does not have a mission similar to that of an existing board or commission,
and GOV should combine duplicate responsibilities to create a single board or commission.
3. GOV should develop policies and procedures to ensure that IQ accurately contains all active state
boards and commissions and the appointments made to them.
Auditee’s Response
Seats that are truly vacant, with no one serving either in term or in holdover, should be filled
with qualified individuals as soon as possible. The report’s concern, however, that there were
vacancies not filled “immediately when the members’ terms ended,” seems to suggest a problem
that does not reflect the reality of board turnover, recruitment, and the appointment process.
This finding ignores that board and commission members appointed by the Governor continue to
serve in holdover before they are re-appointed for a subsequent term or a successor is appointed
to take their seat. General Laws c. 30 § 8, allows members appointed by the Governor to serve
and exercise all the powers of the public office when serving in holdover status. The policy
reasons supporting this provision are well established.
The table [in Finding 1], therefore, would be more accurate if it accounted for holdover service
by calculating the number of days vacant based on the end of a member’s actual service rather
than the end of the member’s term.
We have a related concern that the finding’s focus on whether seats were filled “immediately
when the members’ terms ended,” where no statute or rule specifies a required window for reappointment, suggests a problem of legitimacy or legal authority for members serving in holdover
that does not exist. These members were lawfully serving the Commonwealth.
We also observe that the OSA’s calculation overstates the percentage of vacancies by excluding
all the members who served without a term or coterminous to the Administration because OSA
reviewed only appointments with a term-end date. This methodological approach under-reported
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
10
BCO’s actual performance in filling available seats. OSA’s approach also fails to recognize or give
BCO credit for filling appointments to these seats and further fails to recognize these members’
service.
Finally, the report’s statement that BCO “did not always maintain complete and accurate
information about appointments in IQ” ignores the high accuracy with which most appointment
information was entered into the IQ database during the audit period. The audit report identified
96 instances out of 2,341 seats tested where some appointment information was inadvertently
omitted from the IQ database during the audit period. This data entry error rate (4%) is within
acceptable standards. Additionally, in all instances where appointment information was not
properly entered into the IQ database, the BCO properly maintained the appointment information
in other office records.
Response to Recommendations
Recommendation 1 . . .
The BCO fills vacant seats with qualified individuals as soon as possible. The BCO prioritizes
qualifications of the candidate when filling board and commission seats. It is more important to
find a qualified candidate to serve than to fill a vacant seat immediately with an unqualified
candidate. Additionally, there is no need for the BCO to fill a seat immediately at the end of a
member’s term, if the member is willing to serve in holdover status.
The BCO has a process to monitor all appointments legally required by the Governor. BCO staff
are made aware of vacancies by board members as well as by a board or commission’s staff
member, Secretariat, or agency staff member that provides administrative support to a board or
commission. Vacancies are then recorded in the IQ database, including the reason for departure.
This process was the standard practice followed during the audit period and was memorialized in
a written policy and procedure document that was distributed to all Boards and Commissions
staff members in December 2020.
The BCO has increased the frequency of its meetings with board or commission staff members,
Secretariats, and agency staff members that provide administrative support, to ensure that the
BCO’s records are current. Additionally, the BCO is implementing a quarterly review of the IQ
database data with appropriate board and commission staff, Secretariat, or agency staff, to
ensure that the BCO is aware of any upcoming or unforeseen vacancies.
Recommendation 2 . . .
Most boards and commissions are created by the Legislature. Consequently, the Office of the
Governor does not have legal authority to unilaterally consolidate or alter legislatively-created
boards or commissions, even when they may be “duplicative” of existing bodies. A 2014
legislative audit of state boards and commissions stated, “[t]he Legislature should enact
legislation to allow automatic dissolution or sunsetting of inactive commissions and to create a
formal process to regularly review commissions to determine whether they should be abolished.”
This Office agrees with that conclusion. [On July 30, 2014, the Senate Committee on Post Audit
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
11
and Oversight published “Fulfilling Their Mandates?: A Review of Massachusetts State Boards and
Commissions.”]
In some instances, the Office creates boards and commissions through the Governor’s Executive
Orders. In doing so, it strives to not create duplicative boards or boards with similar missions.
Executive Orders are updated as needs change.
Recommendation 3 . . .
The BCO has developed policies and procedures designed to ensure that IQ accurately contains
the most up-to-date information. The BCO strives for data to be entered accurately 100% of the
time and in a timely fashion. In December 2020, the BCO created a set of comprehensive,
written policies and procedures for recording data in the IQ database and distributed them to
BCO staff. The BCO also instructed BCO staff to enter new boards and commissions and new
members as soon as possible after the appointment letters are issued.
The BCO has recently conducted a thorough review of the IQ database to produce a list of
boards that were considered active and inactive during the audit period. Staff reviewed data from
over thirty years. Through the review, the BCO determined that many boards and commissions
existed in the database that should have been marked inactive by earlier Administrations prior to
2015. Altogether, the BCO found 177 inactive boards that were incorrectly marked as active in IQ
during the audit period. Of these inactive boards, two-thirds of them (118) should have been
marked inactive prior to this Administration. This type of review was a first in the history of the
BCO. To keep our successors from undergoing the same laborious process, the BCO will review
the IQ database in meetings with the boards’ and commissions staff, Secretariats, and other
agency staff members that support each board and commission. This will include a specific
review of whether a board or commission should be marked inactive.
Auditor’s Reply
In our audit, we set out to determine whether GOV immediately filled vacant Governor-appointed seats
on state boards and commissions as prescribed by state laws, regulations, or executive orders. With
information obtained from BCO, we analyzed IQ data, taking into account all information GOV uses to
monitor vacancies. We did not include the 978 appointees that BCO has termed holdovers in the count
of vacant appointments. (See “Other Matters” for further discussion of holdovers.) In our analysis, we
found 248 appointment vacancies as of June 30, 2020. Therefore, we concluded that GOV had not
immediately filled all vacant Governor-appointed seats on state boards and commissions.
In its response, GOV contends that the Office of the State Auditor (OSA) did not consider holdover status
for the table in Finding 1. This table shows the number of days seats remained vacant. Although Section
8 of Chapter 30 of the General Laws allows members appointed by the Governor to serve in holdover
status with the same authority that existed during their terms, there is no identifier in IQ to indicate that
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
12
status. Moreover, there is no indication that GOV knows whether a particular holdover is actively
participating on a board or commission. Therefore, we could not address holdover status when
calculating the length of time between the end of the prior appointment term and June 30, 2020 for
each of the 248 vacant seats.
In its response, GOV states that no statute or rule specifies when a vacant seat must be reappointed. We
believe BCO should monitor appointees as they approach the end of their terms to determine which
boards and commissions are expected to have upcoming vacancies and fill them with new appointments
as soon as possible. The audit identified 248 vacant seats, 230 of which had existed for 200 calendar
days or more. These results were the reason for our recommendation that GOV enhance its monitoring
process to ensure that the Governor fill appointments immediately when members’ terms end.
In its response, GOV states that our audit did not take into consideration appointments made for seats
that did not have term end dates. As detailed in the “Audit Objectives, Scope, and Methodology” section
of this report, our review focused on GOV making timely appointments to vacant seats. The percentages
used in our findings were based on our consideration of board or commission seats that were not
immediately filled upon vacancy. A seat with no term end date would not have a definitive vacancy date,
so we did not consider such seats because there was no affirmative duty to fill them by a specific date.
Additionally, GOV contends that the 96 instances of errors we discovered in IQ in our analysis are
acceptable because the error rate is 4%. GOV states that the correct information was documented in
separate GOV records. However, GOV officials told OSA several times that IQ is the tool GOV uses to
manage appointments. If at least 4% of appointment data in IQ is inaccurate or incomplete, GOV cannot
ensure that it appropriately manages all gubernatorial appointments to boards and commissions.
Regarding the response to our first recommendation, OSA concurs that BCO has a process to monitor all
appointments legally required by the Governor; however, improvements can be made to the process to
ensure that appointments are made to limit vacancies. GOV’s response notes improvements such as the
following:
The BCO has increased the frequency of its meetings with board or commission staff members,
Secretariats, and agency staff members that provide administrative support, to ensure that the
BCO’s records are current. Additionally, BCO is implementing a quarterly review of the IQ
database data with appropriate board and commission staff, Secretariat, or agency staff, to
ensure that the BCO is aware of any upcoming or unforeseen vacancies.
Audit No. 2021-0933-3S Office of the Governor
Detailed Audit Findings with Auditee’s Response
13
Based on these steps, it appears that GOV is taking steps to implement our first recommendation.
Our second recommendation was as follows:
GOV should implement a review process for any new board or commission created to ensure that
it is not duplicative and does not have a mission similar to that of an existing board or
commission, and GOV should combine duplicate responsibilities to create a single board or
commission.
In its response, GOV stated that it did not have the authority to unilaterally change legislatively created
boards and commissions. However, we are not recommending that GOV change boards and
commissions, but that it review and provide input on the missions of proposed ones and assess whether
they duplicate existing boards or commissions. If they do, BCO could propose to the Governor that the
Legislature review and reassess the mission before the legislation is signed. We again urge GOV to
implement our recommendation.
Our third recommendation was as follows:
GOV should develop policies and procedures to ensure that IQ accurately contains all active state
boards and commissions, as well as the appointments made to them.
Based on its response to our audit, GOV appears to have taken steps to address our recommendation.
Audit No. 2021-0933-3S Office of the Governor
Other Matters
14
OTHER MATTERS
The Office of the Governor should improve monitoring of holdover
appointments.
During our audit, the Office of the State Auditor (OSA) analyzed the Office of the Governor’s (GOV’s)
Intranet Quorum (IQ) database to determine whether the Governor immediately filled vacancies on
boards and commissions as prescribed by state laws, regulations, or executive orders. During this
analysis, GOV officials told us that many appointees of the Governor had holdover status as of the date
of our review (June 30, 2020). Holdover status was described to us as a status where the appointment
term has ended, but the appointee continues to serve on the board or commission until a replacement
is appointed. GOV officials identified 978 instances, among the 2,341 seats we tested, of appointees
with holdover status.
During our analysis, we did not see any indication of GOV effectively managing holdover status, such as
inclusion in IQ or any other data source. GOV officials told us that holdover status was intended to be a
short-term option to extend a term of an appointee whose term was about to expire when the
appointee was in the middle of completing a task or there was a delay in filling an appointment. As
shown below, 817 (83.5%) of appointees reported as holdovers have had this status for more than 200
days.
Number of Days in Holdover as of June 30, 2020 Number of Board/Commission Seats in Holdover
Less than 200 161
200–1,000 486
1,001–2,000 213
2,001–3,000 51
3,001–4,000 39
Over 4,000 28
Total 978
We recommend that GOV record in IQ whether an appointment has holdover status, along with the
holdover duration, and document its communications with each board and commission to end the
holdover status by either reappointing the current appointee or recruiting a new one to fill the seat.
Audit No. 2021-0933-3S Office of the Governor
Other Matters
15
Auditee’s Response
As discussed above, any member in an active seat on an active board who is serving beyond his
or her initially appointed term is serving in holdover status and exercises all of the powers of the
appointed position, as provided by G. L. c. 30, § 8. This general rule applies to all public officers
appointed by the Governor, unless the board or commission’s enabling statute provides
otherwise. Members serving in this capacity have the same rights and privileges as those who are
serving with an active term. The policy reasons supporting this provision are well established,
and the BCO sees no reason to devalue the continuing service of these members who often serve
for little to no remuneration.
While there is no field in the IQ database labeled “holdover,” the BCO can use IQ to identify
members serving in holdover status by reviewing active members on active boards who continue
to serve beyond their term end date. The BCO will continue to consult with boards’ and
commissions’ staff members, Secretaries, and agency staff to review those appointees who are
serving in holdover, to confirm their status and consider whether a change needs to be made to
the seat. The BCO is currently sending IQ data to these staff members to increase
communication regarding the holdover status of members. The BCO will focus this outreach on
the seats that have been in holdover status the longest.
Auditor’s Reply
We recommend that GOV improve its management of IQ in relation to members in holdover status. Our
recommendation is that the Boards and Commissions Office (BCO) note in IQ whether an appointee is a
holdover actively serving on a board or commission and, if so, what is the anticipated duration of the
holdover status. We also recommend that GOV document its communications with each board and
commission to end holdover status by either reappointing the current appointee or recruiting a new
one. A new appointment or reappointment does not devalue the continuing service of board or
commission members as GOV suggests above.
GOV’s response above is that BCO can “identify members serving in holdover status by reviewing active
members on active boards who continue to serve beyond the end of their term.” When we reviewed IQ,
there was no identifier that OSA saw, or that was communicated to us, to differentiate holdover
members. GOV’s response stating that it will review appointees currently serving in holdover status to
determine whether they are still serving and whether a seat change is appropriate is what we
recommended. OSA also recommends documenting in IQ how long a member will be available to serve
as a holdover. Also, since most holdovers identified as of June 30, 2020 had been in holdover status for
more than 200 days, we recommend taking an additional step to formally reappoint members who
express a desire to serve on their boards or commissions for additional full-length terms.

Official Audit Report – Issued December 1, 2022
Pension Reserves Investment Management Board
For the period July 1, 2019 through January 15, 2022
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
December 1, 2022
Ms. Deborah B. Goldberg, Chair
Pension Reserves Investment Management Board
84 State Street, Suite 250
Boston, MA 02109
Dear Ms. Goldberg,
I am pleased to provide this performance audit of the Pension Reserves Investment Management Board.
This report details the audit objectives, scope, methodology, findings, and recommendations for the audit
period, July 1, 2019 through January 15, 2022. My audit staff discussed the contents of this report with
management of the agency, whose comments are reflected in this report.
I would also like to express my appreciation to the Pension Reserves Investment Management Board for
the cooperation and assistance provided to my staff during the audit.
Sincerely,
Suzanne M. Bump
Auditor of the Commonwealth
cc: Michael G. Trotsky, Executive Director and Chief Investment Officer, Pension Reserves Investment
Management Board
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 7
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Pension Reserves Investment Management (PRIM)
Board for the period July 1, 2019 through January 15, 2022. In this performance audit, we examined the
PRIM Board’s compliance with its Investment Policy Statement by investing between 5%–10% of all its
investments with emerging managers.1 We also determined whether the PRIM Board developed a plan to
ensure that not less than 20% of its investment managers2 are owned by minorities, women, or people
with disabilities,3 in accordance with Section 23(8)(a–c) of Chapter 32 of the General Laws.
Our audit revealed no significant instances of noncompliance by the PRIM Board that must be reported
under generally accepted government auditing standards.
1. According to the PRIM Board’s Investment Policy Statement, emerging managers are “investment managers with less than
$2 billion of assets under management that may have shorter track records or investment managers that are minority-owned
or women-owned.” Assets under management refers to the total amount of money that an investment manager manages
for its clients.
2. According to the PRIM Board’s Investment Policy Statement, “PRIM employs professional investment managers [also called
investment management companies] and gives them discretion, consistent with specified objectives and guidelines, to
manage the [Pension Reserves Investment Trust] Fund’s assets.”
3. The PRIM Board refers to investment managers that are owned or led by minorities, women, or people with disabilities as
diverse managers. Investment managers that are minority-owned or woman-owned are also considered emerging managers,
as previously stated.
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Pension Reserves Investment Management (PRIM) Board was established by Chapter 661 of the Acts
of 1983, and later amended by Chapter 315 of the Acts of 1996, to oversee the management of the
Pension Reserves Investment Trust (PRIT) Fund. The PRIT Fund, established by the same legislation, is the
investment portfolio for the assets of the Massachusetts State Employees’ Retirement System, the
Massachusetts Teachers’ Retirement System, the State Retiree Benefits Trust Fund, and other
Massachusetts retirement systems that elect to invest in the fund. According to the PRIM Board’s website,
The Pension Reserves Investment Management Board’s mission is to provide a professional
investment service for public employee retirement funds that maximizes the return on investment
within acceptable levels of risk. We broadly diversify its investment portfolio, capitalizing on
economies of scale to achieve cost-effective operations, and provide access to high quality,
innovative investment management firms.
Additionally, the PRIM Board’s website states,
The nine-member PRIM Board acts as Trustee for each retirement system that invests in the PRIT
Fund and is responsible for the control and management of the Fund.
The Treasurer & Receiver-General of the Commonwealth, or his/her designee, is a member ex
officio of the PRIM Board and serves as its Chair. The Treasurer also appoints one member of the
PRIM Board, who is a private citizen with an investment/business background. The Governor, or
his/her designee, is also an ex officio member and appoints two members of the Board: one is a
non-state official or employee, and one is a representative of a public safety union. The StateTeachers’ Retirement System has two representatives on the Board: the members of that
Retirement System elect one, and one is an Elected Member of the Massachusetts Teachers’
Retirement Board. The State Employees’ Retirement System has two representatives on the Board:
the members of that Retirement System elect one, and one is an Elected Member of the State
Employees’ Retirement Board.
The PRIM Board has four advisory committees: an administration and audit committee, a compensation
committee, an investment committee, and a real estate and timberland committee. The purpose of these
committees is to assist the PRIM Board with carrying out its duties, as well as offer counsel on investment
decisions and the PRIM Board’s general operations.
The PRIM Board’s staff consists of an investment team of 28 employees and a finance and operation team
of 28 employees. The investment team works on the management of the PRIT Fund, and the finance and
operation team works on financial reporting for the PRIT Fund and on all other administrative support.
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
3
The members of the PRIM Board, as trustees for each retirement system that invests in the PRIT Fund,
have the authority to employ an executive director, external investment managers, custodians, and
consultants to develop policies and procedures needed to manage the assets of the PRIT Fund. The
executive director creates investment and administrative policy, implements the policies and programs
created by the members of the PRIM Board, and reports to the members of the PRIM Board on the status
of the PRIT Fund and the operations of the PRIM Board.
According to the PRIM Board’s Investment Policy Statement, its investment objective is to manage the
PRIT Fund to “achieve the highest level of investment performance that is compatible with its risk
tolerance and prudent investment practices.” Under Section 22C of Chapter 32 of the Massachusetts
General Laws, by 2040 the PRIT Fund should be fully funded to meet the Commonwealth’s pension
obligations through annual payments made to the fund in accordance with a funding schedule approved
by the Legislature and through the accumulation of investment returns in the fund. The Commonwealth
has adopted a schedule of state pension appropriations that assumes a long-term annualized return of
7% for the PRIT Fund. As of June 30, 2021, the PRIT Fund’s five-year annualized return was 11.9%.
Retirement Systems
Massachusetts county, city, and town retirement systems that choose to invest in the PRIT Fund can be
either participating retirement systems or purchasing retirement systems. Participating retirement
systems are required by Section 22 of Chapter 32 of the General Laws to invest all of their retirement
funds in the PRIT Fund, and their assets must remain in the PRIT Fund for at least five years. Purchasing
retirement systems can invest part of their funds in the PRIT Fund and are able to contribute and withdraw
assets at will. Participating and purchasing retirement systems both share in the investment earnings of
the PRIT Fund based on their proportionate share of investments. As of June 30, 2021, there were 100
retirement systems with investments in the PRIT Fund.
Chapter 84 of the Acts of 1996 allows the retirement systems of Massachusetts authorities, counties,
cities, and towns to invest in the separate accounts of the PRIT Fund as an alternative to investing in the
General Allocation Account.4 This investment option is called segmentation. According to the PRIM
Board’s website, “This ‘segmentation’ of the PRIT Fund’s investment options gives local retirement boards
4. The General Allocation Account is the main aggregate account in which retirement systems can invest their funds. It invests
in all the types of investment assets of the PRIT Fund, which also have their own accounts.
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
4
flexibility to pick and choose specific asset classes in whatever proportions they believe are best suited to
their needs.”
Asset Allocation Targets
The PRIM Board sets target ranges for how much of its funds it wants to allocate to each asset class (i.e.,
type of investment) for the PRIT Fund as a way of reducing risk and delivering the highest investment
return possible. Below is a table highlighting the PRIT Fund’s asset allocation target ranges and the actual
allocations, as of June 30, 2021:
Asset Class Actual Allocation Target Range
Global Equity 42.7% 34%–44%
Core Fixed Income 15.3% 12%–18%
Value-Added Fixed Income 6.9% 5%–11%
Real Estate 8.3% 7%–13%
Private Equity 14.5% 11%–17%
Timberland 3.0% 1%–7%
Portfolio Completion Strategies 8.3% 7%–13%
Overlay 1.0% –
Net assets in the PRIT Fund totaled $74,985,759,000 for fiscal year 2020 and $95,698,845,000 for fiscal
year 2021. The PRIM Board incurred total investment management fees of $140,692,000 and
$164,076,000 for fiscal years 2020 and 2021, respectively.
The PRIM Board incurred investment advisory fees of $11,563,000 for fiscal year 2020 and $13,049,000
for fiscal year 2021; custodian fees of $1,670,000 in fiscal year 2020 and $1,672,000 in fiscal year 2021;
and other administrative fees of $15,973,000 in fiscal year 2020 and $20,147,000 in fiscal year 2021.
Investment Manager Goals
On January 14, 2021, Section 23(8)(a–c) of Chapter 32 of the General Laws was enacted, requiring the
PRIM Board to establish policy regarding investments with diverse managers. Specifically, the PRIM
Board’s goal for investment managers was revised to the following in Section 23(8)(b) of Chapter 32 of
the General Laws:
It shall be the goal of the PRIM Board that not less than 20% of investment managers be minorities,
females, and persons with disabilities. It shall further be the goal of the PRIM board to utilize
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
5
businesses owned by minorities, females and persons with disabilities for not less than 20 per cent
of total contracts awarded [to investment managers].
According to Section 23(8)(c) of Chapter 32 of the General Laws,
Annually, not later than January 15 of each year, PRIM shall file with the house and senate
committee on ways and means and with the joint committee on public service a report detailing its
progress toward implementing the policies and goals outlined [in Section 23(8)(b)]. Such report
shall include documentation related to all minority investment managers considered for investment,
including documentation, where applicable, of the reasons for declining any such investment.
The PRIM Board’s Investment Policy Statement, amended in February 2021, states,
The PRIM Board may consider hiring “emerging managers”, i.e., investment managers with less
than $2 billion of assets under management (AUM) that may have shorter track records or
investment managers that are minority-owned or women-owned. PRIM should consider emerging
managers for all asset classes, and should make a good-faith effort to invest with emerging
managers for all asset classes, and should set an objective of investing between 5 and 10% of all
new and current investments with emerging managers. In this effort, PRIM should consider using
advisors to identify emerging managers.
FUTURE Initiative and the Emerging-Diverse Manager Program
To achieve its goal of having at least 20% of its investment managers be owned by minorities, women, or
people with disabilities, the PRIM Board developed the FUTURE Initiative and the Emerging-Diverse
Manager Program. According to the PRIM Board’s 2021 FUTURE Initiative Annual Report,
In May 2021, PRIM launched the FUTURE Initiative, our strategic plan to achieve the goals set
forth by this legislation as well as position PRIM to be a leader on improving diversity in the asset
management industry. The FUTURE Initiative is focused on four keys to success.
1. Reduce Barriers for Diverse Managers.
The first step is to reduce the barriers to diverse managers. PRIM will work to ensure all
diverse managers have a clear path to work with PRIM regardless of their size or track
record.
2. Enhance [Diversity, Equity, and Inclusion, or DEI] Reporting.
PRIM will enhance the measuring and tracking of comprehensive DEI information across
our investment managers and vendors, and ultimately report on this information, as
required by the new law.
3. Improve the Sourcing of Diverse Investment Managers.
PRIM will work to improve our sourcing and pipeline of diverse managers. . . .
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
6
4. Continue to Allocate Capital to Diverse Managers.
PRIM will continue to allocate capital to diverse managers to achieve our goals. . . .
The first key to our FUTURE success is reducing barriers for diverse managers by creating a path
for all diverse managers, regardless of their size or track record, to manage assets for PRIM. To
achieve this, PRIM launched an Emerging-Diverse Manager Program (Program).
For more established diverse managers, those with higher assets under management (AUM) and
longer track records, PRIM staff will continue to review, conduct due diligence, select, and monitor
these managers as we have successfully done in the past. PRIM’s new Emerging-Diverse Manager
Program was created to ensure that prospective diverse and emerging managers, firms with lower
assets under management or shorter track records, will now have a clear path to partner with
PRIM.
This Program will utilize managers-of-managers as an extension of PRIM staff, to help PRIM source,
conduct due diligence and have full discretion to select and monitor a portfolio of emerging-diverse
managers. . . .
In late 2021, Lenox Park surveyed PRIM’s investment managers. The data from this survey will
provide PRIM a single source of DEI data on our managers that can be utilized to track and monitor
progress over time. . . .
In May 2021, a new page on PRIM’s website (mapension.com) was added that allows any
investment manager to fill out a questionnaire, which is then uploaded into our internal database
of managers and added to PRIM’s pipeline of managers to review. This web page provides a
gateway portal for managers to submit information to us (including their diversity designation) and
enhances PRIM’s ability to track prospective managers.
After Section 23(8)(a–c) of Chapter 32 of the General Laws was enacted, the PRIM Board measured its
investments with diverse managers by extracting performance reports from its custodian bank’s5
information system to calculate what percentage of funds in the PRIT Fund were from diverse managers.
These reports captured the source accounts (accounts designated for each of the PRIM Board’s
investment managers) and net asset values6 of all investment managers, as well as the aggregated net
asset values of the diverse managers. As of September 30, 2021, eight percent of the PRIT Fund was
managed by diverse managers.
5. A custodian bank is a financial institution responsible for safeguarding the assets (e.g., bonds or cash) of its clients and has
possession of the assets. These banks often offer related services, such as account management, handling taxes, and
distributing dividends. The bank is generally not engaged in other consumer banking services, such as lending.
6. According to the United States Securities and Exchange Commission’s website, the net asset value “is the company’s total
assets minus its liabilities.”
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
7
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of Pension Reserves Investment
Management Board (PRIM) Board for the period July 1, 2019 through January 15, 2022.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer and the
conclusion we reached regarding each objective.
Objectives Conclusion
1. Did the PRIM Board meet its objective of investing between 5%–10% of all investments
with emerging managers, as required by Section 21 of the PRIM Board’s Investment
Policy Statement?
Yes
2. Did the PRIM Board develop a plan to ensure that not less than 20% of its investment
managers are owned by minorities, women, or people with disabilities, in accordance
with Section 23(8)(a–c) of Chapter 32 of the General Laws?
Yes
To achieve our audit objectives, we gained an understanding of the internal control environment in the
areas related to the objectives by reviewing applicable PRIM Board policies and procedures, as well as the
PRIM Board’s internal control plan; conducting interviews with management and other employees; and
performing walkthroughs of the processes related to the PRIM Board’s investments with emerging
managers.
To obtain sufficient, appropriate evidence to address our audit objectives, we performed the following
procedures.
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
8
To determine whether the PRIM Board met its objective of investing between 5%–10% of all investments
with emerging managers, as required by Section 21 of the PRIM Board’s Investment Policy Statement,
from July 1, 2019 through January 13, 2021,7 we performed the following tests:
 We obtained and reviewed a list of emerging managers used by the PRIM Board during the audit
period with information on assets under management (AUM) under $2 billion and
woman/minority-owned status. This list was put together by the PRIM Board in a Microsoft Excel
spreadsheet, upon our request. We also obtained and reviewed a list of the PRIM Board’s total
investments, including investments with diverse managers, from the PRIM Board custodian bank’s
information system, NEXEN,8 which included the source account names and AUM amounts of the
PRIM Board’s investment managers. We obtained and reviewed a list of investment managers’
names from Dynamo.9
 We merged the lists from both systems to match the investment managers’source account names
from NEXEN with the investment managers’ names in Dynamo to generate a comprehensive list
of the PRIM Board’s investment managers with their AUM amounts and woman/minority-owned
status.
 Using the data from the Microsoft Excel spreadsheet and comprehensive list of the PRIM Board’s
investment managers, we calculated what percentage of all the PRIM Board’s investments were
with emerging managers.
To determine whether the PRIM Board developed a plan to ensure that not less than 20% of its investment
managers are owned by minorities, women, or people with disabilities (or diverse managers), in
accordance with Section 23(8)(a–c) of Chapter 32 of the General Laws, from January 14, 2021 through
January 15, 2022, we performed the following tests:
 We reviewed the meeting minutes of the three PRIM Board meetings that took place during the
above period to ensure that the four parts of the PRIM Board’s FUTURE Initiative were discussed.
We also reviewed the following:
 the meeting meetings for evidence of the PRIM Board’s approval to allocate capital to the
FUTURE Initiative’s Emerging-Diverse Manager Program
7. When Section 23(8)(a–c) of Chapter 32 of the General Laws came into effect on January 14, 2021, the PRIM Board stopped
measuring its investments with emerging managers and created a plan to measure its investments with diverse managers to
comply with this law. Therefore, we only considered the period July 1, 2019 through January 13, 2021 for this objective.
8. The PRIM Board’s custodian bank, BNY Mellon, uses NEXEN to contain a list of PRIM Board investment managers’ source
account names and AUM amounts.
9. PRIM uses Dynamo as a customer relationship management software. It stores information such as email communications,
investment strategy, quarterly reviews, and company information for each investment manager used by PRIM. It also allows
PRIM to monitor investment manager performance. Dynamo contains investment managers’ names and diversity
information (i.e., whether they are owned by minorities, women, or people with disabilities).
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
9
 a contract signed by the PRIM Board to hire Lenox Park Solutions10 to conduct analytics
related to diversity, equity, and inclusion for the PRIM Board
 PRIM Board presentation materials, which covered the proposal of the FUTURE Initiative and
changes to the PRIM Board’s website.
Data Reliability Assessment
NEXEN
To gain an understanding and assess the reliability of the list of PRIM Board’s total investments,
including investments with diverse managers, from NEXEN, we interviewed PRIM Board officials. We
also reviewed the System and Organization Control reports11 that covered the periods April 1, 2019
through March 31, 2020; April 1, 2020 through March 31, 2021; and October 1, 2020 through
September 1, 2021. These reports described testing of the information system general controls; no
internal control deficiencies were identified. We also obtained and reviewed bridge letters12 for the
periods March 31, 2020 through June 30, 2020; March 31, 2021 through June 30, 2021; and
September 30, 2021 through December 31, 2021. We observed PRIM Board officials perform queries
and extract reports of total investments, including investments with diverse managers, as of each of
the following dates: December 31, 2020; January 31, 2021; and September 30, 2021.
Dynamo
To assess the reliability of the list of all investment managers from Dynamo, we interviewed PRIM
Board officials to gain an understanding of this system and its data and observed PRIM Board officials
extract the list of all investment managers as of March 4, 2022. We reviewed the System and
Organization Control reports that covered the periods April 1, 2019 through September 30, 2019;
January 1, 2020 through September 30, 2020; and January 1, 2021 through September 30, 2021. We
also obtained and reviewed bridge letters for the periods September 30, 2019 through October 8,
2020; September 30, 2020 through September 8, 2021; and September 30, 2021 through
10. Lenox Park Solutions is a data analytics company that works with financial service companies, or similar agencies, to gather
information and statistics on diversity, equity, and inclusion for the company or the vendors it uses.
11. A System and Organization Control report about a service organization’s systems is issued by an independent contractor to
provide assurance about a service organization’s security, processing integrity, confidentiality, and/or privacy controls.
12. A bridge letter, also known as a gap letter, is made available by a service organization to cover the period between the
reporting period end date of a System and Organization Control report and the release of a new report.
Audit No. 2022-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
10
February 24, 2022. These reports described testing of the information system general controls; no
internal control deficiencies were identified.
In addition, we selected a sample of 11 of the 36 diverse managers on the list of all the PRIM Board’s
investment managers. We verified that they were owned by women, minorities, or people with
disabilities by reviewing the results of the survey conducted by the PRIM Board’s vendor Lenox Park
Solutions and Third-Party Due Diligence reports13 on the investment managers.
We also selected a sample of 10 out of 187 investment managers that were not identified as diverse
managers from the list of the PRIM Board’s investment managers. We verified that they were not
owned by women, minorities, or people with disabilities by reviewing Third-Party Due Diligence
reports, hiring documentation, and the PRIM Board’s emails to investment managers to confirm
ownership diversity information.
List of Emerging Managers
To assess the accuracy of the list of emerging managers compiled by the PRIM Board in a Microsoft
Excel spreadsheet, we selected a sample of 10 of the 36 investment managers from the list. We
verified that each investment manager had AUM under $2 billion or was woman/minority-owned by
reviewing Third-Party Due Diligence reports or PRIM Board meeting minutes.
We tested for duplicates and missing data for the lists obtained from NEXEN and Dynamo. We also
reconciled the lists to each other to ensure that terminated managers or data outside our audit period
were not included.
Based on the procedures above, we determined that the data were sufficiently reliable for the purpose of
this audit.
Conclusion
Our audit revealed no significant instances of noncompliance that must be reported under generally
accepted government auditing standards.
13. A Third-Party Due Diligence report is completed in preparation for a business transaction (such as a corporate merger or
purchase of securities). It provides details on a company’s assets, liabilities, contracts, benefits, and potential issues.

Official Audit Report – Issued November 13, 2025
Pension Reserves Investment Management Board
For the period July 1, 2022 through June 30, 2024
State House Room 230 ◼ Boston, MA 02133 ◼ auditor@massauditor.gov ◼ www.mass.gov/auditor
November 13, 2025
Michael Trotsky, Executive Director and Chief Investment Officer
Pension Reserves Investment Management Board
84 State Street
Boston, MA 02109
Dear Mr. Trotsky:
I am pleased to provide to you the results of the enclosed performance audit of the Pension Reserves
Investment Management Board. As is typically the case, this report details the audit objectives, scope,
methodology, findings, and recommendations for the audit period, July 1, 2022 through June 30, 2024.
As you know, my audit team discussed the contents of this report with agency managers. This report
reflects those comments. I appreciate you and all your efforts at the Pension Reserves Investment
Management Board. The cooperation and assistance provided to my staff during the audit went a long
way toward a smooth process. Thank you for encouraging and making available your team. I am
available to discuss this audit if you or your team has any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 8
APPENDIX .........................................................................................................................................................15
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
List of Abbreviations
ii
LIST OF ABBREVIATIONS
AUM assets under management
ESG Environmental, Social, and Governance
NAV net asset value
PRIM Pension Reserves Investment Management Board
PRIT Pension Reserves Investment Trust
ROI return on investment
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Pension Reserves Investment
Management Board (PRIM) for the period July 1, 2022 through June 30, 2024.
The purpose of our audit was to determine the following:
• To what extent did PRIM review the portfolios of emerging managers against its benchmarks to
ensure that the Pension Reserves Investment Trust Fund was not exposed to undue risks in
accordance with Section 5 of PRIM’s “Investment Policy Statement”?
• To what extent did PRIM work toward its goal of having at least 20% of investment managers who
are minorities,1 women, and people with disabilities, in accordance with Section 23(8)(b) of
Chapter 32 of the General Laws?
• To what extent did PRIM review the quarterly calculations of management fees and investment
returns in accordance with the third operational risk category of Section V (Risk Analysis) of its
Investment and Operations Risk Control Document?
Our audit revealed no significant issues that must be reported under generally accepted government
auditing standards. Therefore, we concluded that, during the audit period, PRIM met the relevant criteria
regarding our objectives.
1. For the purposes of this audit report, we use the term minority in the same manner that the Supplier Diversity Office uses
the term minority when defining a minority-owned business in its Comprehensive Annual Report Fiscal Year 2022. The
Supplier Diversity Office defines a minority-owned business “as a business that is owned by a racially or ethnically diverse
individual. While the [term is] meant to define an ethnically or racially diverse individual . . . [it is] not meant to denote a
smaller or lesser status of the individuals . . . Included in this definition.”
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Pension Reserves Investment Management Board (PRIM) was established by Chapter 661 of the Acts
of 1983, and later amended by Chapter 315 of the Acts of 1996, to oversee the management of the
Pension Reserves Investment Trust (PRIT) Fund. The PRIT Fund, established by the same legislation, is the
investment portfolio for the assets of the Massachusetts State Employees’ Retirement System, the
Massachusetts Teachers’ Retirement System, the State Retiree Benefits Trust Fund, and other
Massachusetts retirement systems that elect to invest in the fund.
PRIM’s website states the following:
PRIM serves as a professional investment service for public employees. Our mission is to provide
a professional investment service for public employee retirement funds that maximizes the return
on investment within acceptable levels of risk. We broadly diversify its investment portfolio,
capitalizing on economies of scale to achieve cost-effective operations, and provide access to high
quality, innovative investment management firms.
According to Section 23(2A) of Chapter 32 of the Massachusetts General Laws, PRIM is governed by a
nine-member board. According to PRIM’s website,
The nine-member PRIM Board acts as Trustee for each retirement system that invests in the PRIT
Fund and is responsible for the control and management of the Fund.
The Treasurer and Receiver-General of the Commonwealth, or his/her designee, is a member ex
officio of the PRIM Board and serves as its Chair. The Treasurer also appoints one member of the
PRIM Board, who is a private citizen with an investment/business background. The Governor, or
his/her designee, is also an ex officio member and appoints two members of the Board: one is a
non-state official or employee, and one is a representative of a public safety union. The StateTeachers’ Retirement System has two representatives on the Board: the members of that
Retirement System elect one, and one is an Elected Member of the Massachusetts Teachers’
Retirement Board. The State Employees’ Retirement System has two representatives on the Board:
the members of that Retirement System elect one, and one is an Elected Member of the State
Employees’ Retirement Board.
PRIM has the following five advisory committees: the Investment Committee, the Real Estate and
Timberland Committee, the Administration and Audit Committee, the Compensation Committee, and the
Stewardship and Sustainability Committee. According to PRIM’s website,
The committees review and analyze proposed investments and issues under their jurisdiction and
make recommendations to the Board for consideration and approval by Board vote. Committee
members are appointed by the Chair and approved by the Board.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
3
The board is responsible for appointing PRIM’s executive director, outside investment managers,
custodians, advisors, and other positions it deems necessary to fulfill PRIM’s mission. The board also has
the authority to formulate policies and procedures and to take other actions as necessary and appropriate
to manage the assets of the PRIT Fund.
The executive director is the senior executive in charge of PRIM and is responsible for planning, directing,
and executing PRIM’s administrative and investment activities in accordance with the policies and
directives of the board.
As of June 30, 2024, PRIM’sstaff members consisted of an investment team of 31 employees and a finance
and operations team of 31 employees. The investment team works on the management of the PRIT Fund,
and the finance and operations team works on financial reporting for the PRIT Fund and on all other
administrative support.
According to PRIM’s “Investment Policy Statement,”
PRIM’s overall objective [regarding managing the PRIT Fund] is to achieve the highest level of
investment performance that is compatible with its risk tolerance and prudent investment practices.
According to Section 22C of Chapter 32 of the General Laws, by 2040, the PRIT Fund should be fully funded
to meet the Commonwealth’s pension obligations. The PRIT Fund is and continues to be funded through
annual payments made by the Office of the Comptroller of the Commonwealth in accordance with a
funding schedule approved by the Legislature and through the accumulation of employee contributions
and investment returns in the fund.
Retirement Systems
Massachusetts county, city, and town retirement systems that choose to invest in the PRIT Fund may
choose to enroll as either participating retirement systems or purchasing retirement systems.
Participating retirement systems—of which there are 37 as of June 30, 2024—are required by Section 22
of Chapter 32 of the General Laws to invest all of their retirement funds in the PRIT Fund; also, their assets
must remain in the PRIT Fund for at least five years. Purchasing retirement systems—of which there are
62 as of June 30, 2024—can invest part of their funds in the PRIT Fund; also, they are able to contribute
and withdraw assets at will. Both participating and purchasing retirement systems share in the investment
earnings of the PRIT Fund based on their proportionate share of investments.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
4
For local systems that may want to pursue their own individual asset allocation strategy within the PRIT Fund
(as opposed to adopting the allocation strategy of the overall PRIT Fund), retirement boards can elect to invest
in individual asset classes,2 or segments, of the PRIT Fund. This segmentation option gives local retirement
boards flexibility to choose specific asset classes in whatever proportions they believe best suit their needs.
Investment Managers
Section 23(8)(a–c) of Chapter 32 of the General Laws requires PRIM to establish policies regarding
investments with emerging managers. Specifically, Section 23(8) of Chapter 32 of the General Laws states
the following:
b) It shall be the goal of the PRIM board that not less than 20 per cent of investment managers
be minorities, females and persons with disabilities. It shall further be the goal of the PRIM
board to utilize businesses owned by minorities, females and persons with disabilities for not
less than 20 per cent of total contracts awarded [to investment managers].
c) Annually, not later than January 15 of each year, the PRIM board shall file with the house and
senate committee on ways and means and with the joint committee on public service a report
detailing its progress toward implementing the policies and goals outlined [in Section 23(8)(b)
of Chapter 32 of the General Laws]. Such report shall include documentation related to all
minority investment managers considered for investment, including documentation, where
applicable, of the reasons for declining any such investment.
To achieve its goal of having at least 20% of its assets under management (AUM) by minorities, women,
or people with disabilities, PRIM developed the Emerging-Diverse Manager Program.
PRIM’s “Investment Policy Statement” defines emerging managers as investment managers with less than
$2 billion in AUM; this includes managers new to the investment industry, managers experienced in the
investment industry with smaller portfolios, and managers who identify as minorities, women, or people
with disabilities. PRIM aims to invest with emerging managers across all asset classes and to allocate
between 5% and 10% of all new and established investments to these managers. To support this initiative,
PRIM may consult advisors to help identify suitable emerging managers.
PRIM takes several steps to partner with emerging managers, such as allocating capital to emerging
managers, reducing barriers typically faced by managers new to the investing industry, such as startup
cost and lack of brand recognition in the industry, and encouraging emerging managers to work with
2. According to PRIM officials, an individual asset class is a group of investments that have similar characteristics, such as cash
and cash equivalents, bonds, real assets, and stocks.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
5
PRIM. As part of its commitment to understanding and promoting diversity within investment firms, PRIM
conducts an annual request for investment managers to complete the Lenox Park Survey.3 This survey is
meant to be a self-assessment tool, allowing managers to evaluate and report on their diversity practices
and workforce composition.
According to PRIM officials, PRIM evaluates its emerging managers by reviewing a monthly performance
measurement tool called the Net of Fees report, which its custodian bank4 compiles. The Net of Fees
report provides PRIM with detailed information on the calculation of investment returns after accounting
for any associated fees. This report is essential for assessing whether investment managers meet or fall
short of the target rate of return. Such evaluations can influence future funding rates and liabilities.
Additionally, the board approves benchmarks,5 which are recommended by PRIM’s outside consultant.
PRIM uses these benchmarks to compare the performance of each asset class on a monthly basis against
the approved standards. For example, as of June 30, 2024, the benchmark return for total domestic equity
for the quarter was 3.34%. This means that all assets considered to be within the realm of domestic
equities (e.g., the Standard and Poor’s 500) were compared against a return of 3.34% between April 1,
2024 and June 30, 2024. PRIM staff members conduct a quarterly review with each investment manager
to ensure that undue risks, such as performance that is not meeting the benchmarks, do not affect the
PRIT Fund. For these performance review discussions, PRIM uses the monthly Net of Fees report.
However, if any concerns arise—such as poor investment performance, market volatility, excessive
turnover, or challenges with the investment manager or firm—then PRIM may choose to terminate the
contract, liquidate securities, or close the investment manager’s account within the custodian bank system.
Management Fees
According to PRIM officials, PRIM’s custodian bank calculates the performance of the PRIT Fund, the
amount gross of fees, and the amount net of fees.6 PRIM reports the account balance, including the net
3. Lenox Park Solutions is a data analytics company that works with financial service companies, or similar agencies, to gather
information and statistics on diversity, equity, and inclusion for the companies that contract with it.
4. A custodian bank is a financial institution responsible for safeguarding the assets (e.g., bonds or cash) of its clients and
maintaining possession of these assets. These banks often offer related services, such as managing accounts, handling taxes,
and distributing dividends. These banks are generally not engaged in other consumer banking services, such as lending.
5. According to PRIM officials, a benchmark is a standard used to measure the change in asset class performance over a period.
6. According to PRIM officials, the term gross of fees refers to the total value of assets under management before deducting
any investment-related expenses (which include management fees, investment advisory fees, and custodian fees). In
contrast, the term net of fees refers to the value of assets under management after deducting investment-related expenses
from the gross value.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
6
of fees because it accurately reflects the PRIT Fund’s performance after considering all associated fees
and expenses. Furthermore, PRIM employs various methods to ensure transparency regarding its
investment management fees. For example, PRIM annually prepares a budget that details its estimated
investment management fees; also, PRIM quarterly compares budgeted management fee expenses to the
actual expenses. This comparison is included in the documentation for PRIM Administration and Audit
Committee meetings, as well as for board meetings.
Management fees are established during contract negotiations between PRIM and the investment
manager. Each month, PRIM staff members retrieve the net asset value (NAV)7 on the fourth business day
to calculate the management fee according to the contract terms for each investment manager for public
markets. After completing the calculation, PRIM staff members obtain the Net of Fees report, from the
Microsoft SharePoint database for review. PRIM staff members examine the Net of Fees report to ensure
that all new investment managers added during the month are accurately reflected. The report is then
reviewed for reasonableness on a monthly basis for public markets.
According to PRIM officials, management fees for nonpublic markets are calculated either monthly or
quarterly, depending on the investment manager. Each month, PRIM receives an invoice that details the
monthly accruals. PRIM investment staff members review this invoice, comparing it to the estimated and
final NAV and return information to ensure proper accrual of management fees. Below is an example of a
fee schedule.
Assets Under Management Tiered Annual Fee
First $100 Million 0.28%
Next $100 Million 0.16%
Over $200 Million 0.13%
Fees are not paid until they have been reviewed and approved by PRIM’s investment operations team.
Stewardship and Sustainability Committee
PRIM renamed the Environmental, Social, and Governance (ESG) Committee to the Stewardship and
Sustainability Committee in October 2023. The goal of the committee is to ensure the continued success
7. According to the US Securities and Exchange Commission’s website, the NAV is the fund’s total “assets minus its liabilities.”
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Overview of Audited Entity
7
and protection of the PRIT Fund, particularly regarding environmental, social, geopolitical, legal, and
policy issues. These factors are constantly evolving and as such may impact the PRIT Fund.
Stewardship and Sustainability staff members conduct and provide research, participate with external
organizations and partners in ESG initiatives, and provide ESG education for the Stewardship and
Sustainability Committee and PRIM. Additionally, Stewardship and Sustainability staff members assist in
selecting external ESG advisors and work with PRIM investment staff members to help ensure that
approved Stewardship Priorities are integrated into overall investment strategies.
As of December 2024, the Stewardship and Sustainability Committee does not make any investment
recommendations but worked with the Investment Committee and the Real Estate and Timberland
Committee on suggesting ESG-investment strategies. The Investment Committee and the Real Estate and
Timberland Committee remain responsible for making all investment recommendations to PRIM. The
Stewardship and Sustainability Committee is responsible for providing proxy voting recommendations to
PRIM.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
8
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Pension Reserves Investment
Management Board (PRIM) for the period July 1, 2022 through June 30, 2024.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. To what extent did PRIM review the portfolios of emerging managers against its
benchmarks to ensure that the Pension Reserves Investment Trust (PRIT) Fund was
not exposed to undue risks in accordance with Section 5 of PRIM’s “Investment Policy
Statement”?
To a sufficient
extent
2. To what extent did PRIM work toward its goal of having at least 20% of investment
managers who are minorities, women, and people with disabilities, in accordance with
Section 23(8)(b) of Chapter 32 of the General Laws?
To a sufficient
extent
3. To what extent did PRIM review the quarterly calculations of management fees and
investment returns in accordance with the third operational risk category of Section V
(Risk Analysis) of its Investment and Operations Risk Control Document?
To a sufficient
extent
To accomplish our audit objectives, we gained an understanding of the PRIM internal control environment
relevant to our objectives by reviewing PRIM’s internal control plan and applicable policies and
procedures, conducting site visits, and interviewing PRIM management. We also tested the operating
effectiveness of internal controls for the approval of management fees. In addition, to obtain sufficient,
appropriate evidence to address our audit objectives, we performed the procedures described below.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
9
Emerging Managers
To determine to what extent PRIM reviewed the portfolios of emerging managers against its benchmarks
to ensure that the PRIT Fund was not exposed to undue risks in accordance with Section 5 of PRIM’s
“Investment Policy Statement,
” we selected a random, nonstatistical8 sample of 35 from a population of
221 emerging managers who had assets under management (AUM) under $2 billion. For each selected
emerging manager, we reviewed the performance report from the custodian bank and compared the
benchmarks approved by PRIM to the actual fund performance. We also examined PRIM’s quarterly
monitoring summary reports for each emerging manager in our sample to determine whether PRIM
tracked the performance of these managers. Furthermore, we reviewed the archived meeting details from
the customer relationship management software to confirm that PRIM documented the quarterly
performance meeting between PRIM and the investment manager.
For this objective, we found no significant issues during our testing. Therefore, we concluded that, based
on our testing, PRIM reviewed the portfolios of emerging managers against its benchmarks to ensure that
the PRIT Fund was not exposed to undue risks.
Investment Managers
To determine to what extent PRIM worked toward its goal of having at least 20% of investment managers
who are minorities, women, and people with disabilities, in accordance with Section 23(8)(b) of Chapter
32 of the General Laws, we obtained a list of investment managers associated with 730 management fee
accounts, which also included each manager’s net asset data and quarterly status. We then calculated the
percentage of net assets managed by emerging managers and used this information to compare PRIM’s
progress in meeting the 20% goal. Additionally, we developed a forecast in Tableau9 to estimate when
PRIM is expected to achieve its goal of having 20% of emerging managers overseeing the PRIT Fund, based
on trends that were current as of June 30, 2024. We anticipate that PRIM will reach the target of having
20% of its PRIT Fund investment managers with a status of emerging manager by the first quarter of 2030.
The graph below shows both the growth trend of emerging managers who managed the PRIT Fund during
the audit period and the projected date that PRIM is expected to achieve the 20% goal.
8. Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items
are not similar enough, or there are specific items in the population that the auditors want to review.
9. According to the Tableau website, “Tableau is a visual analytics platform.”
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
10
PRIM’s Emerging Manager Growth Trend
For this objective, we found no significant issues during our testing. Therefore, we concluded that, based
on our testing, PRIM worked toward its goal of having at least 20% of investment managers who are
minorities, women, and people with disabilities.
The table below shows the change in net asset value (NAV) by the emerging manager as of June 30, 2023
and June 30, 2024.
7.23
9.25
11.03
12.00
12.72
14.49
16.27
18.06
19.83 20.25
0
5
10
15
20
25
2022 2023 2024 2025 2026 2027 2028 2029 2030
Percentage of
Emerging Managers
End of Fiscal Year
Actual Estimate
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
11
Emerging Manager NAV at
June 30, 2023
NAV at
June 30, 2024
Percentage Change
1315 Capital $ 1,310,971 $ 2,692,254 105%
Adelante Capital Management 20,620,155 22,397,232 9%
ARGA Investment Management, LP 656,069,819 730,598,433 11%
Bayard Asset Management 26,643,025 32,671,909 23%
BentallGreenOak 25,338,754 60,333,902 138%
Bivium Capital Partners, LLC (82,673) (98,961) 20%
Canvas Capital 139,274,856 106,853,778 -23%
Capula Management Limited 552,382,545 590,622,747 7%
Causeway Capital Management LLC 488,273,873 548,632,911 12%
Complus Asset Management Limited 320,896,089 398,665,442 24%
Consilium Investment Management - 28,168,896 100%
Contrarian Capital Management 290,968,027 455,141,400 56%
Ducenta Squared Asset Management 39,058,331 40,182,833 3%
Fithian LLC 31,099,495 31,797,407 2%
Flagship Pioneering 364,511,275 259,005,835 -29%
Frontier Global Partners, LLC 26,386,152 28,020,195 6%
GIA Partners 31,066,918 34,730,663 12%
Global Infrastructure Finance - 21,461,075 100%
HongShan Capital 2,916,028 8,138,927 179%
Integrated Quantitative Investments - 26,233,662 100%
Kah Capital Management - 11,862,270 100%
Kepha Partners 8,421,769 11,307,227 34%
Keytone Ventures 20,930,213 20,381,311 -3%
Kinzie Capital Partners 1,255,330 3,672,438 193%
Knox Lane - 2,901,998 100%
LM Capital Group 30,518,196 32,839,852 8%
Longfellow Investment Management Co., LLC 472,658,598 602,219,528 27%
Loop Capital Asset Management 31,805,955 33,181,280 4%
Maytech Global Investments - 35,316,292 100%
Merit Hill Capital 7,660,005 8,461,353 10%
New Century Advisors, LLC 397,453,638 487,230,049 23%
NewView Capital 1,800,000 7,513,576 317%
Osso Capital 2,851,483 4,161,766 46%
Pacific Alternative Asset Management Company 712,459,640 771,943,728 8%
Polaris Venture Partners 43,879,850 42,010,188 -4%
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
12
The total NAV for all of PRIM’s investment managers (including emerging managers) for fiscal years 2023
and 2024 were $107,283,709,700 and $115,443,314,509, respectively.
Management Fees and Investment Returns
To determine to what extent PRIM reviewed the quarterly calculations of management fees and
investment returns in accordance with the third operational risk category of Section V (Risk Analysis) of
its Investment and Operations Risk Control Document, we took the following actions. We reviewed a list
of management fee accounts for fiscal year 2023 (totaling $94,747,548) and fiscal year 2024 (totaling
$122,069,820), totaling 730 management fee accounts for both years, and reconciled the data back to the
audited financial statements.
We selected a random, nonstatistical sample of 20 management fee accounts and examined quarterly
invoices from the audit period, totaling 160 invoices (20 accounts multiplied by 8 quarters equals 160).
Using the management fee structure defined in the contracts for each management fee account in our
sample, we recalculated the management fees for each quarter and compared them to the invoice
amount. Furthermore, we ensured that the appropriate individuals approved the invoices on behalf of
Emerging Manager NAV at
June 30, 2023
NAV at
June 30, 2024
Percentage Change
Pugh Capital Management, Inc. 463,911,769 588,985,643 27%
Pzena Investment Management LLC 1,437,082,638 1,472,636,865 2%
Red Arts Capital 4,311,116 5,940,941 38%
RhumbLine Advisers 2,085,611,569 3,830,268,006 84%
RV Capital Management Private Ltd 270,751,035 300,082,001 11%
Tidemark 12,309,457 22,540,922 83%
Trio Net Lease I 1,862,606 6,734,657 262%
Vision Capital 1,106,896 2,866,753 159%
Vista Equity Partners 241,127,196 254,305,375 5%
Vistria Housing Fund - 15,000,000 100%
Wing Ventures - 1,404,202 100%
Xponance Asset Management 346,910,789 370,726,745 7%
Redwood Investments 23,957,263 25,212,390 5%
TowerBrook Capital Partners 288,337,210 340,109,372 18%
Total $ 9,925,707,861 $ 12,738,067,268
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
13
PRIM. Additionally, we reviewed PRIM’s approvals of the returns on the investment calculation, which is
included in the Net of Fees report issued by the custodian bank for each quarter.
To understand how management fees affect return on investment, see Appendix. It clearly shows the link
between management fees and investment performance, which is essential for making informed
decisions.
For this objective, we found no significant issues during our testing. Therefore, we concluded that, based
on our testing, PRIM reviewed the quarterly calculations of management fees and investment returns.
We used nonstatistical sampling methods for testing and therefore did not project the results of our
testing to any population.
Data Reliability Assessment
To assess the reliability of PRIM’s emerging and investment manager AUM and the list of total
management fees from the PRIM investment management system, we interviewed PRIM officials who
were knowledgeable about the data.
We reviewed the System and Organization Control reports10 for the audit period, which included, but
were not limited to, testing security management, access controls, configuration management,
segregation of duties, and contingency planning.
For the list of emerging and investment managers with AUM, we obtained and reviewed a list of PRIM’s
total investments, including investments with emerging managers, from the PRIM custodian bank’s
information system, which included the source account names and AUM amounts of PRIM’s investment
managers. We obtained and reviewed a list of the names of investment managers from the relationship
management system. We reconciled PRIM’s performance measurement from the Net of Fees report for
each quarter of the audit period.
Additionally, we compared the list of all of PRIM’s investment managers to the relationship management
software to verify diversity information, such as investment managers who identify themselves as
minorities, women, or people with disabilities. We randomly selected a sample of 10 emerging managers
10. A System and Organization Control report is a report, issued by an independent contractor, on controls about a service
organization’s systems relevant to security, availability, processing integrity, confidentiality, or privacy.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Audit Objectives, Scope, and Methodology
14
from the list of investment and emerging managers and confirmed that the investment managers who
had a status of AUM under $2 billion was either minority- or woman-owned. We tested the list of
investment and emerging managers for any duplicate records.
For the list of management fees, we reconciled the list with the custodian bank and the audited financial
statements for fiscal years 2023 and 2024.
Based on the results of the data reliability assessment procedures described above, we determined that
the information we obtained during the course of our audit was sufficiently reliable for the purposes of
our audit.
Conclusion
Our audit revealed no significant issues that must be reported under generally accepted government
auditing standards. Therefore, we concluded that, during the audit period, PRIM met the relevant criteria
regarding our objectives.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
15
APPENDIX
We inquired with Pension Reserves Investment Management Board (PRIM) officials about management
fees and whether PRIM management compares them to any benchmarks. The PRIM officials told us that
they do not use any industry standard benchmarks regarding fees because the industry is highly
competitive and there are many factors that determine fees. For example, according to PRIM officials, an
actively managed fund could charge higher management fees while a passively managed fund could
charge lower fees. The asset class is another determining factor. The PRIM officials explained that while
they are concerned with fees, they also consider risk versus return versus cost in determining whether
the risks of the funds and the returns are worth the cost of the fees. PRIM officials stated that PRIM does
not make its decision solely on which investment managers charge the lowest fees.
PRIM officials explained that they use a multi-pronged approach in assessing fees. First, they have a
current market landscape for fees for the specific type of service that they can use as a benchmark in
assessing the fees charged by investment managers. Second, PRIM management collaborates with
consultants who assist in determining whether the proposed fees are reasonable. Third, PRIM
management uses an external database containing information about different investment strategies. By
triangulating data from these sources, PRIM management establishes the current fee rates that are
competitive with the market.
Another tool at PRIM management’s disposal is the inclusion of a “most favored nation” clause within its
contracts with investment managers. This is intended to ensure that PRIM is paying competitive fees to
the investment managers by allowing PRIM to adjust the fees if another client that has a similar mandate
to PRIM is paying lower fees. This is meant to allow PRIM to ensure that it receives competitive fees from
investment managers.
PRIM monitors the fees it pays to investment managers by tracking its analysis of management fees in a
spreadsheet. We observed this spreadsheet and noted that PRIM’s analysis charted instances where it
was paying higher or lower fees or paying around the median. PRIM officials acknowledged that
sometimes PRIM accepts higher fees to secure better investment managers.
PRIM officials stated that the assessment of investment managers ultimately hinges on their overall net
performance. This evaluation typically spans a period of three to five years, rather than being done annually.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
16
PRIM examines whether investment managers are meeting their initial objectives and surpassing
expected benchmarks. If managers are not performing well, even before fees are taken into account, it
raises the question of why they would continue to keep that particular investment manager in their
portfolio.
The tables below compare investment managers’ performances for funds in fiscal years 2023 and 2024 by
showing returns on investment (ROIs), both gross and net of fees, and comparing those returns to the
benchmarks used by PRIM. We also included the funds’ net asset value (NAV) to indicate the size of the
funds.
Investment Managers’ Performances by Fund for Fiscal Year 202311
Fund Name NAV $ Gross ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
SSGA S&P 500 $16,831,883 19.66% 19.66% 0.00% 19.67% -0.01%
RhumbLine S&P
500 2,085,612 19.67% 19.66% 0.01% 19.67% -0.01%
Summit Creek 465,503 6.64% 5.78% 0.86% 18.57% -12.79%
Frontier 524,244 22.3% 21.78% 0.52% 6.04% 15.74%
Riverbridge 403,257 18.03% 17.28% 0.75% 18.59% -1.31%
SSGA R2500 1,062,434 13.53% 13.52% 0.01% 13.58% -0.06%
Acadian US
Microcap 234,075 18.52% 14.64% 3.88% 1.37% 13.27%
Brandywine US
Microcap Value 175,987 8.75% 7.97% 0.78% 2.86% 5.11%
Lord Abbett US
Microcap Growth 200,524 24.97% 24.09% 0.88% 12.67% 11.42%
Driehaus Capital
Management 229,982 22.6% 21.69% 0.91% 12.67% 9.02%
SSGA World Ex-US
Standard 2,279,191 17.53% 17.52% 0.01% 17.19% 0.33%
Marathon-London 2,389,236 19.06% 18.69% 0.37% 17.19% 1.50%
Baillie Gifford 1,734,242 19.52% 19.22% 0.30% 17.19% 2.03%
11. We used the Gross and Net of Fees reports for fiscal year 2023 to create this table. All cash accounts were excluded from this
table because there is no ROI for cash balances.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
17
Fund Name NAV $ Gross ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
Mondrian
Investment 765,973 13.88% 13.5% 0.38% 17.19% -3.69%
ARGA 656,070 23.18% 22.7% 0.48% 17.19% 5.51%
Columbia
Threadneedle* 553,931 23.18% 22.93% 0.25% 29.15% -6.22%
Causeway Capital
Management* 488,274 40.03% 39.66% 0.37% 29.15% 10.51%
Pzena Investment
Management* 408,203 37.69% 37.35% 0.34% 29.15% 8.20%
Xponance 347,003 16.21% 15.63% 0.58% 17.19% -1.56%
SSGA World Ex-US
Small 714,473 10.24% 10.23% 0.01% 10.07% 0.16%
Acadian INTL
Small Cap 360,854 12.19% 11.64% 0.55% 10.07% 1.57%
AQR INTL Small
Cap 234,850 17.39% 16.71% 0.68% 10.07% 6.64%
Driehaus Small
Cap 185,884 13.45% 12.74% 0.71% 10.07% 2.67%
Artisan Small Cap 189,717 15.15% 14.2% 0.95% 10.07% 4.13%
AQR Emerging 687,812 3.22% 2.95% 0.27% 1.56% 1.39%
T Rowe Price Em
Equity Strat 245,725 -0.08% -0.72% 0.64% 1.56% -2.28%
Baillie Gifford
EMM 753,431 8.93% 8.35% 0.58% 1.56% 6.79%
Driehaus Capital 867,303 1.53% 1.06% 0.47% 1.56% -0.5%
Pzena Investment 1,028,880 18.37% 17.67% 0.70% 1.56% 16.11%
Acadian 516,995 19.46% 18.39% 1.07% 13.31% 5.08%
Wasatch 167,218 11.4% 10.34% 1.06% 13.31% -2.97%
Xponance 108,004 15.31% 14.63% 0.68% 13.13% 1.50%
Blackrock Passive 1,399,819 -0.86% -0.87% 0.01% -0.94% 0.07%
PIMCO Core 1,412,343 -0.51% -0.71% 0.20% -0.94% 0.23%
Loomis Sayles
Core 1,758,293 -0.43% -0.55% 0.12% -0.94% 0.39%
AFL–CIO Housing
Investment–ETI 131,834 -2.13% -2.23% 0.10% -0.94% -1.29%
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
18
Fund Name NAV $ Gross ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
Pugh 463,912 0.29% 0.12% 0.17% -0.94% 1.06%
New Century 397,454 -0.89% -1.06% 0.17% -0.94% -0.12%
Longfellow 472,659 -0.06% -0.21% 0.15% -0.94% 0.73%
Blackrock–Strips 2,472,331 -10.64% -10.65% 0.01% -10.81% 0.16%
Blackrock–Short
Term 860,431 0.18% 0.17% 0.01% 0.15% 0.02%
Blackrock—Tips 2,547,807 -1.35% -1.36% 0.01% -1.4% 0.04%
Blackrock ILBs 845,813 -3.56% -3.68% 0.12% -3.72% 0.04%
Bivium 70,820 -0.57% -1.03% 0.46% -0.94% -0.09%
Fidelity 477,973 8.16% 7.82% 0.34% 8.97% -1.15%
Loomis Sayles
High Yield 445,239 7.6% 7.16% 0.44% 8.97% -1.81%
Shenkman 422,333 10.28% 9.87% 0.41% 8.97% 0.90%
Eaton Vance 1,063,045 10.35% 9.92% 0.43% 10.71% -0.79%
Voya 1,038,843 9.68% 9.33% 0.35% 10.71% -1.38%
Ashmore 398,492 4.78% 4.12% 0.66% 6.85% -2.73%
PIMCO EMD 394,216 9.15% 8.79% 0.36% 6.85% 1.94%
Bivium 61,547 5.55% 4.89% 0.66% 8.08% -3.19%
Other Credit
Opportunities 1,420,265 4.98% 4.81% 0.17% 7.36% -2.55%
Private Debt 1,110,078 5.92% 4.69% 1.23% 1.91% 2.78%
Private Equity 16,776,635 -3.45% -4.62% 1.17% -3.57% -1.05%
Invesco Core 2,703,497 -6.51% -6.8% 0.29% -3.91% -2.89%
LaSalle 2,545,916 -2.8% -3.11% 0.31% -3.91% 0.80%
AEW 2,499,230 -1.4% -1.04% -0.36% -3.91% 2.87%
CBRE 671,625 -1.08% -1.82% 0.74% -3.91% 2.09%
Stockbridge 613,952 1.36% 0.55% 0.81% -3.91% 4.46%
PRIM–CORE Real
Estate 738,609 4.21% 4.09% 0.12% -3.91% 8.00%
AEW Core Trans 65,953 -6.38% -6.85% 0.47% -3.91% -2.94%
Invesco Trans 60,933 2.84% 2.39% 0.45% -3.91% 6.30%
Non-Core 574,075 -1.65% -2.62% 0.97% -3.91% 1.29%
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
19
Fund Name NAV $ Gross ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
CenterSquare
Global Reit 674,505 -3.04% -3.36% 0.32% -3.74% 0.38%
Brookfield 343,415 -2.72% -3.16% 0.44% -3.74% 0.58%
Cambridge
Associates 32,857 6.52% -12.94% 19.46% 6.93% -19.87%
Forest
Investments 1,358,346 10.22% 9.99% 0.23% 11.31% -1.32%
The Campbell
Group 1,624,479 2.55% 2.73% -0.18% 11.31% -8.58%
Directional Hedge
Funds 2,105,626 4.92% 4.91% 0.01% 9.09% -4.18%
Stable Value
Hedge Funds** 4,475,167 4.64% 4.64% 0.00% 4.15% 0.49%
Pacific Alternative
Asset
Management
Company**
712,460 2.88% 2.88% 0.00% 5.56% -2.68%
Real Assets 736,761 -5.49% -6.06% 0.57% 10.14% -16.20%
Parametric 143,601 10.63% 10.35% 0.28% 13.46% -3.11%
Total Liquidating
Portfolios 37,246 -13.12% -13.18% 0.06% -13.12% -0.06%
Total Assets
Under
Management
$97,057,205
* These investments were not part of the portfolio long enough to determine a one-year performance. As a result, we used the
inception-to-date benchmark and ROI in our calculations. However, they do have one-year performance metrics for 2024.
** For this hedge fund, management fees are taken out before Gross of Fees reports.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
20
Investment Managers’ Performances by Fund for Fiscal Year 202412
Fund Name NAV $ Gross
ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
SSGA S&P 500 $ 19,365,893 24.64% 24.63% 0.01% 24.65% -0.02%
RhumbLine S&P
500 3,554,179 24.66% 24.65% 0.01% 24.65% 0.00%
RhumbLine—EIA* 276,089 0.51% 0.47% 0.04% 7.60% -7.13%
Summit Creek 406,190 5.21% 4.38% 0.83% 9.13% -4.75%
Frontier 617,830 21.78% 21.27% 0.51% 10.93% 10.34%
Riverbridge 297,852 -0.47% -1.13% 0.66% 9.02% -10.15%
SSGA R2500 1,174,075 10.51% 10.5% 0.01% 10.48% 0.02%
Acadian US
Microcap 252,005 4.97% 4.71% 0.26% 1.72% 2.99%
Brandywine US
Microcap Value 140,854 -5.77% -6.48% 0.71% 7.96% -14.44%
Lord Abbett US
Microcap Growth 238,403 6.12% 5.36% 0.76% 2.4% 2.96%
Driehaus Capital
Management 295,775 15.93% 15.06% 0.87% 2.4% 12.66%
SSGA World ExUS Standard 2,110,617 11.04% 11.03% 0.01% 11.12% -0.09%
Marathon—
London 2,640,410 12.8% 12.44% 0.36% 11.12% 1.32%
Baillie Gifford 1,592,452 1.26% 0.99% 0.27% 11.12% -10.13%
Mondrian
Investment 581,788 11.97% 11.59% 0.38% 11.12% 0.47%
ARGA 730,598 11.36% 10.92% 0.44% 11.12% -0.20%
Columbia
Threadneedle 621,381 12.18% 11.88% 0.30% 11.12% 0.76%
Causeway
Capital
Management
548,633 12.36% 11.95% 0.41% 11.12% 0.83%
Pzena
Investment
Management
450,982 10.48% 10.11% 0.37% 11.12% -1.01%
12. We used the Gross and Net of Fees reports for fiscal year 2024 to create this table. All cash accounts were excluded from this
table because there is no ROI for cash balances.
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
21
Fund Name NAV $ Gross
ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
Xponance 370,905 6.9% 6.36% 0.54% 11.12% -4.76%
SSGA World ExUS Small 555,789 7.88% 7.86% 0.02% 7.79% 0.07%
Acadian Intl
Small Cap 368,748 16.28% 15.74% 0.54% 7.79% 7.95%
AQR Intl Small
Cap 282,618 20.34% 19.62% 0.72% 7.79% 11.83%
Driehaus Small
Cap 209,860 12.9% 12.23% 0.67% 7.79% 4.44%
Artisan Small Cap 195,533 3.07% 2.17% 0.90% 7.79% -5.62%
AQR Emerging 806,131 21.83% 20.12% 1.71% 12.46% 7.66%
T Rowe Price EM
Equity Strat 221,099 0.15% -0.48% 0.63% 12.46% -12.94%
Baillie Gifford
EMM 842,650 15.66% 15.04% 0.62% 12.46% 2.58%
Driehaus Capital 946,079 18.49% 17.97% 0.52% 12.46% 5.51%
Pzena Inv 1,021,655 15.55% 14.85% 0.70% 12.46% 2.39%
Acadian 553,953 28.42% 27.3% 1.12% 19.97% 7.33%
Wasatch 199,731 19.44% 18.29% 1.15% 19.97% -1.68%
Xponance 207,230 13.43% 12.68% 0.75% 12.31% 0.37%
Blackrock
Passive 1,334,455 2.66% 2.65% 0.01% 2.63% 0.02%
Pimco Core 1,646,862 3.72% 3.55% 0.17% 2.63% 0.92%
Loomis Sayles
Core 2,032,781 4% 3.88% 0.12% 2.63% 1.25%
AFL–CIO Housing
Investment–ETI 136,421 3.6% 3.48% 0.12% 2.63% 0.85%
Pugh 588,986 3.56% 3.39% 0.17% 2.63% 0.76%
New Century 487,230 2.88% 2.71% 0.17% 2.63% 0.08%
Longfellow 602,220 4.35% 4.19% 0.16% 2.63% 1.56%
Blackrock—Strips 2,811,363 -13.12% -13.13% 0.01% -13.1% -0.03%
Blackrock—Short
Term 1,001,422 4.51% 4.5% 0.01% 4.51% -0.01%
Blackrock—Tips 2,921,676 2.71% 2.7% 0.01% 2.71% -0.01%
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
22
Fund Name NAV $ Gross
ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
Blackrock ILBs 951,295 1.84% 1.71% 0.13% 1.99% -0.28%
Bivium 73,319 3.53% 3.06% 0.47% 2.63% 0.43%
Fidelity High Yield 633,621 10.94% 10.61% 0.33% 10.53% 0.08%
Loomis High Yield 489,075 9.84% 9.4% 0.44% 10.53% -1.13%
Shenkman High
Yield 464,995 10.1% 9.7% 0.40% 10.53% -0.83%
Eaton Vance 1,165,704 10.08% 9.66% 0.42% 11.11% -1.45%
Voya 1,146,083 10.67% 10.32% 0.35% 11.11% -0.79%
Ashmore 442,322 11.73% 11% 0.73% 8.35% 2.65%
Pimco EMD 437,462 10.97% 10.6% 0.37% 8.35% 2.25%
Bivium 88,977 10% 9.3% 0.70% 9.89% -0.59%
Other Credit
Opportunities 1,896,877 14.41% 14.25% 0.16% 11.89% 2.36%
Private Debt 868,115 8.94% 7.78% 1.16% 8.95% -1.17%
Private Equity 17,885,820 8.97% 7.69% 1.28% 5.94% 1.75%
Invesco Core 2,467,380 -7.45% -7.79% 0.34% -6.4% -1.39%
LaSalle 2,368,676 -6.8% -7.15% 0.35% -6.4% -0.75%
AEW 2,431,080 -4.37% -4.64% 0.27% -6.4% 1.76%
CBRE 829,712 1.72% 0.83% 0.89% -6.4% 7.23%
Stockbridge 795,746 5.87% 4.7% 1.17% -6.4% 11.10%
DivcoWest* 122,589 -0.31% -0.31% 0.00% -0.98% 0.67%
PRIM—Core Real
Estate 565,425 -23.8% -24.03% 0.23% -6.4% -17.63%
Invesco Trans 884 -4.53% -4.87% 0.34% -6.4% 1.53%
Non-Core 581,178 -11.8% -12.71% 0.91% -12% -0.71%
CenterSquare
Global REIT 724,106 7.35% 7% 0.35% 5.48% 1.52%
Cambridge
Associates 68,480 4.31% -0.86% 5.17% -3.11% 2.25%
Forest
Investments 1,464,342 14.14% 13.9% 0.24% 9.85% 4.05%
The Campbell
Group 1,727,940 7.47% 7.36% 0.11% 9.85% -2.49%
Audit No. 2025-1333-3S Pension Reserves Investment Management Board
Appendix
23
Fund Name NAV $ Gross
ROI Net ROI
Difference
(Percentage
ROI Spent on
Fees)
Benchmark
Net ROI
Compared to
Benchmark
Directional
Hedge Funds** 2,555,987 16.35% 16.35% 0.00% 12.5% 3.85%
Stable Value
Hedge Funds** 4,714,294 11.52% 11.52% 0.00% 7.5% 4.02%
Pacific
Alternative Asset
Management
Company**
771,944 8.35% 8.35% 0.00% 7.5% 0.85%
Real Assets 692,521 -8.66% -9.2% 0.54% 5.79% -14.99%
Parametric 177,317 23.5% 23.18% 0.32% 14.15% 9.03%
Total Liquidating
Portfolios 25,465 -6.68% -7.38% 0.70% -6.68% -0.70%
Total Assets
Under
Management
$ 105,870,134
* These investments were not part of the portfolio long enough to determine a one-year performance. As a result, we used the
inception-to-date benchmark and ROI in our calculations.
** For this hedge fund, management fees are taken out before Gross of Fees reports.

Official Audit Report – Issued June 22, 2023
Public Employee Retirement Administration
Commission
For the period July 1, 2019 through June 30, 2021
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
June 22, 2023
John Parsons, Esq., Executive Director
Public Employee Retirement Administration Commission
5 Middlesex Avenue, Suite 304
Somerville, MA 02145
Dear Mr. Parsons:
I am pleased to provide to you the results of the enclosed performance audit of the Public Employee
Retirement Administration Commission. As is typically the case, this report details the audit objectives,
scope, methodology, findings, and recommendations for the audit period, July 1, 2019 through June 30,
2021. As you know, my audit team discussed the contents of this report with agency managers. This report
reflects those comments.
I appreciate you and all your efforts at the Public Employee Retirement Administration Commission. The
cooperation and assistance provided to my staff during the audit went a long way toward a smooth
process. Thank you for encouraging and making available your team. I am available to discuss this audit if
you or your team have any questions.
Sincerely,
Diana DiZoglio
Auditor of the Commonwealth
cc: Philip Y. Brown, Esq., Chair of the Public Employee Retirement Administration Commission
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................................................. 6
APPENDIX .............................................................................................................................................................. 9
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the Public Employee Retirement Administration
Commission (PERAC) for the period July 1, 2019 through June 30, 2021.
In this performance audit, we determined whether PERAC approved members’ applications for accidental
and ordinary disability retirement1 within 30 days in accordance with Section 21(1)(d) of Chapter 32 of
the General Laws. We also determined whether PERAC calculated the correct amount of retirement
benefits for excess earners2 in accordance with Section 91A of Chapter 32 of the General Laws.
Our audit revealed no significant instances of noncompliance by PERAC that must be reported under
generally accepted government auditing standards.
1. Accidental disability retirement is when an employee retires because of an injury or illness they sustained at work while
performing their job duties. Ordinary disability retirement is when an employee retires because of an injury or illness they
sustained outside of their job duties.
2. Excess earners are members whose earnings were greater than their regular compensation, had they continued employment
in the grade they held at the time they were approved for disability retirement, plus $15,000.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The Public Employee Retirement Administration Commission (PERAC) was established by Chapter 306 of
the Acts of 1996 to oversee and regulate the 104 public retirement boards in the Commonwealth of
Massachusetts. According to PERAC’s website, its “mission is to provide regulatory oversight and guidance
for the effective, equitable, and ethical operation of the Commonwealth of Massachusetts’ public pension
systems.” PERAC had 54 employees during the audit period. Its nine units (see Appendix) schedule medical
examinations3 for accidental and ordinary disability retirement applications; certify benefit calculations;
perform and review retirement board actuarial valuations;4 and enforce statutory provisions related to
procurements, investments, and disclosures. In addition, PERAC reviews the financials of all public
retirement board records at least once every three years. PERAC also provides training, legal, and technical
assistance to retirement boards.
PERAC promulgated the Standard Rules for Disability Retirement (Section 10 of Title 840 of the Code of
Massachusetts Regulations), governing disability retirement procedures that retirement boards must
follow. As part of its oversight under these requirements, PERAC schedules medical examinations and
determines whether a member qualifies for disability retirement benefits.
In accordance with Section 49 of Chapter 7 of the General Laws, PERAC has seven unpaid commissioners.
The Governor appoints three of these commissioners5 and the State Auditor6 appoints another three
commissioners.7 These six commissioners then choose a seventh commissioner, the chair, who oversees
PERAC’s administrative functions.
3. Retirement boards request medical examinations on behalf of members, while PERAC schedules the medical examinations.
4. PERAC’s website defines actuarial valuation as “a ‘snapshot’ picture of how well the plan is funded at that time. The valuation
compares the plan’s liabilities (current and future payments to be made upon retirement, death, disability, or termination of
employment) with the plan’s assets (both employer and employee contributions credited with investment earnings).”
5. The three members appointed by the Governor are the Governor or their designee, a representative of a public safety union,
and an expert in the investment of funds.
6. Generally accepted government auditing standards require that organizations be free from organizational impairments to
independence with respect to the entities they audit. To this end, we are disclosing that under Section 49 of Chapter 7 of the
General Laws, PERAC’s board consists of seven members, including the State Auditor or their designee. This disclosure is
made for informational purposes only, and this circumstance did not interfere with our ability to perform our audit work and
report its results impartially.
7. The three members appointed by the State Auditor are the State Auditor or their designee, the president of the
Massachusetts American Federation of Labor and Congress of Industrial Organizations or their designee, and a representative
of the Massachusetts Municipal Association.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Overview of Audited Entity
3
Application and Determination Process
Upon receiving an application for disability retirement from any member of a public retirement board in
Massachusetts, the retirement board requests that PERAC schedule a medical examination of the member
by a medical panel made up of three physicians appointed by PERAC, based on the regional location of
the member. PERAC appoints three physicians to examine the member whose retirement is under
consideration. The physicians of this medical panel must determine, either jointly or separately, whether
they find that the member applying for disability retirement is unable to perform the essential duties of
their job and whether the member’s disability is likely to be permanent. The physicians who conduct the
medical examination(s) must respond to all questions on the Medical Panel Certificate and complete a
narrative report, a Certification of Medical Panel Finding, and either an Accidental Disability Certificate or
Accidental Disability Presumption Certificate (if appropriate). These physicians must then submit all these
documents to PERAC within 60 days of the examination(s). PERAC must forward the medical panel’sreport
to the retirement board within five days of receipt. The retirement board then has the option to request
more information or clarification from the medical panel. Within 30 days of receiving the report, the
retirement board notifies the member of the medical panel’s findings and provides copies of all the
documents completed by the medical panel.
If the medical panel concludes that the member applying for disability retirement is incapable of
performing their essential job duties, the retirement board then determines whether to approve or deny
the application for disability retirement.
If the application is approved, the retirement board sends a Disability Transmittal Request to PERAC.
PERAC’s Legal Unit must either approve or remand the Disability Transmittal Request, then return it to
the retirement board within 30 days of receipt; otherwise, the request is automatically considered
approved.
 If PERAC’s Legal Unit approves the Disability Transmittal Request, PERAC’s retirement
management system automatically generates a Disability Transmittal Letter, which PERAC sends
to the retirement board, alerting the board to the request’s approval and listing the documents
needed to calculate the disability retirement allowance that the member should receive.
 If PERAC’s Legal Unit remands an application back to the retirement board, the retirement board
may attempt to rectify any issues and resubmit the application, or the retirement board may deny
the application and provide the member with a notice of their right to appeal.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Overview of Audited Entity
4
Members’ Disability Retirement Allowance Calculations and Approvals
Once the retirement board receives notification from PERAC that the member has been approved for
disability retirement, the retirement board performs all calculations to determine the member’s monthly
disability retirement allowance. Once the retirement board calculates the member’s monthly disability
retirement allowance, it sends the Disability Transmittal Letter, calculation worksheet, and other
retirement documents (which vary by type of disability) to PERAC for approval. Pursuant to Section 21(3)
of Chapter 32 of the General Laws, PERAC must approve disability retirement allowances calculated by
retirement boards.8
To ensure that the disability retirement allowance calculated by the retirement board is accurate, PERAC
reviews the documents provided and performs its own calculations. If the disability retirement allowance
amounts match, PERAC mails an approval letter to the member applying for disability retirement and
forwards the approval letter to the retirement board. If the disability retirement allowance amounts do
not match, PERAC reviews the retirement documents to determine why there is a discrepancy and
resolves it. PERAC does not approve disability retirement allowance calculations until any discrepancies
between PERAC’s and the retirement board’s calculations are resolved. When the retirement board
receives the approval letter from PERAC, it begins sending payments to the member.
Annual Statements of Earned Income
Section 91A of Chapter 32 of the General Laws requires every person receiving a disability retirement
allowance to submit an Annual Statement of Earned Income, certifying the full amount of their earned
income during the preceding year, along with all pertinent Forms W-2, Forms 1099, and other tax forms
requested by PERAC, on or before April 15 of every year.
According to PERAC management, once the retired member submits the Annual Statement of Earned
Income and supporting financial records, PERAC’s Fraud Unit reviews the records (e.g., Forms W-2 and
Forms 1099) to determine whether the member exceeded their allowable earnings. For members who
have reported earnings in an amount that may result in an adjustment to their disability retirement
allowance, PERAC sends a system-generated task, through their retirement management system, to the
8. The Massachusetts State Employees’ Retirement System and the Mass Teachers Retirement Board are not required to submit
calculations to PERAC for approval. Pursuant to Section 21(3)(a) of Chapter 32 of the General Laws, PERAC reviewed and
approved the automated systems used to calculate disability retirement allowances at each of these retirement boards;
therefore, disability retirement allowance calculations by those systems are considered to be approved by PERAC.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Overview of Audited Entity
5
member’s retirement board, requesting the figures needed to determine the refund amount, if any. Once
the retirement board inputs the figures directly into the retirement management system, PERAC can
immediately do the calculation. If the member did not exceed their allowable earnings, no further action
is necessary. If the member’s earnings were over $15,000 more than what their pay would be in the
position held when they retired, had they not retired, the member must refund the amount in excess of
their disability retirement allowance to PERAC. If PERAC determines that the member is an excess earner,
PERAC mails a letter, called a 91A Excess, to the member, explaining that if they do not refund the excess
amount, their disability retirement allowance will be withheld until the retirement board recovers the
amount owed.
If a member who receives a disability retirement allowance does not submit the Annual Statement of
Earned Income and the supporting financial records to PERAC and does not show good cause for not
submitting them, the member’s retirement board terminates their right to a disability retirement
allowance for noncompliance. According to Section 91A of Chapter 32 of the General Laws, “Prior to any
termination or reduction of benefits . . . the member shall be given a written notice [of the decision] and
an opportunity to be heard by the retirement board.”
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Audit Objectives, Scope, and Methodology
6
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the Public Employee Retirement
Administration Commission (PERAC) for the period July 1, 2019 through June 30, 2021.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer and the
conclusion we reached regarding each objective.
Objective Conclusion
1. Does PERAC approve members’ applications for accidental and ordinary disability
within 30 days in accordance with Section 21(1)(d) of Chapter 32 of the General Laws?
Yes
2. Does PERAC calculate the correct amount of retirement benefits for excess earners in
accordance with Section 91A of Chapter 32 of the General Laws?
Yes
To achieve our audit objectives, we gained an understanding of PERAC’s internal control environment
related to our audit objectives by reviewing the agency’s policies, procedures, and internal control plan,
as well as interviewing PERAC’s staff members and management. We evaluated the design and
implementation of the internal controls related to the approval of calculations for excess earners.
To test the effectiveness of internal controls related to the approval of excess earner calculations during
the audit period, we selected a sample of 50 out of the 145 individual excess earners. For each excess
earner in our sample, we inspected their 91A Excess and confirmed that PERAC’s assistant deputy director
signed it.
To determine whether PERAC approved members’ applications for accidental and ordinary disability
within 30 days, in accordance with Section 21(1)(d) of Chapter 32 of the General Laws, we selected a
nonstatistical, random sample of 60 members whose applications were approved during the audit period,
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Audit Objectives, Scope, and Methodology
7
from the population of 722 members. We reviewed application records to determine whether PERAC’s
Legal Unit approved Disability Transmittal Requests within 30 days of receipt from the retirement board.
To determine whether PERAC calculated the correct amount of retirement benefits for excess earners in
accordance with Section 91A of Chapter 32 of the General Laws, we reviewed two lists of excess earners,
totaling 180 members, for calendar years 2019 and 2020. To determine who the 145 individual excess
earners during our audit period were, we consolidated the two lists of excess earners and removed
duplicate members who appeared on both lists. We also obtained from PERAC the Excess Earner Reports
for calendar years 2019 and 2020, which display information from all financial documents submitted with
members’ Statements of Earned Income and other documents used to calculate excess earnings. From
the population of 145 individual excess earners, we selected a nonstatistical, random sample of 35 excess
earners and recalculated the excess earnings of each based on the income information provided on the
Excess Earner Reports. To determine each excess earner’s refund amount, we found the difference
between each excess earner’s regular compensation, had they continued employment in the grade held
by them at the time they were approved for disability retirement, plus $15,000, minus their disability
retirement allowance and the excess earner’s allowable earnings, minus their reported income. We then
compared our calculations to the Excess Earner Reports to ensure that PERAC calculated and determined
excess earnings in accordance with Section 91A of Chapter 32 of the General Laws.
We used nonstatistical sampling methods for testing and therefore could not project the results of our
testing to the populations.
Data Reliability Assessment
To determine the reliability of the data used in the procedures described above, we performed the
following tests:
 We reviewed the System and Organization Control reports9 that covered the periods April 1, 2019
through March 31, 2020; April 1, 2020 through March 31, 2021; and October 1, 2020 through
September 30, 2021. We verified that the System and Organization Control reports described
testing of certain information system general controls (access controls, application controls,
configuration management, contingency planning, and segregation of duties) and that they were
tested without exceptions.
9. A System and Organization Control report is a report on controls about a service organization’s systems relevant to security,
availability, processing integrity, confidentiality, or privacy issued by an independent contractor.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Audit Objectives, Scope, and Methodology
8
 We performed an application process walkthrough on the staging environment of PERAC’s
retirement management system, where we observed access controls, application controls, and
segregation of duties for the disability retirement application process. We had PERAC’s
information technology director create a fake retiree (to represent a member with a disability)
and run their application through PERAC’s test system from when it initially receives a medical
examination request from the retirement board to when PERAC approves or remands the
application for disability retirement.
 We tested the reliability of the data on the excess earner lists and Excess Earner Reports by
ensuring that there were no duplicates or blank fields.
 We tested the accuracy of the member application data from PERAC’s retirement management
system by obtaining and inspecting a list of all members (2,738)10 who applied for disability
retirement during the audit period. Since only members whose applications were approved can
be moved to the calculation of benefit approval process, we filtered the list to include just the 722
members whose applications were approved during the audit period. From this population of 722
members, we selected a random sample of 20 members and added their disability retirement
allowance amounts to an Excel spreadsheet from PERAC’s retirement management system. We
then traced the disability retirement allowance amounts back to the hardcopy Disability
Transmittal Letters and the calculation worksheetsfrom PERAC’s filing shelvesfor these members.
Based on the results of our data reliability assessments, we determined that the information obtained for
our audit period was sufficiently reliable for the purposes of our audit objectives.
Conclusion
Our audit revealed no significant instances of noncompliance by PERAC that must be reported under
generally accepted government auditing standards.
10. This list contains members whose applications were approved during (722) and after (257) the audit period and members
whose applications were denied (1,759) for disability retirement during the audit period.
Audit No. 2022-1315-3S Public Employee Retirement Administration Commission
Appendix
9
APPENDIX
According to its website, the Public Employee Retirement Administration Commission is divided into the
following nine units.
Actuarial Unit
Ensures the integrity of long-term pension funding for the systems
Administration Unit
Responsible for day-to-day operations, fiscal and procurement policies, and agency budget
Audit Unit
Responsible for system audits
Communications Unit
Manages website, publications, external/internal communications, and public events
Compliance & Investments Unit
Provides regulatory oversight of retirement [board] investment portfolios
Disability Unit
Oversees the disability retirement process for all systems
Fraud Prevention Unit
Investigates reported fraud
Information Systems Unit
Provides technical support for agency and manages self-service portal
Legal Unit
Provides legal representation for the Commission

Official Audit Report – Issued May 22, 2020
State Ethics Commission
For the period July 1, 2017 through June 30, 2019
State House Room 230  Boston, MA 02133  auditor@sao.state.ma.us  www.mass.gov/auditor
erely,
anne M. Bump
May 22, 2020
Mr. David A. Wilson, Executive Director
State Ethics Commission
1 Ashburton Place, Sixth Floor, Room 619
Boston, MA 02108
Dear Mr. Wilson:
I am pleased to provide this performance audit of the State Ethics Commission. This report details the
audit objectives, scope, and methodology for the audit period, July 1, 2017 through June 30, 2019. My
audit staff discussed the contents of this report with management of the agency.
I would also like to express my appreciation to the State Ethics Commission for the cooperation and
assistance provided to my staff during the audit.
Sinc
Suz
Auditor of the Commonwealth
cc: Ms. Maria J. Krokidas, Chair, State Ethics Commission
Audit No. 2020-1053-3S State Ethics Commission
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY ......................................................................................................................... 2
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY............................................................................................... 4
Audit No. 2020-1053-3S State Ethics Commission
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the State Ethics Commission (SEC) for
the period July 1, 2017 through June 30, 2019. The purpose of our audit was to determine whether SEC
complied with Sections 27 and 28 of Chapter 268A of the General Laws, which require SEC to ensure
that elected state and county employees adhere to requirements related to the Conflict of Interest Law
summary acknowledgment receipt and certification for SEC’s online training program.1
Our audit revealed no significant instances of noncompliance by SEC that must be reported under
generally accepted government auditing standards.

1. The training program provides employees who are subject to the Conflict of Interest Law with information on how the law
governs situations where their public responsibilities conflict with personal interests. These situations include receipt of
gifts, favoritism toward family or friends, and decisions benefiting personal financial interests. The training explains how to
recognize and properly address these situations.
Audit No. 2020-1053-3S State Ethics Commission
Overview of Audited Entity
2
OVERVIEW OF AUDITED ENTITY
The State Ethics Commission (SEC) was established in 1978 under Section 2 of Chapter 268B of the
Massachusetts General Laws. SEC consists of five members: three members, including the chairperson,
appointed by the Governor; one by the Attorney General; and one by the Secretary of the
Commonwealth. The commissioners serve five-year terms, and there cannot be more than three
members from the same political party at any time. SEC employs an executive director who oversees its
administrative operations.
As of June 30, 2019, SEC had 25 employees. SEC received state appropriations of $2,093,969 and
$2,254,948 for fiscal years 2018 and 2019, respectively. Its offices are located at 1 Ashburton Place in
Boston.
SEC’s website states,
The State Ethics Commission is an independent state agency that administers and enforces the
provisions of the conflict of interest law and financial disclosure law. . . .
The State Ethics Commission serves the public by fostering integrity in government. The
independent agency provides free advice to all public employees on the conflict of interest law,
and civilly enforces this law. [Any citizen] can contact the Commission to obtain legal advice, file
a complaint, obtain a statement of financial interest or conflict of interest law disclosure form, or
complete statutory conflict of interest law requirements.
Conflict of Interest Law Summary Acknowledgment Receipt
SEC is responsible for ensuring that all elected state and county employees comply with Section 27 of
Chapter 268A of the General Laws by ensuring that they are given a summary of the Conflict of Interest
Law and that they file an acknowledgment receipt with SEC. Accordingly, SEC maintains a list of all
elected state and county employees, which includes each elected employee’s name, agency, position,
and email address. This list is updated as needed to account for general and special election results and
temporary appointments.2 At the beginning of each calendar year, SEC sends an email using Lyris
ListManager3 to all elected state and county employees. The email includes a summary of the Conflict of
Interest Law, an acknowledgment receipt, and instructions to send the acknowledgment receipt back to

2. A temporary appointment is the result of a vacancy of an elected office due to death, retirement, or resignation. Depending
on the date of vacancy, county and municipal boards may choose to fill a position temporarily until a special election date
can be determined.
3. ListManager is an Internet application used to manage email lists and the delivery of emails to large numbers of recipients.
Audit No. 2020-1053-3S State Ethics Commission
Overview of Audited Entity
3
SEC within 30 days. Employees can send the acknowledgment receipt by signing a copy of it and
emailing it back to SEC as a Portable Document Format (PDF) file or by replying to SEC’s email. On
receiving the acknowledgment receipt, SEC logs the date it was received and the email address of the
respondent in a spreadsheet. SEC monitors the responses and makes as many as three attempts,
through either email or phone, to contact non-responders requesting their compliance. At the end of
the fiscal year, SEC’s public education and communications division chief reviews the results of SEC’s
compliance efforts with the executive director.
Conflict of Interest Law Online Training Program
SEC is responsible for ensuring that all elected state and county employees comply with Section 28 of
Chapter 268A of the General Laws, including its requirements related to certification for SEC’s online
training program. During even-numbered calendar years, only newly appointed or special elected state
and county employees are responsible for completing the program; they must do so within 30 days of
becoming elected employees and every two years thereafter. For odd-numbered calendar years, at the
beginning of the year, SEC sends an email using ListManager to all elected state and county employees.
The email requests that recipients complete the online training program by a specified due date4 and
email their online training program certifications to SEC. Employees can email their certificates either as
PDF files or as photographs of printed certifications. SEC logs the date and email address for each
response in a spreadsheet; monitors the responses; and makes as many as three attempts, through
email or phone, to contact non-responders requesting their compliance.

4. The due date for 2019 was April 5.
Audit No. 2020-1053-3S State Ethics Commission
Audit Objectives, Scope, and Methodology
4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the State Ethics Commission (SEC) for
the period July 1, 2017 through June 30, 2019.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer and the
conclusion we reached regarding each objective.
Objective Conclusion
1. Does SEC ensure that elected state and county employees provide signed
acknowledgment receipts for the Conflict of Interest Law summary as required by
Section 27 of Chapter 268A of the General Laws?
Yes
2. Does SEC ensure that elected state and county employees complete the online
training program as required by Section 28 of Chapter 268A of the General Laws?
Yes
To achieve our objectives, we gained an understanding of SEC’s internal control environment related to
our audit objectives by reviewing applicable laws, regulations, and agency policies and procedures, as
well as conducting inquiries with SEC’s staff and management.
Additionally, we performed the procedures described below.
Acknowledgment Receipt for Conflict of Interest Law Summary
We obtained SEC’s list of all 688 elected state and county employees who were subject to the
requirements of Section 27 of Chapter 268A of the General Laws during the audit period. SEC manually
updates this list with the received date for each employee’s annual acknowledgment receipt for the
Conflict of Interest Law summary. We examined 100% of the list for evidence of each employee’s annual
acknowledgment receipt. We obtained the acknowledgment receipts and noted each one’s document
Audit No. 2020-1053-3S State Ethics Commission
Audit Objectives, Scope, and Methodology
5
type (Portable Document Format file or email reply), date, and sender email address. We assessed the
timeliness of acknowledgment receipts by calculating the difference between the due date and receipt
date, noting the number and percentage of respondents who did not respond within 30 days as
required.
The instances of noncompliance that were identified in this area were deemed insignificant by the audit
team and were discussed with SEC management. SEC officials stated that they would be implementing
process improvements to ensure that all acknowledgment receipts were submitted in a timely manner.
Certification of Online Training Program Completion
We obtained SEC’s list of all 358 elected state and county employees who were subject to the
requirements of Section 28 of Chapter 268A of the General Laws during the audit period. SEC manually
updates this list with the received date for each employee’s certification of online training program
completion. We examined 100% of the list for evidence of online training completion. We obtained the
certifications and noted the completion date on each one to determine whether there were any
respondents who did not meet the due date.
Data Reliability
To determine the reliability of SEC’s lists of elected state and county employees, we interviewed the
management personnel who were responsible for the source data and used electronic spreadsheet
functionality to identify hidden cells and rows or other irregularities. We noted no exceptions. To
determine the completeness of the SEC lists, we compared the names and positions on the lists to the
Office of the Secretary of the Commonwealth’s (SOC’s) Public Document 43, which lists the results of
state and county elections certified by SOC and maintained on its website. We determined that the SEC
lists of elected state and county employees were sufficiently reliable for the purpose of this audit.

Official Audit Report – Issued December 30, 2024
University of Massachusetts Amherst
For the period July 1, 2022 through June 30, 2023
State House Room 230  Boston, MA 02133  auditor@massauditor.gov  www.mass.gov/auditor
December 30, 2024
Dr. Javier Reyes, Chancellor
University of Massachusetts Amherst
301 Whitmore Building
Amherst, MA 01003
Dear Dr. Reyes,
I am pleased to provide to you the results of the enclosed performance audit of the University of
Massachusetts Amherst. As is typically the case, this report details the audit objectives, scope, methodology,
findings, and recommendations for the audit period, July 1, 2022 through June 30, 2023. As you know, my
audit team discussed the contents of this report with university managers. This report reflects those
comments.
I appreciate the overall efforts of you and your staff at the University of Massachusetts Amherst, who I am
told demonstrated the utmost professionalism. I am disappointed, however, with reports I received from my
team that the UMass Internal Audit Unit exhibited a tremendous lack of cooperation throughout the audit. I
am hopeful that we can resolve this issue, moving forward, so that our audit teams can look forward to
working together to make government work better. I am available to discuss this audit if you or your team
has any questions.
Best regards,
Diana DiZoglio
Auditor of the Commonwealth
Audit No. 2024-0213-3I University of Massachusetts Amherst
Table of Contents
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................... 1
OVERVIEW OF AUDITED ENTITY............................................................................................................................. 4
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY ............................................................................................... 10
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE...................................................................................... 15
1. The University of Massachusetts Amherst’s website is not fully accessible for all Massachusetts residents
and users. ................................................................................................................................................. 15
2. The University of Massachusetts Amherst’s learning management system, Blackboard, is not fully
accessible for all students. ........................................................................................................................ 16
3. The University of Massachusetts Amherst has not implemented workforce cybersecurity awareness
training. .................................................................................................................................................... 21
OTHER MATTERS ................................................................................................................................................. 23
1. The University of Massachusetts Amherst can further enhance the accessibility of its marketing pages..... 23
2. The University of Massachusetts Amherst can further enhance the accessibility of its cookie banner. ....... 24
3. The University of Massachusetts Amherst does not maintain a full site map of its umass.edu website. ... 25
Audit No. 2024-0213-3I University of Massachusetts Amherst
List of Abbreviations
ii
LIST OF ABBREVIATIONS
ADA Americans with Disabilities Act
CIS Center for Internet Security
LMS learning management system
UMass University of Massachusetts
URL uniform resource locator
W3C World Wide Web Consortium
WCAG Web Content Accessibility Guidelines
WISP Written Information Security Policy
Audit No. 2024-0213-3I University of Massachusetts Amherst
Executive Summary
1
EXECUTIVE SUMMARY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of the University of Massachusetts (UMass) Amherst for the
period July 1, 2022 through June 30, 2023.
The purpose of this performance audit was to determine whether UMass Amherst’s website and its
learning management system (LMS), Blackboard, adhered to the accessibility standards established by the
Web Content Accessibility Guidelines (WCAG) 2.1 for user accessibility, keyboard accessibility, navigation
accessibility, language, error identification, and color accessibility. WCAG ensures that all users, regardless
of ability, can access the content and functions of UMass Amherst’s website and LMS. Further, it supports
UMass Amherst’s commitment to equal access for students, faculty, and visitors, fulfilling legal1 and ethical
standards.
Additionally, we determined whether UMass Amherst ensured that its employees completed
cybersecurity awareness training in accordance with its adopted Center for Internet Security controls.
Cybersecurity awareness is important because adhering to internet security policies helps UMass Amherst
demonstrate the university’s commitment to protecting sensitive information.
Below is a summary of our findings, the effects of those findings, and our recommendations, with links to
each page listed.
Finding 1
Page 15
UMass Amherst’s website is not fully accessible2 for all Massachusetts residents and
users.
Effect Broken or faulty hyperlinks limit users from having equitable access to critical information
and key online services offered by UMass Amherst and increase the likelihood that
Massachusetts residents and students will either access outdated or incorrect
information or be directed to webpages that no longer exist. Videos that lack sufficient
captioning prevent users from engaging with video content in a meaningful way (e.g., lack
of context provided through dialog and important sounds). Additionally, hyperlinks
without sufficient contrast with the surrounding text negatively impact the user
experience by making it difficult to locate other relevant information.

1. Title II of the Americans with Disabilities Act requires that state universities’ and colleges’ websites be accessible.
2. Accessible is defined in our report as compliance with WCAG.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Executive Summary
2
Recommendation
Page 16
The university should continually review its webpages to ensure that all hyperlinks lead
to related information and have sufficient contrast with the surrounding text in order to
provide equitable access to critical information and services offered online by UMass
Amherst. The university should also adopt procedures to ensure that videos have
captioning features enabled when posted to the umass.edu website.
Finding 2
Page 16
UMass Amherst’s learning management system (LMS), Blackboard, is not fully accessible
for all students.
Effect The above instances of noncompliance have the following effects on the user:
Broken or Faulty Hyperlinks
• This can limit Blackboard users from having equitable access to critical information
and key online services offered on the LMS.
• This can increase the likelihood that users will either access outdated or incorrect
information or be directed to webpages that no longer exist.
Missing Search Bars
• This can prevent users from navigating to other relevant information.
Hyperlinks without Identifiable Markers or Sufficient Contrast
• This can negatively impact the user experience by making it difficult to locate other
relevant information.
Zoom in to 200% and 400%
• Users will be unable to read Blackboard content.
Bypass Blocks
• Users will be unable to navigate to the important main content of a page quickly.
Portrait Mode
• Users will be unable to interact with their course content on their mobile devices
effectively.
Keyboard Accessibility/Navigation
• Users who have mobility issues will be unable to access certain features and content.
Titles
• Users with screen readers will lose comprehension of the feature.
Language Attributes
• The lack of language attributes will prevent screen readers from reading the content
to users.
Error Identification
• If users are not informed of errors when making inputs on data entry, it means that
users will be unable to identify their errors and retrieve the content they need.
Recommendation
Page 20
UMass Amherst should review the accessibility statements and reports of its LMS vendor
to determine instances of WCAG noncompliance. UMass management should work with
its LMS vendor to ensure that any potential instances of WCAG noncompliance are
resolved.
Finding 3
Page 21
UMass Amherst has not implemented workforce cybersecurity awareness training.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Executive Summary
3
Effect If UMass Amherst does not educate all employees on their responsibility to protect its
information assets by requiring cybersecurity awareness training, then UMass Amherst is
exposed to an elevated risk of cybersecurity attacks, which may cause financial and/or
reputational losses.
Recommendation
Page 22
UMass Amherst management should update its WISP to require all employees to
complete cybersecurity training at hire and at least annually thereafter. UMass Amherst
should also devise means by which it can enforce and monitor compliance with an
updated training policy. UMass Amherst should enroll all of its employees, contractors,
and interns in cybersecurity awareness training.
In addition to the conclusions we reached regarding our audit objectives, we also identified issues not
specifically addressed by our objectives. For more information, see Other Matters.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
4
OVERVIEW OF AUDITED ENTITY
The University of Massachusetts (UMass) Amherst is a member of the Massachusetts public higher
education system, which consists of 15 community colleges, nine state universities, and five UMass
campuses. In 1964, UMass Amherst became one of the five public institutions of higher learning in the
UMass system, in accordance with Chapter 75 of the Massachusetts General Laws. UMass is led by a
president who oversees the UMass system and by a chancellor at each UMass campus. It is also governed
by a board of trustees composed of 22 members, with 17 members who are appointed by the Governor
for five-year terms and 5 UMass students who are elected by the student body for one-year terms. The
board shapes general policies that govern all five UMass campuses. The chancellor of UMass Amherst, as
the administrative head of the campus, reports to the president and is supported by vice chancellors, a
provost, and the director of athletics.
As of fall 2023, UMass Amherst had a total enrollment of 31,810 students (23,936 undergraduate and
7,874 graduate students) and approximately 9,373 employees (6,135 full-time and 3,238 part-time
employees). According to Section 7 of Chapter 75 of the General Laws, “The [UMass system] trustees shall
prepare and submit a detailed budget in such form and manner as the governor, secretary and general
court may direct.” UMass Amherst had an operating budget of $1,547,122,000 for the 2023 fiscal year
and $1,458,822,000 for the 2022 fiscal year. UMass Amherst had state appropriations of $421,771,000
and $448,412,000 for fiscal years 2022 and 2023, respectively.
Website Accessibility
Americans with Disabilities Act
In 1990, the Americans with Disabilities Act (ADA), a comprehensive civil rights law prohibiting
discrimination based on disability, came into effect. Title II of the ADA covers state-funded programs
such as universities, community colleges, and career and technical education programs, including all
activities of state and local governments, regardless of whether these entities receive federal financial
assistance. (See 42 US Code § 12131B65). More recently, the Justice Department filed a proposed
consent decree to resolve allegations that Miami University in Oxford, Ohio, violated the ADA by using
inaccessible classroom technologies and other technologies. As part of the consent decree, Miami
University had to ensure that its web content and learning management systems conform with Web
Content Accessibility Guidelines (WCAG) 2.0 AA standards. Additionally, the university was required
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
5
to meet with every student who has a disability in order to develop an accessibility plan and procure
web technology or software that best meets various accessibility standards.
WCAG
The World Wide Web Consortium (W3C), an international organization that oversees internet
standards, released WCAG 1.0 in 1999. These guidelines aimed to offer directions on enhancing the
accessibility of web content for people with disabilities. In 2008, W3C published WCAG 2.0. In 2018,
W3C published WCAG 2.1, which was built on WCAG 2.0 to improve web accessibility on mobile
devices and to further improve web accessibility for people with visual impairments and cognitive
disabilities.
Progression of Internet Accessibility Standards
How People with Disabilities Use the Web
According to W3C, people with disabilities use assistive technologies and adaptive strategies specific
to their needs to navigate web content. Examples of assistive technologies include screen readers,
which read webpages aloud for people who cannot read text; screen magnifiers for individuals with
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
6
low vision; and voice recognition software for people who cannot (or do not) use a keyboard or
mouse. Adaptive strategies refer to techniques that people with disabilities employ to enhance their
web interaction.3 These strategies might involve increasing text size, adjusting mouse speed, or
enabling captions. To make web content accessible to people with disabilities, developers must
ensure that various components of web development and interaction work together. This includes
text, images, and structural code; users’ browsers and media players; and various assistive
technologies.
UMass Amherst made efforts to create and maintain an accessible website in the following ways.
Currently at the university, the Assistive Technology Center team performs accessibility reviews of
webpages before they are published. Additionally, the university uses third-party software (called
SiteImprove) to run weekly scans of the umass.edu website to identify accessibility issues.

3. Web interaction refers to the various actions that users take while navigating and using the internet. It encompasses a wide
range of online activities, including, but not limited to, clicking on links, submitting forms, posting comments on webpages,
and engaging with web content and services in other forms.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
7
Common Accessibility Features of a Website*
* This webpage was modified to fit in our report.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
8
Blackboard LMS
According to UMass, Blackboard Learn Original is the third-party vendor learning management system
(LMS)4 chosen by the university to help instructors provide effective and engaging learning in the
classroom. The LMS allows instructors to conduct their courses either partly or entirely online and allows
students to undertake a variety of actions, including taking tests, submitting homework assignments,
watching lecture videos, keeping track of their grades, and engaging in student discussions. Blackboard’s
website indicates that its products are generally designed and developed in alignment with WCAG 2.1
Level AA success criteria.
In spring 2023, UMass Amherst announced that it had selected a new LMS called Canvas. We did not test
Canvas because it was not fully implemented by the university during the audit period. The university
made this transition to address accessibility concerns, increase inclusivity for mobile users, and further
integrate the learning and teaching experience.
Cybersecurity Awareness Training
Starting in 2008, in reaction to significant data losses faced by organizations in the US defense sector, the
Center for Internet Security (CIS) introduced best practice guidelines for computer security known as CIS
Controls. There are 18 controls; they are a set of prioritized cybersecurity actions that organizations can
implement to protect against the most common cyber threats. CIS Control 14 (Security Awareness and
Skills Training) focuses on the importance of developing and sustaining a security awareness program
aimed at shaping employee behavior to be more security minded and adequately trained, thereby
minimizing cybersecurity risks to the organization.
In the 2010s, the transition to cloud computing led to an increased focus on cloud security. At the same
time, the rise of increased cyber threats highlighted the necessity for cooperative strategies to combat
emerging digital challenges. As a result of various data breaches and other cyberattacks, there was an
effort to invest in cybersecurity measures to protect sensitive information across organizations. The
absence of cybersecurity training poses one of the highest risks an organization can face, as untrained
employees are often the weakest link in its security defenses. Recognizing this, organizations have

4. A learning management system, or LMS, is a web-based application that functions like a website. Instructors and students
can access the classes they are assigned to.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Overview of Audited Entity
9
prioritized investments in cybersecurity training to educate their workforce about potential cyber threats,
such as phishing scams and malware.
In 2010, the UMass board of trustees passed a new Information Security Policy (Doc. T10-089), which
commits the university to adopt controls modeled on ISO 27002.5 This includes controls requiring
employees to receive cybersecurity awareness training. According to the university’s President’s Office,
in the intervening years, the university adopted CIS Controls, which require the university’s campuses to
maintain a cybersecurity awareness training program across its entire workforce.
Currently, UMass Amherst has not updated its policies to require all of its employees to complete
cybersecurity awareness training, and it does not enroll all of its employees in cybersecurity awareness
training, although it is made available to employees who request it. There are no procedures or
enforcement mechanisms in place to ensure cybersecurity training completion across UMass Amherst’s
workforce. Depending on their work functions, certain employees in departments where Health Insurance
Portability and Accountability Act, Family Educational Rights and Privacy Act, and/or Payment Card
Industry training is required receive different training programs that also include cybersecurity awareness
training.

5. ISO 27002 is an information security standard published by the International Organization for Standardization that offers
model practices for cybersecurity risk management.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Audit Objectives, Scope, and Methodology
10
AUDIT OBJECTIVES, SCOPE, AND METHODOLOGY
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State
Auditor has conducted a performance audit of certain activities of the University of Massachusetts
(UMass) Amherst for the period July 1, 2022 through June 30, 2023.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the
conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in
the audit findings.
Objective Conclusion
1. Did UMass Amherst’s website and its learning management system (LMS),
Blackboard, adhere to Web Content Accessibility Guidelines (WCAG) 2.1 for user
accessibility, keyboard accessibility, navigation accessibility, language, error
identification, and color accessibility?
No; see Findings 1
and 2
2. Did UMass Amherst ensure that its employees completed cybersecurity awareness
training in accordance with Section 1 of Control 14 (Security Awareness and Skills
Training) of the Center for Internet Security’s (CIS’s) Critical Security Controls?
No; see Finding 3
To accomplish our audit objectives, we gained an understanding of the aspects of UMass Amherst’s
internal control environment relevant to our objectives by reviewing applicable policies and procedures
and by interviewing UMass Amherst staff members and management. In addition, to obtain sufficient,
appropriate evidence to address our audit objectives, we performed the procedures described below.
Web Accessibility
To determine, for the audit period July 1, 2022 through June 30, 2023, whether UMass Amherst’s website
and its LMS, Blackboard, adhered to WCAG 2.1, for user accessibility, keyboard accessibility, navigation
accessibility, language, error identification, and color accessibility, we performed accessibility testing
procedures on the following:
Audit No. 2024-0213-3I University of Massachusetts Amherst
Audit Objectives, Scope, and Methodology
11
 a judgmental sample of the 20 most visited webpages during the last month of the audit period,
from a population of 25,187 UMass Amherst webpages;
 a random, statistical sample of 60 selected pages with a confidence level6 of 95%, expected error
rate7 of 0%, and a tolerable error rate8 of 5%, from a population of the remaining 25,167 UMass
Amherst webpages; and
 all 59 Blackboard student features from a population of 59 student features.
User Accessibility
 We determined whether the webpage could be viewed in both portrait and landscape modes.
 We determined whether, when zoomed in to 200%, content on the webpage was undamaged
and remained readable.
 We determined whether, when zoomed in to 400%, content on the webpage was undamaged
and in a single column.
Keyboard Accessibility
 We determined whether all elements9 of the webpage could be navigated using only a keyboard.
 We determined whether any elements on the webpage prevented a user from moving to a
different element when using only a keyboard to navigate the webpage.
 We determined whether the first focusable control10 is a link that redirects to the main content of
the website. This is known as either a bypass block11or a skip link.
Navigation Accessibility
 We determined whether the website contained a title that was relevant to the website content.
 We determined whether there was a search function present to help users locate content.
 We determined whether related hyperlinks allowed navigation to the intended webpages.
 We determined whether headings within websites related to the content of the header’s section.

6. Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are
representative of the population (parameter), expressed as a percentage.
7. Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the
auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.
8. The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while
still using the sample to conclude that the results from the sample have achieved the objective.
9. An element is a part of a webpage that contains data, text, or an image.
10. The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard.
11. This is a link that brings users to the main content of a webpage.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Audit Objectives, Scope, and Methodology
12
Language
 We determined whether video content found within the website had all important sound12 and
dialogue captioned.
 We determined whether the language used on the webpage was tagged with the correct language
attribute.
 We determined whether words that appeared on the webpage matched the language to which
the webpage was set.
Error Identification
 We determined whether mandatory fields on forms alerted users if the field was left blank.
 We determined whether there was a label for any element that required user input.
 We determined whether the label was programmed correctly.
 We determined whether there were examples given to assist the user in correcting mistakes (for
example, a warning when entering a letter in a field meant for numbers).
Color Accessibility
 We determined whether there was at least a 3:1 contrast in color and additional visual cues to
distinguish hyperlinks, which WCAG recommends for users with colorblindness or other visual
impairments.
Cybersecurity Training
To determine whether UMass Amherst employees completed cybersecurity training in accordance with
CIS Control 14, we interviewed knowledgeable UMass Amherst staff members and reviewed related
documentation.
We used statistical sampling methods for testing, but we did not project the results of our testing to any
population.
Data Reliability Assessment
Web Accessibility
To determine the reliability of the site map that we received from UMass Amherst, we interviewed
knowledgeable UMass Amherst staff members and checked that variable formats (e.g., dates, unique

12. Important sounds are defined as sounds that convey additional meaning or context for the viewer. For example, a horn may
indicate a negative sound or a warning, while a bell might indicate something positive.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Audit Objectives, Scope, and Methodology
13
identifiers, and abbreviations) were accurate. Additionally, we ensured that none of the following
issues affected the site map: abbreviation of data fields, missing data (e.g., hidden rows or columns,
blank cells, and absent records), and duplicate records. We also ensured that all values in the data set
corresponded with expected values.
We selected a random sample of 20 uniform resource locators (URLs)13 from the UMass Amherst site
map and traced them to the corresponding webpages on UMass Amherst’s website, checking that
each URL and page title matched the information on the UMass Amherst webpage. We also selected
a random sample of 20 webpages from UMass Amherst’s website and attempted to trace each URL
and page title to the site map to ensure that there was a complete and accurate population of URLs
on the site map. We were unable to trace 13 webpages from UMass Amherst’s website to the site
map provided by UMass Amherst. We asked UMass Amherst about this and determined that, due to
the decentralized administration of UMass Amherst’s website, UMass Amherst management would
be unable to provide a site map that listed all the URLs on the UMass Amherst website. See Other
Matters 3 below. Although we determined that the URL list provided by UMass Amherst management
was not a complete URL list, we proceeded with selecting a sample from the URL list provided and
performed the following additional procedure: We received a URL list that listed the 20 most-visited
websites during the last month of the audit period. To determine the reliability of that list, we sampled
all 20 URLs and traced them to the corresponding webpages on the UMass Amherst website, checking
that each URL and page title matched the information on the UMass Amherst website.
LMS Accessibility
As part of our review of UMass Amherst’s Blackboard system, we requested and received access to
an online course. To determine the reliability of the Blackboard course we received access to, we
interviewed knowledgeable UMass Amherst staff members regarding the student features of the
website. Additionally, we used publicly available information from Blackboard’s website to determine
what features are available for students and conducted inquiries to determine which features were
available to UMass Amherst students during the audit period. We were able to identify 59 features
that were available to UMass Amherst students during the audit period. We then traced all 59 features
available to UMass Amherst students from the list we obtained from UMass Amherst and from

13. A URL uniquely identifies an internet resource, such as a website.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Audit Objectives, Scope, and Methodology
14
publicly available sources to the Blackboard course to ensure that we received access to a complete
and accurate course.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
15
DETAILED AUDIT FINDINGS WITH AUDITEE’S RESPONSE
1. The University of Massachusetts Amherst’s website is not fully accessible
for all Massachusetts residents and users.
The University of Massachusetts (UMass) Amherst’s website is not fully accessible. We determined that
11 webpages out of a sample of 80 of UMass Amherst’s webpages were not accessible in accordance with
Web Content Accessibility Guidelines (WCAG) for navigational accessibility, language, or color
accessibility. Of these, we determined that 8 had broken hyperlinks, 2 contained videos without proper
captioning, and 1 had links without sufficient contrast14with the surrounding text.
Broken or faulty hyperlinks limit users from having equitable access to critical information and key online
services offered by UMass Amherst and increase the likelihood that Massachusetts residents and students
will either access outdated or incorrect information or be directed to webpages that no longer exist.
Videos that lack sufficient captioning prevent users from engaging with video content in a meaningful way
(e.g., lack of context provided through dialog and important sounds). Additionally, hyperlinks without
sufficient contrast with the surrounding text negatively impact the user experience by making it difficult
to locate other relevant information.
Authoritative Guidance
The Web Accessibility Initiative’s WCAG 2.1 states,
Success Criteria 2.4 .5,
More than one way is available to locate a Web page within a set of Web pages except where the
Web Page is the result of, or a step in, a process.
Success Criteria 1.2 .2,
Captions are provided for all prerecorded audio content in synchronized media, except when the
media is a media alternative for text and is clearly labeled as such.
Success Criteria 1.4 .1,
Color is not used as the only visual means of conveying information, indicating an action, prompting
a response, or distinguishing a visual element.

14. WCAG defines a sufficient contrast as a 3:1 contrast in color and additional visual cues to distinguish hyperlinks.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
16
Reasons for Issue
UMass Amherst management told us that they have a very decentralized website and that it is difficult to
monitor every branch for WCAG compliance. The accessibility issues identified on UMass Amherst’s
website appear to stem from gaps in implementing and maintaining WCAG standards. The issues identified
in this finding reflect broader challenges, such as insufficient accessibility reviews during the audit period
and limited integration of accessibility as a priority, demonstrated by a lack of controls in web design and
maintenance processes.
Recommendation
The university should continually review its webpages to ensure that all hyperlinks lead to related
information and have sufficient contrast with the surrounding text in order to provide equitable access to
critical information and services offered online by UMass Amherst. The university should also adopt
procedures to ensure that videos have captioning features enabled when posted to the umass.edu
website.
Auditee’s Response
The University understands the importance of ensuring the accessibility of its webpages for all
users. It is important to note that accessibility is and has been a priority of the campus and webpage
reviews were performed before a launch and weekly during the audit scope period. The campus
will continue to perform weekly accessibility reviews and resolve issues as they are identified. Also,
the campus has fixed all accessibility issues identified in the audit.
Auditor’s Reply
Based on its response, UMass Amherst has taken, and continues to take, measures to address our
concerns regarding this matter.
2. The University of Massachusetts Amherst’s learning management system,
Blackboard, is not fully accessible for all students.
UMass Amherst’s learning management system (LMS), Blackboard, is not fully accessible. We determined
that 42 of the Blackboard student features out of the 59 Blackboard student features we tested were not
accessible for user accessibility, keyboard accessibility, navigational accessibility, error identification, or
color accessibility. Specifically, we determined the following:
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
17
User Accessibility
 One student feature restricted the view of the webpage and could not be viewed fully in portrait
mode.
 Two student features had text that did not properly resize.
 Thirty-three student features could not be enlarged without issue.
Keyboard Accessibility
 Five student features could not be navigated using a keyboard alone.
 One student feature trapped the focus15 of keyboard users.
 Two student features did not provide a bypass block16 as the first focusable element17 as a way to
skip to a page’s main content.
Navigational Accessibility
 Eleven student features had broken links or links that lead to the incorrect location.
Error Identification
 Two student features did not identify user input errors.
Color Accessibility
 One student feature did not have sufficient contrast to convey the required information.
The above instances of noncompliance have the following effects on the user:
Broken or Faulty Hyperlinks
 This can limit Blackboard users from having equitable access to critical information and key online
services offered on the LMS.
 This can increase the likelihood that users will either access outdated or incorrect information or
be directed to webpages that no longer exist.

15. This is a situation where the user is locked into using only a limited section of the webpage until the page is either refreshed
or the computer is restarted.
16. This is a link that brings users to the main content of a webpage.
17. This is the first element a user will be brought to on a webpage when navigating with a keyboard.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
18
Missing Search Bars
 This can prevent users from navigating to other relevant information.
Hyperlinks without Identifiable Markers or Sufficient Contrast
 This can negatively impact the user experience by making it difficult to locate other relevant
information.
Zoom in to 200% and 400%
 Users will be unable to read Blackboard content.
Bypass Blocks
 Users will be unable to navigate to the important main content of a page quickly.
Portrait Mode
 Users will be unable to interact with their course content on their mobile devices effectively.
Keyboard Accessibility/Navigation
 Users who have mobility issues will be unable to access certain features and content.
Titles
 Users with screen readers will lose comprehension of the feature.
Language Attributes
 The lack of language attributes will prevent screen readers from reading the content to users.
Error Identification
 If users are not informed of errors when making inputs on data entry, it means that users will be
unable to identify their errors and retrieve the content they need.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
19
Authoritative Guidance
The Web Accessibility Initiative’s WCAG 2.1 states,
Success Criterion 1.3.4 Orientation (Level AA)
Content does not restrict its view and operation to a single display orientation, such as portrait or
landscape, unless a specific display orientation is essential.
Success Criterion 1.4.10 Reflow (Level AA)
Content can be presented without loss of information or functionality, and without requiring
scrolling in two dimensions for:
 Vertical scrolling content at a width equivalent to 320 CSS pixels;
 Horizontal scrolling content at a height equivalent to 256 CSS pixels.
Except for parts of the content which require two-dimensional layout for usage or meaning.
Success Criterion 1.4.4 Resize Text (Level AA)
Except for captions and images of text, text can be resized without assistive technology up to 200
percent without loss of content or functionality.
Success Criterion 2.1.1 Keyboard (Level A)
All functionality of the content is operable through a keyboard interface without requiring specific
timings for individual keystrokes, except where the underlying function requires input that depends
on the path of the user’s movement and not just the endpoints.
Success Criterion 2.1.2 No Keyboard Trap (Level A)
If keyboard focus can be moved to a component of the page using a keyboard interface, then focus
can be moved away from that component using only a keyboard interface, and, if it requires more
than unmodified arrow or tab keys or other standard exit methods, the user is advised of the
method for moving focus away.
Success Criterion 2.4.1 Bypass Blocks (Level A)
A mechanism is available to bypass blocks of content that are repeated on multiple Web pages.
Success Criterion 2.4.5 Multiple Ways (Level AA)
More than one way is available to locate a Web page within a set of Web pages except where the
Web Page is the result of, or a step in, a process.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
20
Success Criterion 3.3.1 Error Identification (Level A)
If an input error is automatically detected, the item that is in error is identified and the error is
described to the user in text.
Success Criterion 1.4.1 Use of Color (Level A)
Color is not used as the only visual means of conveying information, indicating an action, prompting
a response, or distinguishing a visual element.
Reasons for Issue
UMass management shared with us that they expected Blackboard to be largely WCAG accessible due to
the marketing of the product and availability and review of the Voluntary Product Accessibility Template.18
UMass management expressed that UMass Amherst was in the process of changing its LMS from
Blackboard to Canvas, with accessibility being one of the factors leading to the change.
Recommendation
UMass Amherst should review the accessibility statements and reports of its LMS vendor to determine
instances of WCAG noncompliance. UMass management should work with its LMS vendor to ensure that
any potential instances of WCAG noncompliance are resolved.
Auditee’s Response
The University understands the importance of utilizing an LMS that is accessible. As noted in the
report, the campus transition to Canvas, a new LMS vendor, was underway in the spring of 2023
before the start of this audit. Blackboard’s accessibility issues were one of the factors that led the
campus to transition from Blackboard to Canvas. The University did monitor Blackboard’s
accessibility statements during the audit scope period. It will continue to review Canvas’
accessibility statements and reports to determine if it meets accessibility requirements since the
vendor is responsible for maintaining their LMS’ accessibility.
Auditor’s Reply
The university states in its response that it monitored Blackboard’s accessibility statements during the
audit scope period. However, as part of our audit testing, we determined that 42 of the Blackboard
student features out of the 59 Blackboard student features we tested (71%) were not accessible for user
accessibility, keyboard accessibility, navigational accessibility, error identification, or color accessibility.

18. The Voluntary Product Accessibility Template is a report prepared by the vendor that describes how well the product
conforms to accessibility standards.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
21
Based on our audit results, and even with a new vendor for this service (Canvas), we recommend that
UMass Amherst implement our recommendation in order to be in compliance with WCAG.
3. The University of Massachusetts Amherst has not implemented workforce
cybersecurity awareness training.
UMass Amherst has not implemented cybersecurity awareness training in accordance with Center for
Internet Security (CIS) Control 14. At UMass Amherst, the Written Information Security Policy (WISP) does
not require employees to complete cybersecurity training at hire and at least annually thereafter.
Additionally, while cybersecurity training courses are made available to employees who request it,
employees are not required to complete the training per the WISP and are not enrolled in it, annually or
at hire.
If UMass Amherst does not educate all employees on their responsibility to protect its information assets
by requiring cybersecurity awareness training, then UMass Amherst is exposed to an elevated risk of
cybersecurity attacks, which may cause financial and/or reputational losses.
Authoritative Guidance
According to UMass system management, UMass Amherst follows Section 1 of Control 14 (Security
Awareness and Skills Training) of the CIS’s Critical Security Controls for the cybersecurity awareness
training of their employees. This control states,
Establish and maintain a security awareness program. The purpose of a security awareness
program is to educate the enterprise’s workforce on how to interact with enterprise assets and data
in a secure manner. Conduct training at hire and, at minimum, annually. Review and update content
annually, or when significant enterprise changes occur that could impact this Safeguard.
Reasons for Issue
UMass Amherst management told us that given the size of the campus’s workforce, it is difficult to
implement policy and enforce compliance. The issue arises primarily due to a lack of clear policy mandates
and insufficient prioritization of cybersecurity awareness training within UMass Amherst’s WISP. Although
high-risk areas are addressed with specific cybersecurity training requirements (e.g., employee training
needed for Health Insurance Portability and Accountability Act or Payment Card Industry compliance),
other departments are lacking clear policy.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Detailed Audit Findings with Auditee’s Response
22
Recommendation
UMass Amherst management should update its WISP to require all employees to complete cybersecurity
training at hire and at least annually thereafter. UMass Amherst should also devise means by which it can
enforce and monitor compliance with an updated training policy. UMass Amherst should enroll all of its
employees, contractors, and interns in cybersecurity awareness training.
Auditee’s Response
Cybersecurity awareness training is only one part of a highly sophisticated and comprehensive
cybersecurity program deployed by the campus to detect and prevent threats to the campus’
information technology infrastructure, assets and data. All new employees will be required to take
the training as part of the on-boarding process. Annually, all employees will be required to take a
refresher course and emails will be sent out with the link to the learning management system
training site. Furthermore, management will monitor whether employees have timely completed
training. The training material will be reviewed periodically and if necessary, the material will be
revised for any new and applicable authoritative guidelines.
UMass Amherst will update its WISP to reflect the new cybersecurity awareness training
requirements.
Auditor’s Reply
Based on its response, UMass Amherst will take measures to address our concerns regarding this matter.
We note that the requirement to provide this training is not new and will follow up on this in
approximately six months as part of our post audit review process.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Other Matters
23
OTHER MATTERS
1. The University of Massachusetts Amherst can further enhance the
accessibility of its marketing pages.
During our audit, we determined that 1 webpage of our sample of 80 University of Massachusetts (UMass)
Amherst webpages did not have a search bar. After discussing with UMass Amherst management, we
determined that this page was used as a landing page for a marketing campaign. This makes the page part
of a process, and therefore, a search bar or other navigable elements are not required for Web Content
Accessibility Guidelines (WCAG) 2.1 compliance. However, certain industry sources argue that including a
search bar and other navigable elements would improve accessibility.
A lack of a navigation bar or search bar prevents users from easily exploring the website or finding
additional information.
Authoritative Guidance
The Web Accessibility Initiative’s WCAG 2.1 states,
Success Criteria 2.4 .5,
More than one way is available to locate a Web page within a set of Web pages except where the
Web Page is the result of, or a step in, a process.
Additionally, private industry sources, like Bureau of Internet Accessibility, state,
Many landing pages remove the navigation bar to improve conversion rates. This isn’t strictly
necessary, and it can be frustrating for people who want to explore your website without reentering
the URL. . . .
The [landing page with a navigation bar] has an effective [call to action (CTA)] and strong sales
copy, but the navigation bar is still accessible. The page’s layout keeps the user’s attention on the
CTA button, but it doesn’t prevent users from visiting other parts of the website.
Reasons for Issue
UMass Amherst told us that this webpage was used for generating marketing leads and is designed to
guide users to engage with the marketing materials. It indicated that navigational tools, like a search bar
or navigation menu, will lead users from the landing (marketing) page.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Other Matters
24
Recommendation
UMass Amherst should consider adding a navigation bar and search box on its landing page as a way of
further enhancing user accessibility.
Auditee’s Response
The University will take this under consideration; however, a marketing landing page is a webpage
that does not include searchable content and is made to be seen only by people who have clicked
on a digital ad from platforms such as Facebook, Instagram, and Google Search. This is a common
practice and an industry standard.
Auditor’s Reply
As noted above, best practices state that including a search bar and other navigable elements improves
website accessibility. A lack of a navigation bar or search bar prevents users from easily exploring the
website or finding additional information. Given these reasons, we recommend that UMass implement
our recommendation in this area.
2. The University of Massachusetts Amherst can further enhance the
accessibility of its cookie banner.
During our audit, we determined that the first focusable element of 46 webpages out of our sample of 80
webpages was not a bypass block but was the cookie banner. WCAG criteria states that the first focusable
element of a webpage must be a bypass block to the main content. At the same time, however, General
Data Protection Regulations require that users be given the option to protect their privacy when logging
on to a website (including from cookies). The cookie banner should not take priority over the bypass block,
the bypass block should be configured to override the cookie banner so it is most accessible to users.
A lack of bypass blocks that allow users to skip to the main content of a webpage without first selecting
their cookie settings may keep disabled users from easily obtaining the information they need.
Authoritative Guidance
Success Criterion 2.4.1 Bypass Blocks (Level A)
A mechanism is available to bypass blocks of content that are repeated on multiple Web pages.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Other Matters
25
Reasons for Issue
Cookie settings have been commonplace since 2018 and were implemented in response to the European
Union General Data Protection Regulation, which required them. The competing standards between
General Data Protection Regulations and WCAG has caused a lack of clarity regarding accessible
implementation, leading to institutions designing cookie banners to be the first focusable element.
Recommendation
Accessibility industry sources suggest that the best practice would be to ensure that the skip link to main
content comes first, followed by a skip link that allows users to skip to the cookie banner. UMass Amherst
should consider adding these skip links to its website.
Auditee’s Response
The University will take this under consideration; however, it is required to follow European Union
General Data Protection Regulations that require its users be given the option to protect their
privacy when initially logging on to a website, including from cookies.
Auditor’s Reply
Based on its response, UMass Amherst is taking measures to address our concerns regarding this matter.
3. The University of Massachusetts Amherst does not maintain a full site map
of its umass.edu website.
During our audit, we determined that UMass Amherst does not keep a full and complete inventory of the
number of webpages and web addresses on the umass.edu website. As part of our audit procedures, we
selected a random sample of 20 webpages from UMass Amherst’s website and attempted to trace the
uniform resource locators (URLs) and page titles to the site map we received to ensure that there was a
complete and accurate population of URLs on the site map. We were unable to trace 13 webpages from
UMass Amherst’s website to the site map provided by UMass Amherst management. We asked UMass
Amherst management about this and were told that, due to the decentralized administration of UMass
Amherst’s website, UMass Amherst management was unable to provide a site map that listed all the URLs
on the UMass Amherst website.
If UMass Amherst does not have a complete inventory of webpages for its umass.edu website, then it
exposes itself to an increased risk of being unable to track or manage the webpages under the umass.edu
domain. This can cause users to be provided with out-of-date and inaccessible information. It is
Audit No. 2024-0213-3I University of Massachusetts Amherst
Other Matters
26
significantly more difficult for UMass Amherst to maintain webpages that are not actively tracked by
university personnel.
Authoritative Guidance
The National Institute of Standards and Technology SP 800-53 Revision 519 states,
CM-8 SYSTEM COMPONENT INVENTORY
Control:
a. Develop and document an inventory of system components that:
1. Accurately reflects the system;
2. Includes all components within the system;
3. Does not include duplicate accounting of components or components assigned to any other
system;
4. Is at the level of granularity deemed necessary for tracking and reporting; and
5. Includes the following information to achieve system component accountability:
[Assignment: organization-defined information deemed necessary to achieve effective
system component accountability]; and
b. Review and update the system component inventory [Assignment: organization-defined
frequency].
Reasons for Issue
While UMass reports that it is moving toward a more centralized model so that the umass.edu website
can be more uniform, it currently operates a largely decentralized website with each department
responsible for maintaining its own website and content. In addition, there is a lack of proactive
management oversight and governance. Specifically, UMass Amherst did not implement a process to
appropriately oversee this decentralized model or ensure departmental accountability for the inventory
of accessible websites.

19. The National Institute of Standards and Technology provides security and privacy controls used by organizations to protect
their operations and assets.
Audit No. 2024-0213-3I University of Massachusetts Amherst
Other Matters
27
Recommendation
UMass Amherst management should complete an inventory of its umass.edu website and adopt
procedures to ensure that it maintains a full list of webpages, while continuing its effort to centralize the
administration of the website.
Auditee’s Response
The University will take this recommendation under consideration. It is important to note that
during the audit scope period the University proactively managed the oversight and governance of
its website and will continue to do so.
Auditor’s Reply
As noted above, we determined during our audit that UMass Amherst does not keep a full and complete
inventory of the number of webpages and web addresses on the umass.edu website. If UMass Amherst
does not have a complete inventory of webpages for its umass.edu website, then it exposes itself to an
increased risk of being unable to track or manage the webpages under the umass.edu domain. This can
cause users to be provided with out-of-date and inaccessible information. Given these reasons, we
encourage UMass to implement our recommendation in this area.

Ultimate World Financial Map

Monday, December 15, 2025

Massachusetts_Audits_Mixed

No comments:

Post a Comment

Delete when done

Report Abuse