Research Supports NRPA’s Certified Playground Safety Inspectors Reduce Injuries:
Part 4 of 4: Closer Look at Research Process and Methods that Support Curtis’s Conclusions and Suggestions for Future CPSI Research
By Kenneth S Kutska, CPSI, Executive Director
International Playground Safety Institute, LLC
April 27, 2016
The purpose of this study was to examine the effectiveness of the playground safety inspector certification programs on reducing reported injuries sustained on public playgrounds in the state of California. Utilizing themes from a review of the literature and the three elements of the working model (CPSI-CEWM), the study examined three hypotheses:
(a) A greater decrease in injuries will occur when cities utilize certified playground safety inspectors
(b) The degree to which supervisors value the playground safety inspector program will affect injury rates
(c) Playground safety inspectors’ beliefs about the system will also influence injury rates.
Based on predominant themes derived from the review of literature and the elements of the CPSI-CE Working Model, three hypotheses emerged, included:
(a) A decrease in injuries occurred when playground safety inspectors were hired.
(b) Playground safety inspectors’ beliefs in the system influenced injury rates.
(c) The value supervisors place on the playground safety inspector program affected injury rates.
Method – Design and Participants
Previous teacher certification effectiveness studies provided excellent guidelines for designing this study. A correlational framework was utilized to examine relationships among injuries sustained on public playgrounds among three different size municipalities in the state of California during a 10-year period and model components such as playground safety inspectors’ and their supervisors’ beliefs and attitudes about certification status, experience level, and available resources.
Three sets of participants from municipalities were sampled for this study, including
(a) active playground safety inspectors (n = 474),
(b) the ‘supervisors’ of the playground safety inspectors (n = 474),
(c) the risk managers from those same municipalities (n = 474).
A response rate of at least 40% for cities from each size category was targeted, but actual response rate was over 63% with 286 communities providing completed responses. Only one active playground safety inspector from each city was included in the study because the number of inspectors could vary from city-to-city, with larger cities having a number of active playground safety inspectors while smaller cities may have only one. Also, the second group of participants, the ‘supervisors’ of the inspectors, typically would have one supervisor overseeing all inspectors. The number of inspectors and supervisors was kept the same to allow perceptions of both groups equal weighting in analyses.
Cities were divided into three population sizes, including:
(a) small (500 – 9,999),
(b) medium (10,000 – 199,999),
(c) large (≥ 200,000) cities.
Cities of less than 500 were eliminated from the study because pilot study data suggested most did not or were not responsible for maintaining public playgrounds. Cities were categorized into three distinctive sizes enabled inferences to be drawn about each of these population-based subgroups that otherwise could be lost in a more generalized random sample. Stratified sampling by city size also afforded an opportunity to assess the variations of results, if any, based on city sizes (e.g., Did more medium-sized cities have CPSIs, and experienced a reduction in injuries as a result, compared to small or large cities?). Only one active CPSI from each city was included in the study because the number of inspectors could vary from city-to-city, with larger cities having a number of active CPSIs while smaller cities may have only one. The second group of participants is the inspectors’ ‘supervisors,’ and depending on the organizational structure of the city, one supervisor could oversee more than one inspector, which influenced their perception.
To examine the effectiveness of playground safety inspector certification on reducing reported injuries sustained on public playgrounds, it was necessary to develop three survey instruments to obtain data to test study hypotheses. These included;
(1) Playground Safety Inspector Assessment Survey (PSIAS),
(2) Playground Safety Inspector Supervisor Assessment Survey (PSISAS),
(3) Playground Safety Inspector Injury Data Survey (PSIIDS).
The instruments were developed specifically for this study and for specific positions within city organizational structures that could provide necessary data and perceptions.
The PSIAS and PSISAS were broken into the three primary dimensions of the CPSI-CE Working Model and were specifically related to the participants’ positions as playground safety inspectors and ‘supervisors’ of playground safety inspectors. The PSIIDS was an instrument designed to collect injury data and was administered to risk managers for each sample city.
(1) Playground Safety Inspector Assessment Survey (PSIAS).
The playground safety inspector was defined as a full-time employee with the responsibility to inspect playgrounds for safety issues and eliminate risks. The PSIAS was developed specifically for this study to gain perceptions and demographic data from playground safety ‘inspectors.’ The PSIAS was based on the elements of the CPSI-CE Working Model and consists of four sections including,
(d) demographic and background information, including 32 perception items and 10 demographic questions.
The PSIAS employed three model-based subscales, including;
(a) 12-item training and current certification subscale to assess perceptions of playground safety inspectors towards training and certification issues,
(b) 10-item experience subscale to examine perceptions of playground safety inspectors toward experience based on the CPSI-CE Working Model,
(c) 10-item adequate resources subscale to assess playground safety inspectors’ perceptions toward available resources based on the CPSI-CE Working Model.
The PSIAS also was comprised of 10 demographic questions related to history as a playground safety inspector, certification status and source, age, and gender.
(2) Playground Safety Inspector Supervisors’ Assessment Survey (PSISAS).
The PSISAS was designed for this study to be completed by the supervisor of the playground safety inspector. Supervisors were defined as full-time employees or elected officials with the responsibility of supervising the city’s playground safety inspector(s). The supervisor completing the PSISAS was not required to be certified or trained as a playground safety inspector. However, each was to be responsible for supervising the playground safety inspector who completed the PSIAS. The PSISAS was developed for administration to the ‘supervisors’ of the playground safety inspectors, and was comprised of the same four assessment dimensions as the PSIAS. The PSISAS was developed to ascertain beliefs and attitudes about how supervisors of playground safety inspectors perceive the effectiveness of their employees’ certifications (or present training status) in reducing reported injuries in playgrounds they were responsible for inspecting, based on CPSI-CE Working Model elements. The three perception subscale items were identical to the PSIAS, except each obtained the perceptions of the supervisor.
(3) Playground Safety Inspector Injury Data Survey (PSIIDS).
The PSIIDS was developed specifically for administration to the risk managers of sample cities to identify the number of reported injuries for each municipality during the years 2000, 2005 and 2010. The risk manager was defined as the municipal employee, elected official or outsourced risk management agency with the responsibility of obtaining and maintaining records of reported injuries on respective city playgrounds. This data was most likely in the form of financial claims against the municipality for injuries sustained on its public playgrounds. Injuries sustained on public playgrounds not requiring medical attention were most likely not be reported. The three data collection years were selected because the dates were recent and would allow a measurable trend analysis. The information obtained from this instrument provided the injury data from which the three hypotheses were examined.
The primary objective of these analyses was to examine the relationships between the certified playground inspector program and injury rates in California for 2000, 2005, and 2010. The second purpose was to investigate perceptions about how effectively the playground inspection program and certification were working, including:
(a) How much influence does employing a CPSI have on reduction of injury rates?
(b) How much do inspector beliefs in the CPSI system influence injury rates?
(c) How much do supervisors’ value and support of the program affect injury rates?
Additionally, more inductive analyses were conducted to identify whether meaningful profile groups could be distinguished based on;
(a) dimensions of perceived inspector effectiveness
(b) change in injury patterns, with differences between profile groups
Supervisors’ Perceptions Impact Injury Rates
The second hypothesis assessed how supervisors’ perceptions about certification impact injury rates in the state of California. These results support the role of supervisors’ certification perception on reducing injury rates. Survey results generated a four-factor solution accounting for 78% of the total variance among 21 supervisor perception items of the certification program. The high factor loadings throughout each of the four factors demonstrates strong supervisor dimensions which were used to examine the relationship between supervisors’ perceptions on the CPSI program and California’s injury trends across the decade from 2000 to 2010, thus providing support for Hypothesis 2. Less than adequate resources for inspectors’ results in fewer inspections and less frequent repair of equipment, perhaps contributing to increased injuries (Hudson et al., 2004; Kutska, 2009). Support of the CPSI program by the supervisor would then seem crucial to program success. Additionally, supervisor support for the CPSI would also seem to play a critical role in advocating for resources from the local government lawmakers (Iverson & Payne, 2008-2009; Kustka, 2008; Kutska, 2008-2009). Contributing additional support to Hypothesis 2, results revealed generally positive relationships between supervisors’ perceptions about the CPSI system and injury rates, indicating that supervisors’ subscales (i.e., supervisor experience, resources, CPSI value, and employee benefits) are positively related to the reported steady decline in playground injuries in the three assessment years of 2000, 2005, and 2010.
Influence of Inspectors’ Certification Perceptions on Injury Rates
Playground safety inspectors’ perceptions generated a five-factor solution accounting for 75% of the total variance among 20 inspector effectiveness items. The high factor loadings for each of the five factors indicate strong inspector perception dimensions thought to mediate injury rates, and thus provides support for Hypothesis 3.
The combined reduction in playground injury rates and the results on the perceptions of the inspectors were consistent with previous conceptual predictions and research. Peterson (2002) argued that being keenly aware of how to identify the hazards through CPSI training and experience is a priority risk management skill, but only 49% of employees have any interest in the CPSI certification (National Recreation and Parks Association, 2010).
It goes without saying that a significant challenge is to obtain and retain certified inspectors. Miller and Svara’s (2009) and Mulvaney’s (2010) concern with city operating budget shortfalls has the potential to create less than adequate resources that prompt fewer inspections and repair of equipment, increasing the likelihood of more injuries (Hudson, 2004; Kutska, 2009). A strong positive perception of the program by inspectors would then seem crucial. Results revealed a generally positive relationship between inspector beliefs in the CPSI inspection and maintenance program and injury rates. The results indicate that inspector perception dimensions are related to the reported steady decline in playground injuries in the past decade, providing additional support to the Hypothesis 3.
Supervisors’ and Inspectors’ Perceptions.
The analysis results for supervisor and inspector perceptions produced four significant correlations, all of which were significant. When inspector perceptions of resources, experience and the value of certification were low, their supervisors had corresponding low beliefs about these certification perceptions. Similarly, when inspectors perceived employee benefits, experience and supervisor resource beliefs positively, their supervisors demonstrated similar perceptions for these three variables, even though they reported a somewhat negative perception of the value of certification. These results parallel what Hudson et al. (2004), Kutska (2009), and Mulvaney (2010) suggested about playground maintenance, particularly that as maintenance suffers injury rates increase if resources and experience are lacking. In other words, if the resources and experience are limited, the perceptions of both inspectors and supervisors believe injury rates could increase.
Analysis among inspectors and supervisors had similar positive perceptions about the value and benefits of certification, although supervisors also perceived experience as moderately beneficial as well, and inspectors and their supervisors who valued experience highly also believed their resources were low. Kutska (2008/2009) asserted that maintenance staff really do care about the quality of their work, but require the means to do it. With only 49% of playground maintenance employees having any interest in the CPSI certification, it seems logical that there is a need by both inspectors and supervisors to have positive perceptions about the values associated with certification (National Recreation and Parks Association, 2010). The results of the analysis broaden support to this third hypothesis.
Results of this study found that strong positive CPSI’s perceptions of certification benefits promoted reduction of injuries. Additionally, these results suggest that helping prospective CPSIs understand the benefits of certification for both injury reduction and personal advancement needs to be made a more important part of the certification process.
Correlational analyses about the relationship between injuries and inspector perceptions and between inspector and supervisor perceptions, provided some support for Hypothesis 3.
Injuries and inspectors’ Perceptions.
The analysis found the relationship between injuries and inspector perceptions to be non-significant. Results produced three correlations but none were significant. This finding did not lend support to Hypothesis 3, but the lack of significant effects doesn’t necessarily indicate that certification has no impact (Jepsen, 2004).
Surprisingly, the relationship between injury and inspector perceptions demonstrating minimal relationships with injury rates. Because injury rates generally declined across the decade, it begs the question why 2005 correlation results were particularly low. This finding could be a reflection of the new CPSI requirements the state of California instituted in 1999 needed time for cities to adjust and demonstrate injury reductions from the implementation of the new program. Despite Kutska’s (2008) assertion that CPSI efforts should have a substantial bearing on decreasing injury rates, realistically it would take time to implement a Playground Safety and Inspection program and results would lag several years behind implementation, thus explaining this unexpected finding. Though the results for 2005 were low, the modest positive relationship further supports the Hypotheses 2 and 3.
Dwight Curtis’s Research Conclusions:
Curtis goes on to say the implementation of the California Playground Safety Regulations (R-39-97) in 1999 provided an excellent information base to examine the effectiveness of the playground safety inspector certification. While the reported national playground injury rates have been increasing, California’s have decreased over a ten-year period. The NPPS playground safety report card for both 2000 and 2004 gave California higher grades of B- than the nation overall.
Curtis went on to say his study did not include a statistical analysis comparing the downward trend of California’s reported injury rates, the timing of the law requiring certified inspectors for new playground installations, and the NPPSs’ Safety Report Card results. However, the three all point toward a positive relationship (i.e., a downward trend of injury rates from 2000 through 2010), and the enactment of the law in 1999, and California receiving above the mean NPPS playground safety grades (i.e., “B-” in 2000 and 2004 while the national average was a “C-” in 2000 and a “C+” in 2004; NPPS, 2004). Even though the results of this study for inspector perceptions were not as strong as might be desired, they do provide support for Hypothesis 1 that a decrease in injuries is likely if a Certified Playground Safety Inspectors are hired. These findings also provide support to all three critical elements of the Certified Playground Safety Inspector-Certification Effectiveness Working Model (CPSI-CEWM).
The data collected through this exploratory study sought to examine California’s playground safety inspectors’ certification status, its supervisory support, and if and how they have influenced injury rates in that state. The data analyses clearly support this study’s three hypotheses of;
(a) decreasing injuries if certified playground inspectors are on staff,
(b) the value supervisors place on the playground safety inspector program affects injury rates, and
(c) playground safety inspectors’ beliefs about the playground safety inspection program also influences injury rates, and delivers solid support for the Curtis Playground Safety Inspector Certification Effectiveness Working Model (CPSI-CEWM).
Dwight Curtis’s Implications for Future Research and Practice:
Curtis went on to say his study was exploratory and established an information base upon which future research can be built. Results revealed a host of implications for future research and practice. It would be interesting to know the under the demographics of the study;
• number of inspections conducted each year
• number of times inspectors find issues while inspecting
• number and types of issues addressed during a specific period of time
• reasons certain issues are not addressed.
What is the True Value of the California Law requiring CPSI designation by the NRPA.
Curtis went on to suggest that because California has the unusual circumstance of being the only state to require that certified playground inspectors approve the installation of playgrounds before they can be opened for public use (C. Smith, personal communication, May 27, 2012) it would be useful information to know whether this already is an inspirational goal nationwide.
Curtis suggests a national goal generated from the results of this study may enhance the playground certification movement, but other states need to better understand the benefits of California’s law on injury rates.
These questions would require knowing more fully what the costs versus the benefits of certification are. This study also shows that playground injury reduction in California is statistically significant, but is it practically significant? Knowing the costs versus the benefits would also assist in answering this ques