Allegheny Child Care Matters Pilot Program Evaluation Report

Administrative Data Analysis

Administrative data analysis relied on two data sets:

1. The first data set included enrollment data about families participating in the ACCM Pilot Program. This dataset—provided by DCI—included characteristics of families and children who had participated in the ACCM Pilot Program as of December 2024 and was updated as families reported changes (e.g., changes in child care provider, income). 2. The second data set included data on regulated child care providers participating in the ACCM Pilot Program. This dataset—provided by the ELRC—included information such as provider addresses, STAR ratings, and other variables on providers who had participated (or were currently participating) in the program as of December 2024. The ELRC also provided MEF with a supplementary data set on relative care providers who participated in the program at any point during its implementation. Once MEF was granted access to these datasets, staff thoroughly reviewed data dictionaries and met with DCI and the ELRC to understand variable definitions, formats, and relationships. To analyze the data about families participating in the ACCM Pilot Program , MEF first developed code using Stata to clean and provide proper version control mechanisms. MEF also maintained detailed scripts of all extraction methods and transformations applied to the dataset, enabling reproducibility and transparent documentation of the entire data analysis process. Next, MEF implemented a data cleaning and validation process to ensure quality and integrity of the data. We ran diagnostic checks to identify missing values, outliers, and inconsistencies using summary statistics and visualization tools. Every time an inconsistency was found, we discussed the appropriate methods to handle it with DCI. For each key variable of interest as described in the research questions, we created frequency tables and cross tables to describe the population. After completing the analysis, we exported all tables from Stata into Excel for comprehensive reporting. We implemented a quality control and review process, documenting the data handling procedures, reviewing any assumptions made during the process, and checking output consistency. To analyze the data about providers , MEF created summary tables in Excel. We implemented a quality control and review process, documenting the data handling procedures, reviewing any assumptions made during the process, and checking output consistency. Locations of providers and families were reported based on addresses and cities. To obtain the families’ and providers’ municipalities, we accessed U.S. Census geographic data available in Census Geocoder (https://geocoding.geo.census.gov/geocoder/). To assess the match in location between families and providers, these locations were compared to each other within each family. To identify the characteristics of parents’ employers (e.g., industry, organizational type), we conducted an online search and classified each organization based on information available on their official websites and other publicly accessible sources. This process was supported by the use of ChatGPT Pro. Specifically, we instructed ChatGPT Pro to:  Conduct an online search for information about each organization that parents had listed as their place of employment in the ACCM Pilot Program administrative data.  Classify each organization as a for-profit, non-profit, government organization, or “not enough information”—and provide a justification for each classification.

26

Made with FlippingBook Digital Proposal Creator