CYBHI NEES Toolkit
Introduction: Welcome to the NEES toolkit The Network & Ecosystem Experiences Survey (NEES) is a tool originally developed by Mathematica for the purposes of the CYBHI evaluation to help counties understand how well organizations in the behavioral health system work together. The NEES applies established network analysis methods 1 to examine how organizations interact and connect with one another, providing actionable information communities can use to strengthen local networks and improve systems of care for residents. The tool captures how connected the various agencies, nonprofits, education partners, and other organizations are in your county. The results from your use of this tool can highlight both strengths and opportunities to deepen collaboration to support the needs of children and youth in your county, or other populations of interest. Research shows that network analysis information, when effectively presented, can have practical value by contributing to system planning efforts. 1 Using NEES data, counties can create interactive network maps to visualize the relationships or develop an integration score to track changes in collaboration and network connectedness over time. The network maps give communities helpful information that they can use alongside other information, such as demographic data, community feedback, and local knowledge, to better understand how organizations in their area are connected. Although NEES results offer valuable insights, they should be considered alongside other information to ensure they are used appropriately for community planning. For examples of how the NEES is used alongside other contextual community information, review the CYBHI evaluation county case studies, available at https://cybhi.chhs.ca.gov/evaluation-hub/.
This toolkit provides:
- A step-by-step guide for organizations to use the NEES for their purposes
-
Options for survey and analysis platforms
- Recommended visuals to help communicate findings
- Downloadable templates to support each step
- Additional opportunities for external support
Step 1: Prepare your list of organizations and targeted respondents Start by identifying key organizations that play a role in your county’s behavioral health system, regardless of age or population served. Creating an accurate and comprehensive list is essential because this list becomes the roster of organizations that respondents see in the NEES. Next, identify individuals or roles within the organization (such as program director, executive director, etc.) that will be asked to complete the survey. Since the survey results will reflect this person’s perspective, carefully consider the information you hope to learn to guide which staff you will ask to complete the survey. For example, asking senior leaders (such as an executive director) to respond can provide a “birds-eye view” of the network, but may lead to less “on the ground” accuracy about relationships. Encouraging multiple respondents from each organization can help, but survey tracking and analysis becomes increasingly complex as the number of respondents increases. A complete roster, based on your unique needs, will reduce bias and ensure that you capture relationships across your full ecosystem. Counties can draw from existing collaborative structures, such as county behavioral health taskforces, memoranda of understanding (MOUs), advisory groups, or input from well-connected leaders, to ensure all
1 Provan, Keith G., Mark A. Veazie, Lisa K. Staten, and Nicolette I. Teufel-Shone. “The Use of Network Analysis to Strengthen Community Partnerships.” Public Administration Review, vol. 65, no. 5, September/October 2005.
Mathematica ® Inc.
1
CYBHI NEES Toolkit
organizations in the ecosystem are included. Because each county’s system looks different, take care to include partners across diverse sectors.
Organizations to consider:
- County agencies (behavioral health, education, public health, child welfare, juvenile justice)
- Community based organizations (behavioral health providers, family resource centers, and community groups)
- Education partners (school districts/local education agencies, COEs, higher education)
-
Tribal entities
-
Managed care plans
You can use the roster template to help you gather the information needed for survey setup.
Step 2: Administer the survey The NEES asks respondents one core question about how their organization works with each listed entity. This question repeats for every organization on the roster. You can administer the survey using any method, from paper to online tools. We recommend using an electronic platform. Online survey tools make it easier to distribute the survey, track responses, and export data for analysis.
Commonly used platforms include:
-
Google forms (pictured below)
-
Qualtrics
-
SurveyMonkey
Mathematica ® Inc.
2
CYBHI NEES Toolkit
Mathematica recommends using this exact wording to ensure consistent results (including the intentional underlining): “How does your organization currently work with the organizations below to support the behavioral health of children and youth ages birth to 25 years?”
If you are using the NEES for a different topic, you may adapt the second part of the question:
“How does your organization currently work with the organizations below to support the [topic area] of [population group]”
Mathematica ® Inc.
3
CYBHI NEES Toolkit
The NEES uses a rating scale adapted from the Tamarack Institute’s Collaboration Spectrum. Because this scale is evidence-based, do not change the wording. Even small edits can change how respondents interpret the options.
Survey administration checklist:
✔ Distribute unique survey links . Most platforms allow you to track responses while keeping answers confidential. ✔ Set a clear response window. Include due dates and reminders in your outreach. ✔ Send follow up messages. A simple template could be: o “Hello, this is a friendly reminder to complete the Behavioral Health Ecosystem Survey. Your input is essential to understanding countywide collaboration. Please use your unique link below. Thank you!” ✔ Monitor response rates . Check response rates regularly through your selected platform to track progress and identify gaps. ✔ Follow up strategically with underrepresented organizations or sectors to improve network completeness. ✔ Aim for broad representation . You may not reach 100% but ensure that all core partners are represented. Step 3: Analyze your data Once you’ve collected your data, it’s time to begin analysis. Organizing your data Use your platform’s data export function to export responses into a CSV or Excel file. From there, create an analysis worksheet with columns such as:
- Respondent_org – the organization completing the survey - Reciprocal_org – the organization being rated - Rating – numeric score of the rating (1 – 5) - Rating_label – descriptive label for the rating (coexist, cooperate etc.) Calculate the “Strength Score” Between Two Organizations
The NEES uses a mutual average for each pair of organizations. This means the score reflects how both organizations view their relationship with each other. To calculate the mutual strength score between Organization A and Organization B:
- -
Take A’s rating of B. Take B’s rating of A.
- Add the two ratings together and divide by 2.
Mathematica ® Inc.
4
CYBHI NEES Toolkit
If more than one person from an organization completes the survey, first calculate that organization’s average rating of the other organization. To do this, add all of their ratings of the other organization together and divide by the number of people who responded about the other organization. Then use those organization-level averages to calculate the mutual score. Example (one respondent from each organization): One person from Organization A rates the relationship with Organization B as 2, and one person from Organization B rates the relationship with Organization A as 3. The mutual strength score is (2 + 3) ÷ 2 = 2.5. Example (multiple respondents from an organization): Three people from Organization A rate the relationship with Organization B as 2, 3, and 2. A’s average rating of the relationship is (2 + 3 + 2) ÷ 3 = 2.33. One person from Organization B rates the relationship with Organization A as 4. The mutual strength score is (2.33 + 4) ÷ 2 = 3.17 .
Handling one-way data When analyzing the strength of a relationship, ratings from both organizations reflect the shared perception of that partnership. However, sometimes only one organization provides a rating for the relationship. You can either: 1. Use the available rating as the strength score . For example, if A rates B but B does not respond, you use A’s rating as the strength score. 2. Exclude the pair. You might select this option if you want to focus only on fully reciprocal data, although you can add the pair back in once both organizations complete the survey.
Choose the approach that best fits your goals and your level of missing data.
Additional Helpful Metrics
These are additional metrics that can be easily calculated with your results:
1. County-wide average integration score. Average of all mutual strength scores.
• This metric provides a single number that a county can track periodically to assess how collaboration changes over time.
2. Integration score for each organization. Average of an organization’s mutual strength scores.
3. Count of strong connections . Number of relationships above a chosen threshold.
Step 4: Visualize the network Visualizing your network helps make patterns easier to see. A good visualization can highlight strong partnerships, show isolated organizations, and identify which sectors tend to work closely together. This visualization might look like:
Mathematica ® Inc.
5
CYBHI NEES Toolkit
A network map
Tools like Kumu or Graph Commons can create interactive network maps (we used Kumu for the example above). These maps help you explore how organizations are connected and where gaps exist. For example, you might find there is a missing connection where one is expected to be. A bar chart A simpler option is to create bar charts that rank organizations by their average strength score or number of strong connections. This can be easily created using Google Sheets, Excel, or PowerPoint. Step 5: Interpret and Share Findings
This step transforms your NEES results into insights your county can act on. The goal is not just to understand the data, but to use it to strengthen collaboration across your behavioral health ecosystem.
After reviewing your summary metrics and visualizations, consider bringing county leaders and partners together to reflect on the results. Sharing the findings transparently helps build a shared understanding of current collaboration patterns, areas of strength, and opportunities for deeper connection. These conversations often surface context and nuance that quantitative findings alone cannot provide, such as existing initiatives, historical relationships, or capacity constraints that influence how organizations work together. Before starting to share findings, consider whether you will share the names of the organizations that responded to the questionnaire. Depending on your unique context and the purpose of your survey, you may wish to keep the specific names of organizations private to protect respondent privacy. To do so, you can consider a label such as “social services provider” or “behavioral health provider”, such as the labels used in the example above. When describing governmental agencies, such as a “county behavioral health
Mathematica ® Inc.
6
CYBHI NEES Toolkit
agency”, it is impossible to mask the specific name of the agency. In these cases, consider using the agency name to increase clarity and understanding by the reader.
When reviewing findings collaboratively, partners can:
● Confirm whether results resonate with on-the-ground experience ● Identify root causes behind strong or weak connections ● Discuss whether certain gaps reflect missing relationships or simply missing data ● Prioritize relationships or sectors where deeper collaboration would have the greatest impact ● Identify immediate next steps or longer-term strategies to strengthen the system
This shared reflection is often the most valuable part of the NEES process. It transforms the tool from a descriptive snapshot into a catalyst for coordinated action.
Guiding Questions
Overall integration: ● What does the average strength score say about our county’s level of collaboration?
● Are most relationships in the “cooperate”, “coordinate”, or “collaborate” range?
● Do some sectors have notably stronger or weaker relationships?
Identifying hubs:
● Which organizations appear most connected? Least connected?
● Do these patterns align with what you expect? Are there collaborative structures that are working well to enable these hubs?
Identifying opportunities for further connection:
● Where are relationships weakest or missing?
● Are certain sectors or organizations siloed?
Change over time (if the NEES is administered several times):
● Have any relationships grown stronger or weaker?
● Are new partnerships and connections forming? Example takeaways -
“Our strongest connections are between county agencies and LEAs.”
- “County agencies serve as bridges across sectors that otherwise operate separately.”
- “CBO A shows more relationships rated ‘coordinate’ or stronger compared to last year.”
Mathematica ® Inc.
7
CYBHI NEES Toolkit
How Mathematica can help Mathematica provides services that can support counties at any stage of the NEES process. We are an approved California vendor for IT and non-IT services 2 , allowing counties to contract with us through existing contract vehicles. Mathematica can support your NEES efforts through a full spectrum of services, from light-touch technical assistance all the way up to full implementation support, including survey design and programming, outreach and survey fielding, data management and analysis, visualization and reporting, interpreting findings and identifying next steps, and facilitation of discussions to turn insights into actionable next steps. Contact us at cybhievaluation@mathematica-mpr.com to discuss your needs.
2 Non-IT Services CMAS # 4-25-09-1064; Technology, Digital and Data Consulting Agreement #5-22-70-25-19
Mathematica ® Inc.
8
Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8Made with FlippingBook Ebook Creator