Hello fellow evaluators,
A couple of weeks ago (on Monday, June 29), we held our first roundtable discussion using the OD2A Eval CoP online platform. To facilitate the discussion, a workbook was provided.
Though we had a robust discussion without the help of the workbook, some had commented that it might be a useful tool for facilitating discussions with the other OD2A team members or even with outside contractors. In this blog, I would like to demonstrate how this tool might be used to come up with talking points for these discussions.
The workbook uses the CDC evaluation framework and an established facilitation method (ORID) to guide the discussion about the evaluation of a particular activity. For each step in the framework, there are four types of questions asked:
- An Objective question,
- a Reflective question,
- an Interpretive question,
- and a Decisional question.
I applied these steps and questions to the following activity: Provide training in evidence-based interventions for SUD care teams based in FQHCs.
This is the table that I created by using the discussion workbook to address this activity. (See below.) Before meeting with my team or with the contractor that is carrying out the above activity, I would populate this table. The hope is that such preparation would help the conversation be more productive. I have not tried it yet but when I do, I will let you know how it went. 😊
Excelsior!
JKF
Engage Stakeholders
| ORID Questions | Talking Points |
| How are you engaging stakeholders when conducting this activity? | Working with the contractor and the FQHC directors |
| What do you find most difficult about engaging these stakeholders? | Getting access to the care team members |
| What would be helpful to you when engaging these stakeholders? | Working with FQHC directors to gain access to care team members |
| Regarding this step, what do you plan to do now? | Connect with the FQHC directors |
Describe the program/activity
| ORID Questions | Talking Points |
| How have you adapted the OD2A logic model as it pertains this activity? | I have not yet adapted the logic model to fit this activity |
| About which part of this logic model adaptation do you feel most uncertain? | When to begin measuring outcomes |
| What would bring greater clarity to the description of this activity? | When the interventions will occur and what to expect in terms of outcomes from those interventions |
| Regarding this step, what do you plan to do now? | Get a better handle on what to expect from the interventions |
Focus evaluation design
| ORID Questions | Talking Points |
| What evaluation question(s) have you decided to pursue for this activity? | Questions from the evaluation profile—especially interested in process questions right now |
| Which of these questions do you feel is most crucial to answer? | Context questions about process |
| What is the best approach to answering this question? | Key informant interviews |
| Regarding this step, what do you plan to do now? | Continue conducting interviews |
Gather credible evidence
| ORID Questions | Talking Points |
| What data sources and/or collection methods are you planning to use to answer this question? | Mostly interviews. The contractor will conduct surveys and pre-/post-tests. |
| With which data source and/or collection method do you feel most uncomfortable or unfamiliar? | Surveillance data sources, including knowing when they are helpful or can be used as cross-cutting indicators |
| What steps could you take to improve your competence and confidence? | Work closely with the surveillance team |
| Regarding this step, what do you plan to do now? | Continue to work closely with the surveillance team |
Justify conclusions
| ORID Questions | Talking Points |
| What types of analyses are you planning to use once the data is collected for this activity? | Have not got this far yet. I plan to use FQHC directors and care team members to help with analysis and interpretation |
| How confident are you with the types of analysis that you plan to use? | Not confident |
| What steps could you take to improve your competence and confidence? | Looking to enhance skillset via CoP connections and working with external evaluators |
| Regarding this step, what do you plan to do now? | Continue to work with the CoP and external evaluators |
Ensure use and lessons learned
| ORID Questions | Talking Points |
| How do you plan to disseminate the evaluation results for this activity to stakeholders? | Through the community workgroups in our stakeholder group. |
| In which results do you think your stakeholders will be most interested? | They will want to see increased preparedness and response at the local level |
| What can you do to make it more likely that your stakeholders will actually use the evaluation results to make future decisions? | Incorporate the evaluation results into the deliverables of the workgroups |
| Regarding this step, what do you plan to do now? | Create the workgroups and then determine level of interest and influence in this activity |
Leave a Reply