COUNTY OF LOS ANGELES-DEPARTMENT OF MENTAL HEALTH

Program Development & Outcomes Bureau

 

Updated 3/26/2019

 

 

Frequently Asked Questions (FAQs) Regarding the Use of Outcomes Measures for Evidence Based Practices (EBP) and the Prevention and Early Intervention (PEI) Outcome Measure Application (OMA)

 

DATA COLLECTION When should I administer PEI Outcome Measures? What do you do if you cannot collect either a “Pre,” “Update,” or “Post” outcome measure? Can “Update” outcome measure scores be entered if “Pre” outcome measure scores were not collected? Can “Post” outcome measure scores be entered if “Pre” outcome measure scores were not collected? Should my agency collect a “Post” outcome measure(s) if an EBP was not fully completed? When does a lockout occur in PEI OMA, causing PEI OMA to reject data? How often/frequently should I enter data into PEI OMA? How do we collect and enter data in PEI OMA if the client is receiving multiple EBPs simultaneously from a single legal entity (e.g. ART and Seeking Safety)? If a client was too young to complete any PEI Outcome Measures do I still have to enter the client into PEI OMA? I made a mistake when I entered a client into PEI OMA and now I can’t fix it. What should I do? Can completed outcome measures be uploaded to a client's electronic health record (EHR)? ADMINISTRATION OF OUTCOME MEASURES Can you bill for administering, scoring, and interpreting outcome measures? Can “Pre” outcome measures be administered during the initial assessment phase? Can case managers and/or non-clinical staff persons administer, score or interpret outcome measures if they've received training on the measure? If a client has a birthday during treatment and ages out of the “Pre” outcome measure, do you still give the same “Post” outcome measure, and if necessary, “Update” outcome measure? Can family members, clinicians, or other persons provide interpretation of outcome measures from one language to another? What outcome measures may be administered verbally (i.e., read) to respondents? Can you answer a clarifying question that a respondent has about an outcome measure's item(s) meaning? What should be done if, during administration of an outcome measure, it is determined that the respondent does not comprehend the items? Is there a standard definition of the terms “EBP Completed” and “Client’s treatment a success,” in PEI OMA? Does the same caregiver have to complete the outcome measures at "Pre, "Update", and "Post" treatment? TRAINING How Can I register for a PEI Outcome Measures Training? Does each staff person who will be administering, scoring and/or interpreting outcome measures have to come to the PEI outcome measure trainings? DISTRIBUTION OF OUTCOME MEASURES How do I obtain outcome measures? Can outcome measures provided by LACDMH be sent to our agency either electronically or by mail? How do I get non-English versions of outcome measures? PTSD-RI Did DMH switch from using the PTSD-RI for DSM-IV to using the PTSD-RI for DSM-5 and PCL-5? If a child answers “No” to the first page of the Child/Adolescent Self-Report, that no “bad thing” happened to them, would you continue completing the measure or stop?  What if you have more information about specific trauma incidents which could be troubling the child but the child does not mention them when completing the outcome measure? OTHER OUTCOME MEASURES Do I have to administer and enter data for both the ECBI and SESBI-R? Do I have to administer and enter data for both the parent and teacher RBPC? USE OF OUTCOME MEASURE DATA Does DMH release PEI Outcomes data to providers?  How will data be used with regard to individual, agency, and EBP performance?

 

DATA COLLECTION

 

 

When should I administer PEI Outcome Measures?

 

"Pre/post" treatment outcome measures can be completed as early as 7 days prior to the date of the first/last session, on the date of the first/last session, and up to 14 days after the date of the first/last session. "Update" outcome measures should be administered every six months, if treatment lasts longer than 6 months. There is no hard administration window for "update" outcome measure completion.

 

 

What do you do if you cannot collect either a “Pre,” “Update,” or “Post” outcome measure?

 

There is an “Unable to Collect” field in PEI OMA. If a numeric score was not collected, instead provide the “Unable to Collect” reason.

 

 

 

Can “Update” outcome measure scores be entered if “Pre” outcome measure scores were not collected?

 

Yes. “Update” outcome measure scores can be entered into PEI OMA, as long as an “Unable to Collect” reason is entered in lieu of “Pre” outcome measure scores. Each required outcome measure needs be acknowledged in PEI OMA in one of two ways: (a) collect a numeric score or (b), choose an “Unable to Collect” reason if a score was not collected.

 

  

Can “Post” outcome measure scores be entered if “Pre” outcome measure scores were not collected?

 

Yes. DMH is interested in the “Post” treatment data as well as data reflecting change from “Pre” to “Post” treatment. Therefore, “Post” outcome measure scores should be entered into PEI OMA, as long as the "Pre" treatment outcome measures are already acknowledged in PEI OMA by choosing an "Unable to Collect" reason. Remember, each required outcome measure needs be acknowledged in PEI OMA in one of two ways: (a) collect a numeric score or (b), choose an “Unable to Collect” reason if a score was not collected.

 

 

Should my agency collect a “Post” outcome measure(s) if an EBP was not fully completed?

 

If the client does not complete the EBP,  "Post" measures may collected but their collection is not required by DMH. Each agency determines policies regarding collecting “Post” outcome measures when an EBP is not completed.  

  

However, entering End of Treatment Information into PEI OMA is still required, which includes date of last treatment session, number of treatment sessions, and treatment disposition, e.g., the client moved away or started a different treatment.

 

   

When does a lockout occur in PEI OMA, causing PEI OMA to reject data?

 

When the outcome measure is completed outside of the administration window or when a required field(s) is left blank, PEI OMA will generate an error message indicating that the data cannot be submitted.

 

 

How often/frequently should I enter data into PEI OMA?

 

Entering data as soon as it is available will ensure that any reports developed will contain the most up-to-date information. This in turn helps providers, DMH and the State receive the most up-to-date information. Persons entering data into PEI OMA should follow the policies/procedures set by each legal entity regarding the frequency of data submission into PEI OMA.

 

 

How do we collect and enter data in PEI OMA if the client is receiving multiple EBPs simultaneously from a single legal entity (e.g. ART and Seeking Safety)?

 

Before answering this question, it needs to be stated that in most cases, clients should be enrolled in only one PEI funded EBP at a time.

 

To answer the question posed above, data for clients enrolled in more than one EBP simultaneously can be entered into PEI OMA if the EBPs do not have the same focus, e.g., data can be entered for client simultaneously is enrolled in IPT, a depression focused treatment, and ART, a disruptive behavior focused treatment but not if the client is enrolled both in ART and PATHS, treatments that are both in the Disruptive Behavior Disorder focus.

 

 

If a client was too young to complete any PEI Outcome Measures do I still have to enter the client into PEI OMA?

 

Yes. In these situations you would open a Treatment Cycle in PEI OMA and provide the required client information. PEI OMA will recognize instances when an outcome measure's score(s) is not entered due to a client’s age being below minimum age requirement and it will not create space for entering a score(s).

 

 

I made a mistake when I entered a client into PEI OMA and now I can’t fix it. What should I do?

 

Certain entries can’t be edited or undone by users, for example, if you save the wrong Date of First Service or if you start your client under the wrong EBP. When the entry can't be edited, erase the client's treatment cycle, then create a new one with the correct information.

 

 

Can completed outcome measures be uploaded to a client's electronic health record (EHR)?

 

It depends on the copyright rules established by the outcome measure's publisher.

 

Completed outcome measures that may be uploaded to a client's record in an EHR:

• OQ Series (YOQ, YOQ-SR, OQ)
• PTSD-RI 
• PTSD-RI-5
• RCADS
• PHQ-9
• GAD-7
• ECBI
• SESBI-R
• TSCYC
• RBPC 
• FAD
• PCL-5
• PCL-C
• SOPS
• DERS

 

Completed outcome measures that may not be uploaded to the client's record in an EHR: 

• DECA-I/T
• PDS 

 

 

 

  

ADMINISTRATION OF OUTCOME MEASURES

 

 

Can you bill for administering, scoring, and interpreting outcome measures?

 

See Quality Assurance Bulletin February 3, 2017, No. 17-02, located in the Quality Assurance Bulletin page of LACDMH's internet site.

 

 

Can “Pre” outcome measures be administered during the initial assessment phase?

 

It depends. “Pre” outcome measures need to be administered within the administration window so agencies can elect to administer outcome measures at intake if the intake occurs 7 days or less prior to the treatment start date. Agencies should keep in mind several factors in making this determination: 1.) if there is a long wait between intake and the next session of the EBP, there is the potential that the “Pre” outcome measure score(s) collected during the initial intake won't be accepted by PEI OMA because it was completed outside of the administration window 3.) the challenges clinicians may face if they attempt to complete an initial assessment while simultaneously conducting the first session of an EBP and collecting outcome measures.

 

 

Can case managers and/or non-clinical staff persons administer, score or interpret outcome measures if they've received training on the measure?

 

For all PEI outcome measures, interpretation must be done by a clinical staff person. Each measure has its own rules regarding what constitutes a clinical staff person. The authors of each measure also have rules regarding who can administer and score outcome measures and these rules vary from measure to measure. Information on these rules is available in outcome measure Quick Guides, at outcome measure trainings and in outcome measure manuals.

 

 

If a client has a birthday during treatment and ages out of the “Pre” outcome measure, do you still give the same “Post” outcome measure, and if necessary, “Update” outcome measure?

 

Yes.

 

 

Can family members, clinicians, or other persons provide interpretation of outcome measures from one language to another?

 

No, this would invalidate the outcome measure because the person interpreting may not interpret items exactly as they are meant by the measure’s author. If an outcome measure cannot be completed due to language difficulties and there is no authorized translation in available, then the appropriate “Unable to Collect” reason code should be indicated in PEI OMA.

 

What outcome measures may be administered verbally (i.e., read) to respondents?

 

The following outcome measures may be verbally administered to respondents:

• OQ Series (YOQ, YOQ-SR, OQ)
• CBCL-1.5-5/CBCL
• DECA-I/T 
• ECBI
• PTSD-RI
• PTSD-RI-5 
• GAD-7
• PHQ-9
• DERS
• FAD
• PCL-5
• PCL-C 
• RCADS
• SESBI-R 
• YSR

 

 

Can you answer a clarifying question that a respondent has about an outcome measure's item(s) meaning?

 

Rules vary across outcome measures. Information on these rules is available in outcome measure Quick Guides, at outcome measure trainings, and in outcome measure manuals.

 

 

What should be done if, during administration of an outcome measure, it is determined that the respondent does not comprehend the items?

 

Administration of should be discontinued if the respondent cannot complete it in any of the appropriate modes of administration. The outcome measure would be considered invalid and the appropriate reason the score(s) was not collected should be entered into PEI OMA in the “Unable to Collect” section.

 

 

Is there a standard definition of the terms “EBP Completed” and “Client’s treatment a success,” in PEI OMA?

 

No, because rules vary across EBPs. Agencies should contact their EBP Practice Leads to clarify the practice’s definitions of “success in treatment” and “completion of treatment.” 

 

 

Does the same caregiver have to complete the outcome measures at "Pre, "Update", and "Post" treatment?

 

Yes. For example, a child in foster care begins treatment and his foster parent completes the "Pre" treatment YOQ. Then, half way through treatment the child is moved to a different foster home and has a different foster parent. The second foster parent should not complete a "Post" or "Update" YOQ. In PEI OMA, you would enter the correct "unable to collect" reason.

 

 

TRAINING

 

 

How Can I register for a PEI Outcome Measures Training?

  

Complete a PEI Outcomes Training Registration form, available on the OMW Wiki's Outcomes Questionnaire Training for Clinicians/Clinical Team page then email it to PEIoutcomes@dmh.lacounty.gov.

 

  

Does each staff person who will be administering, scoring and/or interpreting outcome measures have to come to the PEI outcome measure trainings?

 

DMH recommends that each agency send as many staff involved with PEI Outcome measures as possible to the PEI outcome measure trainings in order to receive the same training on the administration, scoring, and interpretation of these measures and to learn about the protocols for the collection and submission of outcome measure data. However,  agencies may send one representative to get trained on a measure and then have that person train other staff persons at the agency.

 

 

DISTRIBUTION OF OUTCOME MEASURES

 

 

How do I obtain outcome measures?

 

• Outcome measures available freely online:
• PHQ-9 www.phqscreeners.com  
• GAD-7 www.phqscreeners.com
• PCL-C www.ptsd.va.gov/professional
• PCL-5 DSM-5 Trauma Measures page on the OMA Wiki
• RCADS http://www.childfirst.ucla.edu/resources.html

 

• Outcome measures that can be ordered either from DMH or vendors (Note, DMH does not provides reimbursement for the cost of measures the agency purchases):
• YOQ
• YOQ-SR
• OQ
• PTSD-RI
• ***PTSD-RI for DSM-5***, instructions for obtaining this set of measure are available on the DSM-5 Trauma Measures page of the OMA Wiki.
• FAD
• DECA I/T 
• ECBI 

 

• SESBI–R
• PDS
• RBPC 
• TSCYC

 

• Outcome measures we don't carry that agencies may purchased from the publisher (Note, DMH does not provides reimbursement for the cost of measures they agency purchases):

 

• CBCL 6-18
• C-TRF
• TRF
• YSR

 

 

  

To order outcome measures from DMH, complete a PEI Outcome Measures Order form, available on the Forms, Guides and Supports section of OMA Wiki's PEI Outcomes Frontpage, then email it to PEIoutcomes@dmh.lacounty.gov.

 

 

Can outcome measures provided by LACDMH be sent to our agency either electronically or by mail?

 

The only outcome measure DMH may send electronically is the PTSD-RI for DSM-5. All other outcome measures must be distributed, in person, to agency personnel by a staff member from the DMH's Program Development and Outcomes Bureau.

 

 

How do I get non-English versions of outcome measures?

 

 The options are:

• Option 1: DMH has some translated outcome measures available and you may order them from DMH. It is the goal of DMH to eventually have all outcome measures translated into each of the County's 13 threshold languages. DMH will provide updates as new translated outcome measures become available. To find out which translations are currently available for distribution by DMH, refer to the PEI Outcome Measures Table located in the Forms, Guides and Supports section of OMA Wiki's PEI Outcomes Frontpage.

 

• Option 2: Legal entities can contact vendors selling the outcome measure to see if translations of the outcome measure are available for purchase. If the outcome measure is not currently available in a particular language, the legal entity can attempt to work with the vendor to create a translation.

 

 

PTSD-RI

 

 

Did DMH switch from using the PTSD-RI for DSM-IV to using the PTSD-RI for DSM-5 and PCL-5?

 

Yes. PEI practices that were using the UCLA Posttraumatic Stress Disorder-Reaction Index (PTSD-RI) for DSM-IV, both the Child/Adolescent and Parent versions, switched to new versions that were updated to comply with the new DSM-5 diagnostic criteria. In addition, the UCLA PTSD-RI Adult Short Form was replaced by the Posttraumatic Stress Disorder Checklist-5 (PCL-5) for clients 19 and above. These new measures are required to be used for PEI treatment cycles started on/after November 1, 2015. More information on the changes and instructions for obtaining the PTSD-RI-5 and PCL-5 are available on the DSM-5 Trauma Measures page.

 

 

If a child answers “No” to the first page of the Child/Adolescent Self-Report, that no “bad thing” happened to them, would you continue completing the measure or stop? 

   

You would discontinue administration if a client did not meet criteria A (were not exposed to a single traumatic event). The first part of the PTSD - RI Child/Adolescent version (the "Trauma History Profile" and the "Self-Report Trauma History Profile") establish if Criterion A was met; in other words, whether or not the client was exposed to at least one traumatic event in their lives. In PEI OMA, you would select the most appropriate "Unable to Collect" reason.

 

 

What if you have more information about specific trauma incidents which could be troubling the child but the child does not mention them when completing the outcome measure?

 

You can make a note of supplemental information on the Trauma History Profile and in the client’s charting notes but allow the child to  self-identify what is bothering him/her the most (Self-Report Trauma Profile) and his/her reactions to it (Reaction Index). The outcome measure is designed to isolate the trauma experience and symptoms the child reports are most bothersome at the time. If the child later identifies a different trauma as most bothersome you may administer the Self-Report Trauma Profile and the Reaction Index again.

 

 

 

OTHER OUTCOME MEASURES

 

 

Do I have to administer and enter data for both the ECBI and SESBI-R?

 

No.  It is recommended to administer the ECBI if possible and administer the SESBI-R when, in your judgment, the ECBI should not or cannot be administered.

           

If you do have scores for the ECBI and SESBI-R, you may enter both. If you have scores for one of the two outcome measures you would enter the scores for the one you collected and enter the other measure as “Unable to Collect."

 

Do I have to administer and enter data for both the parent and teacher RBPC?

 

No. While you may administer both the RBPC Parent and RBPC teacher you do not have to administer both. Administering the RBPC to a parent/caregiver is recommended, if possible, and administration to teacher is recommended when, in your judgment, the outcome measure should not be or cannot be administered to a parent/caregiver.  

           

Regarding data entry for the RBPC, you can enter scores for the teacher or the parent. If you have valid scores for both, use your best clinical judgment in determining which to enter into PEI OMA. 

 

 

USE OF OUTCOME MEASURE DATA

 

 

Does DMH release PEI Outcomes data to providers? 

  

Not but we plan to make PEI OMA data available for providers to create their own reports. This will be part of the third release of PEI OMA. Until the third version is released, request PEI Outcomes reports from DMH using the PEI Outcomes Report order form, available in the Forms, Guides and Supports section of OMA Wiki's PEI Outcomes Frontpage.

 

 

How will data be used with regard to individual, agency, and EBP performance?

 

Data will be utilized at multiple levels (e.g., client, service area, state) to evaluate services and inform decisions.

 

 

 

Return to PEI Outcomes main page.