33
THE LEGAL BRIEF
VOLUME 42, ISSUE 3
(Continued from page 28) Bias in Judicial Performance Evaluations, the Problem and Proposed Solutions by Andrea Răchită and Judge Rebecca Glasgow
Bar associations can also focus the respondent on observable, concrete courtroom behaviors by introducing the survey with instructions for a structured free recall exercise to be performed before the person answers the survey questions. Structured free recall tasks prompt respondents to recall and list specific observed behaviors, both positive and negative, before beginning the survey. This brings actual behaviors to front of mind before the respondent answers survey questions. Preliminary instructions can also expressly tell respondents not to base answers on reputation, hearsay, rumor, or social contact, but instead on professional behavior observed in the workplace. Question drafters should take care to avoid double barreled questions that force the respondent to consider more than one observable behavior in a single question. Terms that may be susceptible to multiple interpretations should be defined. And if the survey requires the respondent to rate the judge on a scale, then the scale should be well defined and appropriate for the individual question, for example (5= almost always, 4= frequently, 3=sometimes, 2=rarely, 1=almost never), rather than an undefined scale of 1 to 5. The limited research on judicial evaluations shows bias is more prevalent in answers to open ended questions that are not focused on observable behaviors. Studies of employee evaluations show similar issues with bias in open ended comments. At the very least, bar associations should not publish narrative responses to such questions because the evidence that they tend to be infused with bias is so strong. Bar associations should also recognize that questions like, “ should Judge Doe be retained? ” are particularly vulnerable to bias because they contain no defined criteria or connection to observable behavior. Consider eliminating this question altogether or publishing a note with the results that allows readers to recognize that studies have found disparate responses to this question based on immutable characteristics like race and gender.
In sum, as the TCBA revisits its judicial performance evaluation survey, it should consider:
• Whether the TCBA should continue to conduct the survey • What are the purposes and goals of the survey? Improving judicial performance? Informing appointing authorities and voters? Something else? • Whether the pool of respondents should be expanded beyond the bar association members • Whether attorneys should be invited to respond only if they have appeared in front of the rated judicial officer within the previous year • Whether improvements can be made to preliminary instructions to both expressly call out the potential for bias and to bring concrete courtroom behaviors, good and bad, to front of mind before the survey is completed • Whether some questions should be eliminated, especially the opportunity to provide narrative comments and the question: Would you vote to re - elect this judge? • Whether double barreled questions have been divided into separate questions • Whether key terms and scales have been clearly defined • Whether there is information that should be published with the survey results to promote better public understanding of the limitations of the survey
Made with FlippingBook - professional solution for displaying marketing and sales documents online