|
Publications with Abstracts
|
Categories
- Conceptual Foundations of DBR
(C)
- Psychometric foundations related to
single item DBR scale instrumentation (I)
- Psychometric foundations related to
single item DBR scale procedures (P)
- Validation of single item DBR scales
in progress monitoring assessment (PM)
- Validation of single item DBR scales in
screening assessment (S)
|
Conceptual Foundations of DBR
Chafouleas, S.M., Riley-Tillman, T.C. &
Christ, T.J. (in press). Direct Behavior Rating (DBR): An
Emerging method for assessing social behavior within a
tiered intervention system.Assessment for Effective
Intervention.
Abstract - School-based professionals have long
relied on informal rating of student behavior for the
purpose of understanding, communicating about, and
intervening with behavior. Despite this history of use,
attention within the literature has not been systematically
organized around the possibilities of such ratings in
assessment for effective intervention. In an era of
data-based and outcomes-oriented decision making for all
students, capitalizing on the contextual relevance and
efficiency of these ratings is important, assuming
technical adequacy can be clearly established. Through this
special series, we continue the presentation of Direct
Behavior Rating (DBR) as a behavior assessment method,
which encompasses standardized versions of these types of
behavior ratings. In this introduction, we discuss the
conceptual foundation for DBR as a unique and potentially
viable behavior assessment method. This contemporary
framework should facilitate organization of lines of
research dedicated to evaluating the potential of DBR use
within problem-solving assessment models, such as used in
multi-tiered intervention systems. Briefly, within a tiered
intervention framework, schools are charged with the task
of using data to determine how to systematically align
resources so that solid core instruction is provided to all
students, core plus additional supports are provided to
some students, and core plus intensive supports are
provided to a few students. As noted within this
definition, use of data implies that good assessment
sources are available for use across relevant domains of
student functioning. And, it has become increasingly clear
that the domain of behavior is highly important toward
understanding overall student success. This implication
regarding the necessity of data about behavior forms the
rationale for this special series, which is to extend the
empirical presentation of DBR as an emerging option for
behavior assessment within a multi-tiered intervention
model.
Other categories: N/A
Christ, T.J., Riley-Tillman, T.C., &
Chafouleas, S.M. (in press). Foundation for the
development and use of Direct Behavior Rating (DBR) to
assess and evaluate student behavior.Assessment for
Effective Intervention.
Abstract - Direct Behavior Rating (DBR) is a
method of social-emotional and behavior assessment that
combines the immediacy of systematic direct observation
and the efficiency of behavior rating scales. The purpose
of this paper is to document the defensibility and
usability of DBR. This article provides a brief summary
of 1) the past, present and future directions of
social-emotional and behavior assessment methods in
schools; 2) the defining features of DBR; 3) the guiding
principles for DBR development and evaluation; and 4) a
brief synthesis of DBR research. Special emphasis is
placed on DBR single item scales (DBR-SIS) and three
general outcome behaviors that are most relevant for use
in schools. Research and recommendations for standard
DBR-SIS instrumentation and procedures are reviewed,
along with future directions for research and
practice.
Other Categories: N/A
Sanetti, L., Chafouleas, S.M., Christ, T.
J., Gritter, K. (in press). Extending DBR use beyond
student assessment: Applications to treatment integrity:
Assessment within a multi-tier model of school-based
intervention delivery.Assessment for Effective
Intervention.
Abstract - To make data-based decisions about
intervention effectiveness, it is essential to
formatively assess student outcomes along with treatment
integrity. Over the past decade, there have been
significant advances in technologies for formative
assessment of student outcomes, yet treatment integrity
assessment has not experienced the same development. Few
treatment integrity assessments with adequate
psychometric properties are available, and of those, most
are intervention-specific and thus have questionable use
most of those that are available are intervention
specific. Results of several recent studies demonstrate
that Direct Behavior Rating (DBR) can be used as a social
behavior assessment tool. To date, DBR has been evaluated
solely as a tool for assessing student behavior. However,
DBR represents an assessment tool that could be extended
to treatment integrity and may result in an efficient and
feasible treatment integrity assessment method that could
be applied across multiple types of interventions. The
purpose of this paper is to review how DBR can be
extended to treatment integrity assessment. Implications
for research and the practice of school psychology will
be addressed.
Other Categories:N/A
Christ, T.J., &
Boice, C.M. (in press). Rating scale items: A brief
review of nomenclature, components, and formatting to
inform the development of Direct Behavior Rating (DBR).
Assessment for Effective Intervention.
Abstract - Ratings scales are a common component
of many multi-source, multi-method frameworks for
social-emotional and behavior assessment of children.
There is a modest literature-base to support the use of
attitudinal, behavioral, and personality rating scales.
Much of that historic literature focuses on the
characteristics and interpretations of specific scales,
which are mostly Likert-type scales. There are many more
scale-types and item-types that receive less attention
within the literature and less application in practice.
This article provides a brief summary of the literature
relevant to formats, components, and nomenclature
associated with rating scale item types. This article is
intended to provide basic information that might inspire
and contribute to the development and evaluation of novel
rating scales with a variety of item types; especially
those relevant to Direct Behavior Rating (DBR) methods of
assessment.
Other Categories: N/A
Chafouleas, S.M., Riley-Tillman, T.C., Christ,
T.J., & Kilgus, S.P. (in press). Direct Behavior
Ratings: Linking assessment, communication, and
intervention. In A. Canter, L. Paige, and S. Shaw (Eds),
Helping children at home and school II: Handouts for
families and educators. Bethesda, MD: National Association
of School Psychologists.
Abstract - NASP Chapter in Helping children at
home and school: Handouts from your school psychologist
focusing on the assessment application of Direct Behavior
Ratings.
Other Categories: PM
Clayton, K., Hutcheson, S., Kilgus, S., Harrison,
S., & Music, A. (2009). The development and evaluation
of Direct Behavior Ratings (DBRs): Past, present, and
future. SASP News, 1, 26-30.
Abstract - Many assessment tools are available for
quickly and accurately evaluating children’s academic
progress. However, there are very few assessment tools for
monitoring children’s behavior without using time
intensive methods. Thus, in order to efficiently understand
and intervene with problematic behavior, quick and accurate
behavioral assessments are also needed. The purpose of this
review is to highlight the theoretical foundations, present
state of research and development, and future directions of
this type of behavioral assessment.
Other Categories: N/A
Chafouleas, S.M, Riley-Tillman, T.C., Sugai, G.
(2007). School-based behavior assessment and monitoring for
informing instruction and intervention. New York, NY: The
Guilford Press
Description from Publisher - This indispensable
book helps school practitioners go beyond putting a label
on student behavior problems--it shows how to use
assessment to partner with teachers and students to develop
effective solutions. The authors provide a framework for
determining the types of behavioral data that are needed in
a given situation, selecting appropriate measures, and
interpreting and organizing the results. Case examples tie
it all together, demonstrating how different assessment
strategies can be used to support positive behavior and
monitor progress at the level of the individual, the
classroom, or the entire school. In a large-size format
with convenient lay-flat binding, the volume features 15
reproducible checklists and forms.
Other Categories: N/A
Riley-Tillman, T.C., Chafouleas, S.M., &
Briesch, A.M. (2007). A school practitioner’s guide
to using Daily Behavior Report Cards to monitor
interventions. Psychology in the Schools,
44, 77-89.
Abstract - With the growing popularity of a
response to intervention model of service delivery, the
role of intervention management is becoming more prominent.
Although many aspects of intervention management have
received significant attention, one area that warrants
further development involves feasible methods for
monitoring student behavior in a formative fashion. By
formative, we mean behavior that is frequently monitored,
such as on a daily basis, with the premise that the
information will be used to make appropriate intervention
decisions. Within a problem solving model of intervention
development, implementation, and evaluation, at least one
educational professional must be responsible for using an
effective tool for monitoring behavior. Yet, identifying
and using such a tool can be a challenge in applied
settings in which resources are often limited. The purpose
of this paper is to briefly review available tools for
behavior monitoring, with emphasis on reviewing the
potential of the Daily Behavior Report Card (DBRC) to serve
as a supportive methodology to more established measures of
behavior assessment. Examples and guidelines for the use of
the Daily Behavior Report Card in behavior monitoring are
provided.
Other Categories: PM
Riley-Tillman, T.C., & Chafouleas, S.M.,
Eckert, T., & Kelleher, C. (2005). Bridging the gap
between research and practice: A framework for building
research agendas in school psychology. Psychology in
the Schools, 42, 459-473.
Abstract - In this article, we discuss the history
behind efforts to transfer school psychology research into
practice and review the literature pertaining to treatment
acceptability, participatory action research,
organizational change, and generalization programming. We
then present a model for the systematic programming of this
transfer and propose a three-step framework that emphasizes
multiple conceptual bases to transfer research into
practice. This three-step framework includes creating
usable knowledge, transferring usable knowledge, and
supporting usable knowledge. It is our intention that the
proposed framework will provide a starting point that can
be informed by researchers and practitioners in the field
of school psychology. We conclude with examples of our
research efforts to systematically study the transfer of
research into practice.
Other Categories: N/A
Riley-Tillman, T.C., Kalberer, S.M., Chafouleas,
S.M. (2005). Selecting the right tool for the job: A review
of behavior monitoring tools used to assess student
response to intervention. The California School
Psychologist, 10, 81-91.
Abstract - The educational accountability movement
has demanded that educators implement and also monitor
students’ responses to positive behavioral
accommodations in schools as well as communicate this
information to others. This new responsibility has left
many educators struggling with ways to monitor students
effectively. This article provides a brief overview of how
to choose a behavioral monitor strategy effectively. Four
options for behavioral monitoring are reviewed: (a)
permanent products; (b) behavior rating scales; (c)
systematic direct observation; and (d) behavior report
cards. In addition, the strengths and weaknesses of each
method are discussed along six areas of consideration: (a)
goodness of fit; (b) directness; (c) generalization; (d)
feasibility; (e) training; and (f) intrusiveness. Finally,
the methods are considered in relation to each stage of the
intervention process. This article provides a brief guide
for school-based professionals focusing on behavior
problems – one that provides multiple options for
assessment and monitoring procedures and outlines
considerations for selecting among these options.
Other Categories: PM
Chafouleas, S.M., Riley-Tillman, T.C., &
McDougal, J.L. (2004). Daily Behavior Report Cards (DBRCs):
Useful tools for monitoring and changing behavior. In A.
Canter, L. Paige, S. Carroll and I. Romero (Eds.),
Helping Children at Home and School: Handouts From Your
School Psychologist (Second Edition). Bethesda, MD:
National Association of School Psychologists.
Abstract - NASP Chapter in Helping children at
home and school: Handouts from your school psychologist
focusing on the assessment application of Daily Behavior
Report Cards.
Other Categories: PM
Chafouleas, S.M., Riley-Tillman, T.C., &
McDougal, J.L. (2002). Good, bad, or in-between: How does
the daily behavior report card rate? Psychology in the
Schools, 39, 157-169.
Abstract - The purpose of this paper was to define
and review the daily behavior report card (DBRC) as a
monitoring and/or intervention technique. For this paper, a
measure was considered a DBRC if a specified behavior was
rated at least daily, and that information was shared with
someone other than the rater. In general, it has been
suggested that DBRCs may be feasible (e.g., Nolan, &
Gadow, 1994; Pelham, 1993), acceptable (e.g., Turco &
Elliott, 1986), effective in promoting positive student
behavior (e.g., Blechman, Schrader, & Taylor, 1981;
Dougherty, & Dougherty, 1977), and a way to increase
parent/teacher communication (e.g., McCain & Kelley,
1993). In addition, DBRCs are highly adaptive in that they
represent a broad array of both monitoring and intervention
possibilities rather than having a single, scripted
purpose. All of these characteristics make the DBRC
appealing for use in applied settings. However, an
extensive, methodologically-sound literature base does not
yet exist. Despite the appeal of using DBRCs, widespread
endorsement cannot be made without caution. The paper
concludes with implications for use in practice and
highlights areas in need of further investigation.
Other Categories: PM
|
Psychometric foundations related to single item DBR scale instrumentation (I)
Riley-Tillman, T.C., Chafouleas, S.M., Christ,
T.J., Briesch, A.M., & LeBel, T.J. (2009). The impact
of wording and behavioral specificity on the accuracy of
Direct Behavior Ratings (DBRs). School Psychology
Quarterly, 24, 1-12.
Abstract - Direct Behavior Ratings (DBRs) combine
aspects of both systematic direct observation and behavior
rating scales to create a feasible method for social
behavior assessment within a problem solving model. The
purpose of the current study was to examine whether
accuracy of DBR ratings was impacted depending on the
behaviors selected to be rated using a DBR. Specifically,
the impact target behavior wording (positive vs. negative)
and degree of specificity by which the behaviors were
defined was investigated. One hundred and forty five
participants, assigned to one of four experimental
conditions, were asked to view video clips of a classroom
setting and rate target student behavior. Results indicated
that the wording and specificity of behavior included on a
DBR may influence the accuracy of ratings. The most
accurate ratings of academically engaged behavior occurred
when the behavior was defined with a global definition and
positive wording. In contrast, the most accurate ratings of
disruptive behavior were obtained with a global definition
and either positive or negative wording. Limitations,
implications and future directions are discussed.
Other Categories: N/A
Chafouleas, S. M., Christ, T. J., &
Riley-Tillman, T. C. (2009). Generalizability and
dependability of scaling gradients on direct behavior
ratings. Journal of Educational Measurement, 69,
157-173.
Abstract - Generalizability theory was used to
examine the dependability of scaling gradients of Direct
Behavior Ratings (DBRs). DBRs refer to a type of rating
scale in which a brief rating occurs following
observation of a target behavior and requires minimal
inference to discern the target(s) of measurement.
Variance components associated with scale gradients were
estimated using a random effects design for persons (p)
by raters (r) by observations (o). Participants included
106 undergraduate students who viewed a videotape of a
student engaged in a difficult task, and then rated the
behaviors of 6 students on two behaviors across three
observations. Results suggested that scale gradient did
not substantially contribute to observed score variance.
In contrast, the largest proportions of variance were
attributed to rater and error. Implications, limitations,
and future research considerations are discussed.
Other Categories: N/A
Chafouleas, S.M., Christ, T., Riley-Tillman,
T.C., Briesch, A.M., & Chanese, J. (2007).
Generalizability and dependability of Direct Behavior
Ratings to measure social behavior of preschoolers.
School Psychology Review, 36, 63-79.
Abstract - One potentially feasible tool for use
in the formative assessment of social behavior is the
Daily Behavior Report Card (DBRC), yet empirical support
for the reliability of its use in progress monitoring is
limited. In this study, generalizabilty theory was used
to provide preliminary data regarding the
generalizability and dependability of the Daily Behavior
Report Card (DBRC) to measure the social behavior of
preschoolers. Two typical preschool behaviors (Works to
Resolve Conflicts, Interacts Cooperatively with Peers)
were selected for investigation within the DBRC created
for this study. Overall, results varied depending on
which behavior was rated and the number of raters whose
ratings were considered. The results suggested that a
fairly substantial proportion of measurement variance was
attributable to the different raters, and that the 4
raters varied in their mean level of ratings within and
across the 15 students. In addition, although the actual
number of days was dependent on the number of ratings
collected per day, results suggested DBRCs are likely to
approximate or exceed .70 after seven ratings are
collected across four to seven days, and .90 after 10
ratings. Limitations, future directions, and implications
are discussed.
Other Categories: N/A
|
Psychometric foundations related to single item DBR scale procedures (P)
Schlientz, M.D., Riley-Tillman, T.C., Briesch,
A.M., Walcott, C.M., & Chafouleas, S.M. (in press). The
impact of training on the accuracy of Direct Behavior
Ratings (DBRs). School Psychology
Quarterly.
Abstract - To date, extant research has not
established how rater training affects the accuracy of data
yielded from Direct Behavior Ratings (DBRs). The purpose of
the current study was to examine whether providing users of
DBRs with a training session utilizing practice and
performance feedback would increase rating accuracy. It was
hypothesized that exposure to direct training procedures
would result in greater accuracy than exposure to a brief
familiarization training session. Results were consistent
with initial hypotheses in that ratings conducted by
trained participants were more accurate than those
conducted by the untrained participants. Implications for
future practice and research are discussed.
Other Categories: N/A
LeBel, T.J., Kilgus, S.P., Briesch, A.M., &
Chafouleas, S.M. (in press). The impact of training on
the accuracy of teacher-completed Direct Behavior Ratings
(DBRs). Journal of Positive Behavioral
Interventions.
Abstract - The purpose of this study was to
investigate the effects of three levels of training
(direct, indirect, none) on teachers’ ability to
accurately rate video of student behavior. Direct and
indirect training groups received 20-minute instructional
sessions on DBRs, with the direct group receiving
opportunities for practice and feedback. Each training
group viewed a video of a typical classroom environment,
and then rated the proportion of time the student
displayed disruptive behavior and academic engagement.
Accuracy was examined through comparison of the teacher
rating with direct observation data. Teacher
acceptability of using the DBR was also assessed.
Overall, results suggested that the most intensive level
of training (i.e., direct) did not improve the accuracy
with which teachers rated behaviors using a DBR.
Limitations and future implications are discussed.
Other Categories: N/A
|
Validation of single item DBR scales
in progress monitoring assessment (PM)
Riley-Tillman, T.C., Methe, S.A., & Weegar,
K. (in press). Examining the Use of Direct Behavior Rating
Methodology on Classwide Formative Assessment: A Case
Study. Assessment for Effective Intervention.
Abstract - High quality formative assessment data
are critical to the successful application of any
problem-solving model (e.g. Response to Intervention).
Formative data available for both academic and social
behavior outcomes, and at different levels (e.g.
individual, class, school), facilitates decisions about
student responsiveness. The purpose of the current study
was to examine the use of an individualized Direct Behavior
Rating (DBR) methodology in the assessment of a whole class
social behavior. A class-wide intervention was applied in
an A-B-A-B-A design using both systematic direct
observation (SDO) and class-wide DBR collected to evaluate
effects. Results indicated that class-wide DBR data were
consistent with SDO data across phases, suggesting that DBR
data are sensitive to classroom-level intervention effects.
Implications for the use of such data in multi-tiered
problem solving are discussed.
Other Categories: N/A
Riley-Tillman, T.C., Chafouleas, S.M., &
Eckert, T. (2008). Daily Behavior Report Cards and
Systematic Direct Observation: An investigation of the
acceptability, reported training and use, and decision
reliability among school psychologists. Journal of
Behavioral Education, 17, 313-327.
Abstract - More than ever, educators require
assessment procedures and instrumentation that are
technically adequate as well as efficient to guide
data-based decision making. Thus, there is a need to
understand perceptions of available tools, and the
decisions made when using collected data, by the primary
users of those data. In this paper, two studies that
surveyed members of the National Association of School
Psychologists with regard to two procedures useful in
formative assessment, (i.e. Daily Behavior Report Cards;
Systematic Direct Observation), are presented.
Participants reported greater overall levels of training
and use of Systematic Direct Observation than Daily
Behavior Report Cards, yet both techniques were rated as
equally acceptable for use in formative assessment.
Furthermore, findings supported that school psychologists
tend to make similar intervention decisions when
presented with both types of data. Implications,
limitations, and future directions are discussed.
Other Categories: N/A
Riley-Tillman, T.C., Chafouleas, S.M., Sassu,
K.A., Chanese, J.A.M., & Glazer, A.D. (2008).
Examining the agreement of Direct Behavior Ratings and
Systematic Direct Observation for on-task and disruptive
behavior. Journal of Positive Behavior Interventions,
10, 136-143.
Abstract - The purpose of this study was to
replicate previous findings indicating a moderate
association between teacher perceptions of behavior as
measured by direct behavior ratings (DBRs) and systematic
direct observation (SDO) conducted by an external
observer. In this study, data regarding student on-task
and disruptive behavior were collected via SDO from
trained external observers and via DBRs from classroom
teachers. Data were collected across 15 teachers and
three observation sessions, and the agreement between the
two methods was compared as a way to examine concurrent
validity. Results supported previous work suggesting that
DBRs are significantly correlated with SDO data, thereby
suggesting that the DBR might be used as a compatible
tool with SDO. Implications for practice, limitations of
the study, and directions for future research are
discussed.
Other Categories: N/A
Chafouleas, S.M., & Riley-Tillman, T.C.,
Sassu, K.A., LaFrance, M.J., & Patwa, S.S. (2007).
The consistency of Daily Behavior Report Cards in
monitoring interventions. Journal of Positive
Behavior Interventions. 9, 30-37.
Abstract - The consistency of data collected
across raters using either a daily behavior report card
(DBRC) or direct observation was examined. Three teachers
served as participants who used the DBRC to rate a
student’s on-task behavior while two observers
simultaneously collected information about that
student’s behavior. One observer also completed a
DBRC whereas the other conducted direct observation.
Overall, results provided further evidence that a
performance-based behavior recording such as the DBRC can
provide data consistent with that obtained via direct
observation or another rater using a DBRC. Although the
observations across all ratings were generally
consistent, some variation did occur, particularly with
regard to the teacher DBRC rating. Implications,
limitations and future research directions are
discussed.
Other Categories: I, P
Chafouleas, S.M., Riley-Tillman, T.C., &
Sassu, K.A. (2006). Acceptability and reported use of
Daily Behavior Report Cards among teachers. Journal
of Positive Behavior Interventions, 8,
174-182.
Abstract - In this study, a sample of teachers
was surveyed regarding their reported use and
acceptability of daily behavior report cards (DBRCs).
Almost two thirds of responding teachers indicated that
they have used versions of DBRCs in their practice.
Respondents’ use of DBRCs was not restricted to a
single purpose or situation. Additional findings
suggested that the format of DBRCs varies widely,
suggesting that teachers have found the DBRC to be highly
adaptive in representing a broad array of possibilities
rather than having a single, scripted purpose. An
additional noteworthy finding relates to the general
acceptance of DBRCs by teachers as both
behavior-monitoring tools and as components in
interventions. In summary, results provide support to
previous claims that the DBRC is both a used and accepted
tool in practice, suggesting that DBRCs deserve closer
attention in research and practice related to positive
behavior supports. Limitations, future directions, and
implications are discussed.
Other Categories: I, P
Chafouleas, S.M., McDougal, J.L.,
Riley-Tillman, T.C., Panahon, C.J., & Hilt, A.M.
(2005). What do Daily Behavior Report Cards (DBRCs)
measure? An initial comparison of DBRCs with direct
observation for off-task behavior. Psychology in the
Schools, 42, 669-676.
Abstract - This study investigated the
similarity of information provided from a daily behavior
report card (DBRC) as rated by the teacher to direct
observation data obtained from external observers. In
addition, the similarity of ratings was compared with
variations of problem severity (mild, severe) and teacher
training (none, some). Results suggested a moderate
association between teacher perceptions of behavior as
measured by DBRC ratings and direct observation conducted
by an external observer. In addition, 23 to 45% of the
variance in DBRC ratings was consistent with the direct
observation data. Severity of the behavior problem or the
inclusion of training was not found to significantly
impact the similarity of ratings. In summary, results
tentatively suggest that the DBRC may be a viable
supplement to direct observation for estimating behavior
in applied settings. Limitations, future research
directions, and implications are discussed.
Other Categories: N/A
|
Validation of single item DBR scales
in screening assessment (S)
Chafouleas, S.M., Kilgus, S.P., & Hernandez,
P. (in press). Using Direct Behavior Rating (DBR) to screen
for school social risk: A preliminary comparison of methods
in a kindergarten sample. Assessment for Effective
Intervention.
Abstract - In this study, preliminary evidence of
the potential for Direct Behavior Rating (DBR) in screening
assessment of school social behavior was provided through
evaluation of the concurrent validity of DBR with a
commonly used criterion measure. The teacher-completed form
of the Social Skills Rating System (SSRS; Gresham &
Elliott, 1990) was selected as the criterion measure given
similarity to DBR in both broad construct of interest and
intended use. Classroom teachers completed DBRs of Academic
Engagement and Disruptive Behavior twice daily, along with
Fall and Spring completion of the SSRS. Overall results
supported a moderate to strong association between the
methods, with stronger correlations resulting in the Fall
assessment period. In addition, evidence of teacher
acceptability for DBR is provided. Along with specific
recommendations for future research, implications are
discussed regarding selection and interpretation of
behavior assessment methods for screening purposes.
Other Categories: N/A
|
|
