NEW RESEARCH FINDINGS

A scale for rating conversational impairment

de Villiers J, Fine J, Ginsberg G, Vaccarella L, Szatmari P
Journal of Autism and Developmental Disorders
Published online November 2, 2006

This paper presents a scale for assessing conversation skills in individuals with Autism Spectrum Disorder (ASD).  Based on semi-structured conversations of 47 youths with high-functioning autism and Asperger Syndrome, the paper describes 5 areas of conversational difficulty for speakers with ASD: atypical intonation; semantic drift; terseness; pedantic speech; and perseveration.  The scale may be a useful initial tool for assessing the pragmatics of conversation in ASD and for evaluating change in conversation skills over time.

Abstract  

There are few well-standardized measures of conversational breakdown in Autism Spectrum Disorders (ASD).  The study’s objective was to develop a scale for measuring pragmatic impairments in conversations of individuals with ASD.  We analyzed 46 semi-structured conversations of children and adolescents with high-functioning ASD using a functional linguistic paradigm.  Five constructs were developed that assessed difficulties related to the pragmatics of conversation: atypical intonation; semantic drift; terseness; pedantic speech; perseveration.  The scale shows good inter-rater reliability and variation in the scales is not simply a reflection of IQ or language competence.  This tool represents a way of characterizing language use in ASD and is an initial step towards developing a tool to evaluate change in degree of social impairments in conversation.

Introduction

There are few reliable, validated rating scales to measure conversational impairments in Autism Spectrum Disorders (ASD).  Some scales assess pragmatic difficulties qualitatively (e.g., Ghaziuddin & Gerstein, 1996; Bishop, 1998), but do not look at ASD specifically nor incorporate the range of aspects of conversational impairment that appear possible.  Ghaziuddin and Gerstein’s (1996) scale looks at the presence of pedantic speech in Asperger Syndrome.  It is mainly derived from clinical observations and from concepts in the field of conversation analysis (e.g., Grice’s maxims, Grice, 1975). Bishop’s Children’s Communication Checklist (CCC) (Bishop, 1998) assesses aspects of impairment in social communication, incorporating items concerned with social relations and restricted interests.  However, the CCC is a parent–teacher report measure, lacks a clinician report component and is not specific to ASD.  Tools that are specific to ASD are needed to evaluate the degree of impairment and to measure change in the context of natural history and treatment.  In this paper, we use a descriptive linguistic technique to develop a list of measures specific to ASD for assessing conversational difficulties.  The paper presents a simple to use and reliable measure of linguistic communication.  We look at both high-functioning autism and Asperger Syndrome to allow for sufficient coverage and variation, we assess inter-rater reliability, and we estimate correlations with standard measures of IQ and language skills.

Methods

Participants:

There were 46 participants in total.  All had fluent language abilities.  Thirty-eight participants came from a follow-up study of children with high-functioning autism and Asperger Syndrome and eight were added to this sample to increase sample size for the reliability analysis.  The group of 46 consisted of children: age range 124–164 months, mean 142 months, SD = 10.7; Leiter IQ (Levine, 1986) range 48–123, mean 82, SD = 20.4.  Thirty-two had a diagnosis of autism, fourteen had a diagnosis of Asperger Syndrome.  The age and IQs of the two groups did not differ significantly.

To ensure a basal level of language ability (i.e., being verbal and spontaneously speaking in full sentences) so that conversation was possible, the autism sample had to have a McCarthy oral vocabulary (McCarthy, 1972) score greater than 5.  Even so, the Asperger syndrome participants had higher scores (m = 25.67, SD = 2.83) than the autism (m = 16.62, SD = 10.30) participants on the test of language development (TOLD) grammatical completion sub-test (t = −2.57, P = .02) as well as on the grammatical understanding subtest (Asperger: m = 23.44, SD = 2.24; autism: m = 20.14, SD = 3.92) (t = 2.36, P = .03).  A full description of the characteristics of the participants at inception and of the diagnostic differentiation is given in Szatmari et al. (2000).

Psychometric Tests:

The participants underwent a battery of psychometric tests, which included:

Data Collection and Transcription Procedures:

Children diagnosed with ASD were audio taped having semi-structured conversations with a research technician for 10 min.  The same research technician conversed with all the participants and was blind to the purposes and objectives of the study.  Conversations took place in the child’s home and covered topics of school, hobbies, and family life.

There were several standard questions about what the child had for dinner and what television programs the child liked to watch.  The recording was transcribed in its entirety using standard orthography and the format and conventions were consistent with the codes for the human analysis of transcripts (CHAT) transcription and coding format of the child language data exchange system (CHILDES; MacWhinney, 1995).  A second transcriber checked the transcript.  A coding scheme was developed to measure types of conversational breakdown based on an initial analysis of 23 recorded conversations using a broad systemic-functional linguistic framework (Gregory, 1985; Halliday, 1994) and delineating recurrent linguistic patterns and differentiating features in the data.  There was continuous interplay between examining the transcripts, listening to the recorded conversations and linguistic analysis of the language.  Two independent raters scored features independently and blindly on a 0, 1, 2 scale (0 = did not occur; 1 = occurred sometimes; 2 = occurred frequently or very salient).  The two raters were different from those who developed the coding scheme.

Rating Scale

Scale of Language Features:

The preliminary analysis yielded a scale of nine features characterizing the language of children with ASD.  (Click here to view examples.)

Formal Intonation
This feature is recognized when speech has a flat, monotone or “wooden” quality.

Topic Switching
“Topic switching” includes tangential language and shifting topic abruptly.  It is often characterized by making “out of the blue” comments.

Terse
Terseness involves minimal responsiveness.  Speakers initiate rarely and respond with concise, short answers.

Pedantic Speech
With “pedantic speech”, speakers typically offer more factual, accurate, specific or technical information or more detail than the conversation demands.  The language also has a stereotypic quality, sounding imitative or rehearsed.  There may also be unnecessary repetition or self-corrections.

Perseveration
“Perseveration” involves excessive persistence on a chosen topic without turning attention to new topics or situations.

Pausing
“Pausing” is recognized when there is either a delayed response or a failure to respond to a conversational partner, or where an individual needs prodding to respond to questions.

Disengagement from Verbal Context
“Disengagement from verbal context” includes banging, singing or self-stimulating noises such as blowing.

Attention to Outside Environment
A speaker focuses attention on elements of the environment outside the conversation. This may include fixating on something extraneous or having to be reminded to stay with the conversation.

Atypical Stress Selection
“Atypical stress selection” is phonological stress (i.e., intonation and inflection) without contextual support for that stress.

Nine features were first defined, and five sub-scales were derived from these to simplify the framework within a common conceptual frame.  In creating these five sub-scales, the nine features were reanalyzed conceptually and in terms of the distribution of their scores in the conversation (by combining those with greater frequency).  To collapse the nine scales to 5, atypical intonation and atypical stress selection were combined into one (“atypical intonation”). Terse and Pausing were also combined into one (“terse”), as features associated with receptive interactivity and lack of ongoing availability in the conversation.

The category of “semantic drift” combined three features which all relate to the creation of flowing, coherent talk: (1) Disengagement from verbal context includes banging, singing, self-stimulating noises, and laughing; (2) Attention to outside environment diverts the conversation from its major semantic focus; (3) Topic switching indicates a change of semantic material without the usual conversational cues.  Together these three scales indicate that meaning is not being developed coherently and that the conversation is drifting from one kind of meaning to another.

Perseveration and Pedantic speech remained the same. A score of 0, 1 or 2 was still assigned for the pedantic scale as well as the perseveration scale.  The atypical intonation scale and the terse scale were assigned a score ranging from 0 to 4, and the semantic drift scale was assigned a score that ranged from 0 to 6.

Results

Inter-Rater Reliability:

All the conversations were rated independently on the nine dimensions by two raters on a three-point scale (0, 1, 2).  Kappas were computed to estimate inter-rater reliability.  The reliabilities are at least moderately high on most scales (Table 1).  “Perseveration”, although reliable, was scored positively in only four children.

Table 1: Inter-Rater Reliability for Rubscales (0, 1, 2)

Scoring: 0 = did not occur; 1 = occurred sometimes; 2 = occurred frequently or very salient

Correlations among the Five Subscales:

The next analyses focus on the reduced set of five scales.  There were weak but significant correlations between “atypical intonation” and “terseness” (r = .35, P < .02), as well as between “pedantic speech” and “perseveration” (r = .36, P < .01).  Generally speaking however, there was little or no correlation between the five scales.  The language scale “terse” was also significantly but marginally correlated with the Leiter IQ (r = −0.33, P = .04).  The higher the IQ, the lower the score was on this scale.  None of the five sub-scales was correlated with variation in age in this sample.

Regression Analysis – Relationship between the Five Conversation Scales, Leiter IQ and Language Measures:

A regression analysis was then performed using each of the five scales as the dependent variable and Leiter IQ and language measures (McCarthy oral vocabulary, TOLD grammatical understanding, and TOLD grammatical completion) as independent variables.  The model was significant for just 2 of the scales (semantic drift and terseness).  The amount of variance explained was 39% for semantic drift, but there was no independent contribution from any variable.  The only variable to be independently associated with terseness was grammatical completion (See Table 2).

Table 2: Regression Analysis – Relationship between the Five Conversational Scales, Leiter IQ, and Language Measures

Language measure: McCarthy oral vocabulary, TOLD grammatical understanding, grammatical completion.

Discussion

The study applied a descriptive linguistic approach to the conversations of individuals with ASD.  We produced a set of scales for assessing difficulties in communication for speakers with ASD that are relatively independent of age, IQ, and language abilities.  The study also differentiated five sub-scales, which may represent a more useful version of the scale.  In fact, given the association between terseness, IQ and language, we would recommend eliminating this sub-scale, which may reflect more general cognitive and language impairment, and retaining the other four.

These sub-scales are more explicit about the nature of communication problems that signal difficulties in social reciprocity.  The study identifies some of the linguistic bases for clinical judgments about communication impairments in ASD.  As such, the scale represents an initial step in understanding difficulties related to the pragmatics of conversation, particularly as it benefits the evaluation of change over time.  The scale adds to the repertoire of existing scales related to pragmatic and functional aspects of conversation such as Ghaziuddin and Gerstein’s (1996) scale and Bishop’s (1998) CCC.

The strengths of our scale are that it was based on a very careful, “microanalytic” approach, was rated from the data itself and, finally, appears to have good inter-rater reliability.  The results of the study suggest that some variable, other than IQ and language, determines conversational breakdown in children with ASD.

It is generally assumed that conversation relates to theory of mind (Baron-Cohen, Tager-Flusberg, & Cohen, 1993; Hadwin, Baron-Cohen, Howlin, & Hill, 1997).  While this is a reasonable assumption, this hypothesis has never been formally tested.  It is also possible that certain impairments in conversation are related to weak central coherence, a featural, as opposed to more global, processing style (Frith, 1989).  Further work needs to be done on this hypothesis as well.

The study had several limitations.  This is a preliminary scale with a relatively small sample size and without a control group.  However, this scale is to be used to measure variation within an ASD population not as a diagnostic measure to differentiate ASD children from controls.  A standardized script was not used.  The context of a semi-structured conversation, though parallel to a clinical interview, is more restricted and artificial than a casual social setting.  A less formal task might yield different conversational styles.  It is, in addition, difficult to generalize about children’s “typical” communication using 10-min samples with the same adult. “Literalness”, which is rare, for instance, was omitted but might be useful (cf. Adams & Bishop, 1989).

In the future, these preliminary patterns of pragmatic impairment could be related to underlying mechanisms by correlating data from neuropsychological tests and cognitive measures to see which explanatory construct provides a more complete account of variation in the linguistic patterns when compared with controls.  In addition, the scale could be used in intervention studies to potentially measure change in conversation skills.

Author Note

Jessica de Villiers, Department of English, University of British Columbia, Vancouver, BC, Canada. Jonathan Fine, Department of English, Bar-Ilan University, Ramat-Gan, Israel.  Gary Ginsberg, School of Public Health and Ministry of Health, Hebrew University, Jerusalem, Israel.  Liezanne Vaccarella and Peter Szatmari, Department of Psychiatry and Behavioural Neurosciences, McMaster University, and McMaster Children’s Hospital, Hamilton, ON, Canada.

Acknowledgments  

This research was supported by grants from the Ontario Mental Health Foundation and by a Social Sciences and Humanities Research Council grant to Jessica de Villiers. We want to thank the children and families who participated in this project.