Exploring Perceptions of Second Language Speech Fluency Through Developing and Piloting a Rating Scale for a Paired Conversational Task

Much research has explored how perceptions of speech fluency are influenced by a variety of temporal speech features (e.g. speech rate). However, less is known about the influence of non-temporal and conversational speech characteristics, as well as listener characteristics such as accent familiarity and conversational style, on fluency perceptions. To address this gap, the present study explored the influence of these characteristics through developing and piloting a fluency rating scale for a paired conversational task for assessment for learning purposes. A two-phase mixed-methods sequential exploratory design (Creswell, 2009) provided the methodological framework for the present study. In the first phase, seven trained English for Academic Purposes (EAP) instructors watched videos of seven-minute conversations, elicited from 14 intermediate-to-advanced EAP learners, who performed the conversational task twice with two different partners. Afterwards, instructors were audio-recorded discussing their observations about learners' fluency. These recordings were coded using in-vivo and pattern coding techniques (Saldaña, 2009). Six themes were identified: smoothness, efficiency, sophistication, clarity, facilitating topics and turns, and supporting the conversation partner. These themes informed the development of a multi-item fluency rating scale, used in the second phase of the study. In this phase, a new group of 35 EAP instructors watched four seven-minute video-recorded conversations between eight learners, and then used the scale to rate the performances. Before watching each video, instructors reported their familiarity with students' accents on a six-point scale. Once rating was completed, instructors completed a conversational style questionnaire. The results were as follows. First, a Principal Component Analysis of scale items produced two separate components - individual fluency and conversational fluency. Second, temporal measures of within-clause pause rate correlated significantly with items representing individual fluency whereas measures of filled-pause rate correlated significantly with items representing conversational fluency. Third, non-parametric analyses of group differences showed that accent familiarity moderately affected fluency assessments. Finally, correlational analyses revealed significant correlations between fluency assessments and questionnaire items representing certain conversational style characteristics of the listeners. The overall findings showed that speech and listener characteristics affected fluency perceptions to varying degrees, providing implications for the assessment of 'higher-order fluency' (Lennon, 2000).