Learning Differences, Cognitive Diversity and Examinations in Higher Education: bringing disability services and faculty together

In Part 1 of this article, published in AHEAD Journal No 3, the authors covered the cognitive demands of exams and the increase in diversity in higher education. In Part 2 the authors look at how to develop practice based on this knowledge.

Part 2: Putting research to practice: service providers and faculty have the power to change exam design

Creating exams that are accessible to a wide population of learners can be challenging given the rigorous teaching and research demands placed on faculty. In hopes of helping and encouraging faculty members and service providers alike, to embrace creating accessible exams, we pull from and add to recent research related to Universal Design and offer a three-pronged framework which focuses on:

• exam presentation
• exam output
• exam content

Specifically, we offer disability service providers recommendations to ensure that exam presentation and exam output are designed in a manner that maximizes accessibility and ensures that exams are designed in a manner that allows for a more valid assessment of a student’s mastery and knowledge. Next, we offer faculty recommendations for designing exams. We conclude with a proposed Faculty Training Series, which disability service providers can offer faculty as a means of encouraging and fostering exam design that is useable for our increasingly diverse postsecondary student populations.

Redesigning exam presentation

Simply put, exam presentation is how exams appear or, better said, how students can take in exam information. Rose and Meyer (2002) suggest ‘flexibility in presentation’. Because most exam material is absorbed and processed through visual and/or auditory channels, suggestions are provided to help service providers maximize exam accessibility through effective visual design and auditory design.

Exam presentation and visual design

In this era of ever evolving technology, which is empowering all individuals, service providers would do well to consider ways to offer electronic versions of paper exams to students via computer. Computer-based exams allow service providers to offer exams that can accommodate multiple representations of exam content (Thurlow, Lazarus, Albus, & Hodgson, 2010). Examples of visual design consideration are listed in Table 1 and suggestions are noted below:

• Create exams with digital text, which allows for the altering of font type and font size; the same is true for digital images (Rose & Meyer, 2002). In fact, larger font size (size 14 versus size 12) benefits students with and without disabilities (Fuchs, et al., 2000). While font is routinely adjusted in everyday school, work, and personal tasks, rarely is it offered as a tool to increase access and engagement on exams.
• Particularly for students with low-incidence visual impairments such as blindness, service providers need the technology and equipment to turn text into Braille (Laitusis & Attali, 2011) and tactile graphics. Even in this era of text to speech, there are many Braille users and individuals with low-vision who rely not only Braille, but textured tactile graphics. More research is needed on the use of tactile graphics by individuals with learning differences, however with emerging technologies, more and more individuals may benefit from the use of tactile graphics as a learning tool as well as a way to be access exam content.
• Visual content should be designed so that it is simple and not cluttered. Examples include avoiding using Roman numerals, which can be difficult to visually discriminate, allowing appropriate spacing between questions, and placing keys and legends directly under the text where they are to be applied (Gaster & Clark 1995; Anderson-Inman & Horney, 2007). Perception, engagement, and attention span can be improved by allowing adaptable foreground and background colors.
• The National Center for Supported Electronic Text (NCSeT) has a list of “Typology Resources” (Anderson-Inman & Horney, 2007). Amongst others, they suggest “notational” resources. In other words, if a computer-based exam is being offered, then the student should be able to make notations such as underling, highlighting, or writing notes.

Exam Presentation and Visual Design

• Font Size
• Spacing
• Organization - Chart with key underneath
• Braille
• Highlighting

Table 1: Exam presentation and visual design

As a cautionary note, we cannot assume that all computerbased assessments are universally designed. For example, older students may be less familiar with computers therefore, taking computer-based exams potentially leads to more, rather than fewer barriers for this group of students (Thompson, et al., 2002).

To this end, there can be limitations in the accessibility of certain software and hardware. The above further underscores the need for ‘flexibility in presentation’, such that computer-based exams should also be able to be presented in paper form. Moreover, older students may benefit from workshops or training in computer usage.

Exam presentation and auditory design

Similar to visual design, new technologies are improving the auditory design of exams. Several examples are provided:

• An accommodation for some students, such as students with learning disabilities in reading or visual impairments, is access to a reader (Sireci & Pitoniak, 2007). It can be costly and time consuming to have a human reader. Recent advances in text-tospeech software and screen readers can be more cost-effective and support student independence. Also, there are software programs that read text aloud and simultaneously highlight the image of text as it is being read.
• Access to text-to-speech software can also be helpful with editing writing samples. For example, poor readers may have difficulty reviewing what they have written in order to make corrections. Similarly, students with limited attention may benefit from hearing and seeing their writing as a tool for editing. An empirical analysis by Garrison (2009) indicates that despite some other limitations, text-to-speech software can facilitate proofreading.

Since most technology uses speech synthesis for either text to speech or speech to text, it is helpful to be cognizant of how to best use synthesized speech when transforming exams. In the postsecondary setting, this would most often occur when a faculty member decides to allow students to take an exam via a computer with speech output. Research from the RNIB Centre for Accessible Instruction, (Cryer & Home 2008) found that the subjective acceptance of synthetic speech may depend on the users’ experience, as people were found to ‘get used to’ synthetic voices.

The Centre’s research also found that synthetic speech may be less intelligible than natural speech, particularly with background noise, and may need to be presented more slowly to be fully understood. However, measures of reading performance with synthetic speech improve with experience. Finally, some users of synthetic speech prefer less expressive synthetic voices as they felt it helped them to focus on the content of the text. This is important as it tells us an exam may not be the best situation to try synthetic speech for the first time. Furthermore, headsets are warranted when exams are used with speech synthesis to eliminate background noise.

Exam Presentation and Auditory Design

• Speed - The pace at which auditory information is presented to the student
• Voice - Male or female, dialect, accent
• Volume - Level of sound

Table 2. Exam presentation and auditory design

Exam Output

Exam output refers to how students demonstrate their knowledge in an exam (e.g., handwriting, typing, drawing, or speaking). Exam output is particularly critical because it is what faculty members use to grade and evaluate students. Students should be offered alternate means for demonstrating concept mastery on exams. Examples and considerations are offered:

• Access to a computer for writing limits visual motor integration or graphomotor barriers. Moreover, when working on a computer, students can more easily edit (e.g., cut, copy, and paste) their work versus having to erase and re-write.
• Speech-to-text (dictation) software allows students to dictate responses, limiting graphomotor and visual barriers. Dictation software should be implemented with care. Dictation software includes a learning curve. Advanced planning and practice is needed to ensure that the student has access to a dictation program that has been trained to process his or her voice.
• Text-to-speech (dictation) software is a valuable proofreading tool for many students who benefit from the chance to listen to their writing and catch mistakes, in the same manner that some individuals read aloud their own writing in order to self-monitor their prose.
• Allow students to document answers directly on the exam booklet. Many times exams have separate components: an exam booklet and a response sheet such as a Scantron. Separate components are inherently biased for students with poor visual motor integration (Thompson, et al., 2002). Moreover, students with attention weaknesses may lose their attention set while transferring answers from the exam booklet to the Scantron or other response form.

Recommendations for faculty members

One role of faculty is to facilitate the acquisition of knowledge and skills within a given discipline. The reciprocal role of students is to prove mastery of that knowledge, typically by taking an exam. As the creators and authors of exams, faculty have the power to design exams that accurately measure student knowledge without bias. Several suggestions are provided to facilitate faculty in designing exam content that is sensitive to cognitive diversity within student populations.

Exam content

Exam content refers to the meat or heart of the assessment; it consists of the course concepts that faculty want their students to master. According to Thompson, et al. 2002

An important function of well-designed assessments is that they measure what they actually intend to measure

Accordingly, faculty should create exams that accurately assess course goals and objectives (Ofiesh, et al. 2006). The following factors should be taken into consideration to ensure that exam content is accessible to as many adult students as possible:

• Avoid using irrelevant graphs or pictures.
• Verbal content should be clear, concise, and specific. Questions should be easy to understand regardless of the student’s experience, knowledge or language skills, or current concentration levels. (Thompson & Thurlow, 2002).
• Advanced and technical vocabulary should be used only when it is part of the content to be measured, not as an exercise in verbosity.
• Directions and questions need to be in simple, clear, and understandable language. “Compound, complex sentences should be broken down into several short sentences, stating the most important ideas first… All noun-pronoun relationships should be made clear… When presenting instructions, sequences steps in the exact order of occurrence.” (Thompson & Thurlow, 2002; Gaster and Clark, 1995).
• Exam content should not be biased based on a student’s socioeconomic status or experience outside of school (Thompson, et al., 2002)
• Very carefully consider the role of timing in an exam. Is the test one of speed or power? It is rare that tests of knowledge require exams to be so tightly timed that all students in a class can’t finish the exam. Most exams should test knowledge or skills, not speed.

Gaster and Clark Eight Readability Guidelines (1995)

1. Use simple, clear, commonly used words, eliminating unnecessary words.
2. When technical terms must be used, they should be clearly defined.
3. Compound, complex sentences should be broken down into several short sentences, stating the most important ideas first.
4. Introduce one idea, facts or process at a time; then develop the ideas logically.
5. All noun-pronoun relationships should be made clear.
6. When time and setting are important to the sentences, place them at the beginning of the sentence.
7. When presenting instructions, sequences steps in the exact order of occurrence.
8. If processes are being described, they should be simply illustrated, labeled, and placed close to the text they support.

Table 3. Gaster and Clark - Eight readability guidelines

Faculty training series: a tool for disability services providers and faculty

Faculty and disability service providers have a common goal: educating students. However, each brings different and equally valuable expertise and knowledge to a university. On one hand, a postsecondary faculty member is a master of her subject. She has thorough training in her content area, and, in some cases, less direct training in pedagogy and teaching. On the other hand, a disability service provider has a background in disabilities, including how disabilities can impact learning and assessment of knowledge.

Both faculty and disability service providers are charged with the responsibility of not only offering accessible exams for all students, but also ensuring that exam design is an accurate and valid estimate of a student’s true mastery. Furthermore, valid exam results from student exams can better inform a professor’s potential need to modify/improve subsequent instruction.

Disability service providers have a wealth of information in terms of disabilities, functional limitations associated with disabilities, and necessary academic accommodations or supports. Dispersing or sharing this information one faculty member at a time during a phone conversation or over a chain of emails can be challenging. Disability service providers must then decide how best to share their knowledge with faculty members. Murray, Lombardi, and Wren conducted a survey on the effect of disability-focused training on university staff members, and their findings are encouraging.

Murray, Lombardi, and Wren’s study included two key findings. Firstly, university staff that received ‘disability-focused’ training experiences in the past report more positive attitudes towards students with learning disabilities.

Secondly, 112 survey participants that had not received prior learning disability training expressed interest in receiving learning disability training and felt that they needed more knowledge in regard to how to support students with learning disabilities (Murray, Lombardi, & Wren, 2011).

The timing for a revised notion of how we assess knowledge in postsecondary settings and inform faculty about our roles is critical. Despite years of research on extended time for example, stigma still exists and faculty remain unclear about why so many students are provided with the more time on exams (Trachtenberg, 2016). The main problem with this question is that it comes from an erroneous frame of reference that imparts the idea that a tightly timed exams is needed to measure knowledge. As disability service providers our roles now require educating many individuals on campuses so that students do not experience fear and stigma in order to demonstrate their knowledge and perform at par with their intelligence. Given the inherent interest and need, we offer a model training series (see Tables 4 and 5).

Faculty Training Series: Designing Accessible Exams Parts 1-3

Part 1 Introduction to Disability Resource Center

Introduction: Universities can be worlds unto themselves with a breadth of programs and services. Introduce faculty and other staff to the disability resource center, including its staff, supports, and services. Additionally, many faculty and staff would benefit from a basic overview of the variations in learning processes among different types of learners so that they understand why they make accommodations.

Part 2 Accommodations

Accommodations: Briefly review regulations that mandate the delivery of academic accommodations to students who have a disability that obfuscates learning. Reinforce how accommodations, such as extended time or access to a keyboard, serve to level the playing field and provide equal access; they do not offer an advantage or leg-up.

Allocation of Accommodations: Given a lack of knowledge, some may question the process behind determining who receives what accommodations. Review procedures for determining reasonable accommodations.

Part 3 Accessible Exam Design versus Accommodations

Accessible Exam Design: Accessible exam design can remove the need for many accommodations. Combat misperceptions, especially the notion that accommodations and universally designed exams lower standards. The article on Promoting Thoughtful Assessment by Ofiesh, Rojas & Ward, (2006) is a good resource for faculty.

Two examples of myths that need to be dispelled:

Everyone would perform better with more exam time.

Universally designed assessments are designed so that all students pass the exam.

Table 4. Faculty training series: designing accessible exams parts 1 – 3

Faculty Training Series: Designing Accessible Exams Parts 4 – 6

Part 4 Promote “Thoughtful Assessment”. Help faculty to understand the relationship between the goals and objectives of the course and, how that translates into exam content.

*Inquire: Given a hypothetical Science-based syllabus, ask participants what they think students should be able to do by the end of the course?

Nurture Insight: Now ask what would a student need to show you in order to be able to demonstrate this competence to you? What are the goals and objectives in YOUR class? What would a student need to do to demonstrate competence in those areas?

Application: Given a hypothetical exam from the Science course noted above, ask participants if the exam is a good measure in terms of content and based on the syllabus and what they noted a student needs to show to demonstrate competence.

Ask participants: Do YOUR exams match YOUR goals and objectives?

Part 5 Application - Review, revise, and redesign exam content with a hypothetical example.

Review: Provide samples of a traditional exam.

*Revise: Revise the exam. Provide a simple, uncluttered, and organized handout with examples and solutions for improvement. Allow participants to these ideas to foster redesign.

Redesign: Redesign exams to include accessible format and content. The challenge is to put research to practice and design exams that are accessible to the diverse population of postsecondary students.

Part 6 Application- Review, revise, and redesign exam content with a personally created exam.

*Review: Ask faculty participants to bring a copy of an exam to share and revise.

Revise: Revise non-accessible exam content. Share examples with others in a similar academic domain for input and more ideas.

Redesign: Redesign exams to include accessible content. Put research to practice and ask participants to consider what else can be done to redesign their own exams that are accessible to the diverse population of postsecondary students.

*Divide participants into small groups for feedback and sharing.

Table 5. Faculty training series: designing accessible exams parts 4 - 6

There are always cases which engender more questions than answers. In these cases, ongoing collaboration between faculty and disability service providers is needed to ensure that individual needs are met.

Ofiesh et al. (2006) use the term ‘thoughtful assessment’ to refer to assessments that serve both faculty and students in postsecondary settings. Specifically, thoughtfully designed exams measure intended content, allowing faculty to evaluate their teaching, and create exams that are accessible to a variety of learners.

A thoughtful, universally designed assessment consists of a multitude of considerations, including, but not limited to, subject content, electronic flexibility, English language usage, format options (e.g., essay, short-answer), time limits, text characteristics, a direct link from the goals and objectives of the course, instruction, and informational delivery system, and more. (Ofiesh et al., 2006).

Researchers at the National Center for Education Outcomes created a list of key elements of assessments that maximize access to a wide range of learners with varying cognitive characteristics (Thompson, Johnstone, & Thurlow, 2002). These elements are:

1. inclusive assessment population e.g. exam design takes into account all types of learners: those who need large font due to aging, users of Braille, individuals with migraines who may want to adjust font and background colors on computer screens etc.
2. precisely defined constructs e.g., what the exam is designed to measure in terms of content, skills, knowledge base, and what one is required to be able to do to take the exam are clearly laid out.
3. accessible, non-biased items amenable to accommodations e.g. words with double meanings or that are more readily understood by males or females are eliminated.
4. simple, clear, and intuitive instructions and procedures e.g. understanding how to take the exam should not be part of what is being measured.
5. maximum readability e.g. large font, adjustable foreground and background colors, speech output options etc.
6. maximum legibility e.g. options for use of speech recognition systems, scribe, adjustable font size, different paper options when needed.

What the future holds

Fortunately, research regarding how the brain, learning, and engagement and innovations in the field of computer technology are growing in parallel. Future studies are likely to inform and improve the creation and application of accessible exams, also allowing for greater exam validity. Research tells us that the act of taking an exam or, more specifically, the act of retrieving previously learned information, promotes learning Pastotter, Schicker, Niedernhuber, Bauml (2011). We can’t deny exams serve a useful purpose. While the continued development and implementation of accessible exams rests on faculty members and disability service providers, it is equally important that administration provide financial and systemic support (Rose and Meyer, 2002). More than ever, in order to prepare young adults for their lifespan, it will be important to come together across levels of education and within systems of higher learning in order to rethink how we design exams. We have the tools and knowledge to design exams that are flexible, and to simultaneously remain committed to high standards and the goals and objectives of programs of study. Universally designed exams mean making the world a better place by allowing more people to take part in education. In doing so, we will continue to create a world where a greater portion of humanity can participate fully in society through the use of their strengths and talents. Education and training is the catalyst for each of us to fulfill our potential. Disability service providers can help to make the case for change: intellectually, administratively, and economically. This would seem to be a noble goal given all the promise that higher education can hold for so many individuals.