Using brain computing to maintain access to language with i…

Using brain computing to maintain access to language with individuals affected by Amyotrophic Lateral Sclerosis (ALS) by Andreas Forsland and John Standal 2023 October / November Closing The Gap Solutions Volume 42 #4

? instruction, literacy & inclusion

Using brain computing to maintain access to language with individuals affected by Amyotrophic Lateral Sclerosis (ALS) Summary: Overall, the combination of non-invasive Brain-Computer Interface (BCI) technology and the occipital lobe's ability to pro- cess visual information can provide a powerful means of communication for patients with severe motor disabilities, such as ALS. BCI has the potential to revolutionize the way we support these patients, allowing them to maximize their interaction with the world around them even as the disease progresses and other accessibility tools, such as eye tracking, fail. Although BCI tech- nology can be particularly beneficial for patients in the later stages of ALS, it has applications in many speech-generating areas of interest which Cognixion is aggressively exploring. Cognixion is excited to become the first SGD manufacturer to produce an FDA-approved BCI device for patients with neuromotor impairments.

cle movement. As the disease progresses with the individual, the motor neurons in the brain and spinal cord begin to degen- erate, which ultimately leads to the loss of muscle control and function. This neural degeneration can result in difficulties with speaking, swallowing, and breathing, and loss of mobility and strength in the limbs. ALS tends to affect adults between ages 40-75 but can strike at any age with an average age of 55 based upon data from the ALS Association. As many as 600,000 diagnoses of ALS per year worldwide (~30,000 in the US). ALS is a fatal diagnosis with an average life expectancy of 3-5 years after diagnosis. Currently, there is no cure for ALS. Although ALS is devastating to the func- tion of nerves and muscles it does not affect the five senses. In most cases, cognitive functions-solving, and decision-making remain intact.

Amyotrophic Lateral Sclerosis (ALS) is a progressive and de- generative disease that affects the nerve cells responsible for controlling voluntary muscle movement. As ALS progresses the motor neurons in the brain and spinal cord begin to de- generate, which ultimately leads to the loss of muscle control and function. This neural degeneration can result in difficulties with speaking, swallowing, and breathing. Cognixion has tak- en on the challenge to provide language and environmental access for late-Stage ALS patients utilizing EEG to control their speech-generating device. FIRST AND FOREMOST, “WHAT IS ALS?” Amyotrophic Lateral Sclerosis (ALS), also known as Lou Geh- rig's disease, is a progressive and degenerative disease that af- fects the nerve cells responsible for controlling voluntary mus-

ANDREAS FORSLAND, CEO, Cognixion. Andreas has extensive experience as a former product, customer experience, and brand leader for Phillips, Citrix, IBM, and Progressive Insurance. Andreas started Cognixion when his mother was in the ICU and was unable to communicate effectively with current technology.

JOHN STANDAL, MS/CCC-SLP, ATP, Vice President of Clinical Affairs, Cognixion. John has been working directly with people with communication and learning impairments since 1993. John has worked with Words+, Viking Software, Tobii, n2y and Specially Designed Education Services in the past.

BACK TO CONTENTS

ALS affects slightly more men than women, with a male-to- female ratio of about 1.5:1. The average life expectancy for peo- ple with ALS is two to five years from the time of diagnosis. How- ever, about 10% of people with ALS live for 10 years or more, and some people with the disease can live for several decades. The incidence of ALS is highest in the United States and Europe, and it is rare in other parts of the world. The reason for this geo- graphic variation is not fully understood. There is no known cause of ALS, but research has identified several potential risk factors, including genetics, environmental factors, and lifestyle factors such as smoking and physical activ- ity. STAGES OF ALS How does ALS present itself to individuals or Person’s with ALS (pALS). While the end point is the same for all people with ALS, the disease progression varies among all those diagnosed. The muscles affected first are different from person to person. Some individuals progress quickly through the disease process while others may plateau at times, or even revert for brief peri- ods before resuming the ultimate decline. Because there is no single test that can definitely diagnose ALS, a healthcare provider will conduct a physical exam and review your full medical history. A neurologic examination will test the patient's reflexes, muscle strength, and other responses and will be held at regular intervals to assess whether symptoms such as muscle weakness, muscle wasting, and spasticity are progressively getting worse. Here are current tests physicians might perform to help di- agnose and or rule out ALS from the National Institue of Health (NIH ALS Diagnosis). These usually involve muscle and imaging tests to rule out other diseases and confirm the diagnosis of ALS: • Electromyography (EMG) is a recording technique that de- tects electrical activity of muscle fibers and can help diag- nose ALS. • A nerve conduction study (NCS) measures the electrical ac- tivity of your nerves and muscles by assessing the nerve's ability to send a signal along the nerve or to the muscle. • Magnetic resonance imaging (MRI) is a noninvasive proce- dure that uses a magnetic field and radio waves to produce detailed images of the brain and spinal cord. • Blood and urine tests may be performed based on your symptoms, test results, and findings from the examination by a doctor. A physician may order these tests to eliminate the possibility of other diseases. • A muscle biopsy may be performed if your doctor believes you may have a muscle disease other than ALS. Under local anesthesia, a small sample of muscle is removed and sent to the lab for analysis.

ness or twitching in one or more limbs. They may also notice dif- ficulty with fine motor movements, such as writing or buttoning a shirt. The symptoms may be mild and not interfere significant- ly with daily activities. Noted symptoms can include: • Muscle weakness, twitches, and cramping, followed by problems with balance, coordination, and gait. • Increasing effort to breathe, slurred speech, and some dif- ficulty chewing and swallowing. • Typically, no impact to behavior or cognition • This is where voice banking may begin but often patients do not fully engage in this process, incorrectly believing they have more time to complete this activity. • Touch, eye tracking or switch-based computer access may be prescribed. MIDDLE STAGE OF ALS - at this stage two to three regions of the body are now involved. The symptoms in this stage be- come more pronounced and begin to affect the patient's ability to perform daily tasks. The weakness spreads to other limbs and muscles, including – often early in the disease - those responsi- ble for speech, swallowing, and breathing. Patients may require assistive devices such as a wheelchair and may also require as- sistance with daily tasks such as grooming and feeding. Noted symptoms could include; • Muscles become increasingly paralyzed, atrophied, or per- manently constrained. • A wheelchair and/or respirator are typically prescribed, and constant care is needed for daily activities. • A small portion of ALS patients may spontaneously laugh or cry (pseudobulbar affect), and nearly 50% show some signs of Aphasia-like language impairments (An Evolving Understanding of ALS with Frontotemporal Degeneration). • Users will require eye gaze or complex switch access through the entirety of this phase for language/communi- cation or environmental access. • The end of this stage is typically marked by gastronomic intervention (a feeding tube) as secretion or bolus control becomes problematic. LATE STAGE OF ALS – The end of life. In the final stage of the disease, the patient is completely dependent on others for care. They may be unable to move, speak, or breathe on their own and may require mechanical ventilation. For the most part, the patient's cognitive function remains intact, but they are unable to act on this cognitive awareness. • The top causes of death for pALS are respiratory failure, pneumonia, and heart complications (in order). • Around 30% of patients will exhibit dementia and/or se- vere aphasia by this stage (An Evolving Understanding of ALS with Frontotemporal Degeneration). • The Trochlear nerve is often the last to be affected by ALS, so users can use eye gaze to generate communication

EARLY STAGE OF ALS – first region of involvement in the body is noted. In this stage, the patient may experience weak-

BACK TO CONTENTS

through eye-tracking devices as long as the ocular mus- cles stay intact. • As the Trochlear nerve becomes affected, because eyesight is still intact, the only solution that remains for patients to communicate is the use of a Brain Controlled Interface (BCI). • Typically, the end of life is 3-5 years after initial diagnosis, but those diagnosed younger than 50 generally exceed this timeframe, if the patient survives past 5 years PLS (Pri- mary Lateral Sclerosis) would be under consideration. The disease typically follows a predictable pattern of progres- sion, with symptoms worsening over time. Some patients may progress through the stages more quickly or slowly than others, and the order of symptoms may also vary. However, understand- ing the stages of ALS can help patients and their caregivers pre- pare for the changes that may occur and make the necessary adjustments to maintain the patient's quality of life. As ALS pro- gresses individuals move through the stages of the disease at varying rates. In general, the late stage is the shortest in longev- ity but that is not always a standard expectation. CURRENT SGD (SPEECH GENERATION DEVICES) ACCESS METHODS FOR PATIENTS WITH ALS The need for SGD technologies occurs early in the disease due to emergent speech disabilities. Initially, ALS patients may face intelligibility issues for their speech, and subsequently, they may have no functional speech and require speech-generating devices to communicate at all. Dysarthria occurs in more than 80% of ALS patients and is seen earlier in those with bulbar on- set who may become anarthric after a few months of disease on- set. Approximately 80% of ALS patients utilize some form of SGD device, and on average, rely upon it for 2-3 years. Most move from a direct select access (i.e., typing) to possibly a finger drag typing style and then usually end up using eye-tracking as their main access method. Eye tracking computer systems (ETCS) can allow cursor control by eye movement and represent a current standard of care. As stated previously, eye movements can be less fatiguing and at later stages of the disease can be the only remaining volitional movements that allow patients to commu- nicate. Currently, clients with ALS use the following access methods to their Speech Generating Devices (SGD); touchscreen, switch interfaces, eye tracking and/or head mouse, these access meth- ods have been used with limited success with many late-stage pALS. In isolation and at the appropriate stage, access can be successful, but as the disease progresses, the ability to transition from access method to access method is always needed. The largest void of AAC (Augmentative and Alternative Com- munication) access for people with ALS is at the late stage or closer to the end of life. At this stage, even eye tracking becomes difficult if not impossible due to the disease progression and/or

ocular comorbidities. In a study looking at ALS patients and ab- normal eye movements anywhere from 60-70% of ALS patients were reported to have ocular issues. (Eye Movement Abnormali- ties in Amyotrophic Lateral Sclerosis in a Tunisian Cohort) Because the eyes and forehead are bilaterally innervated, they are usually the last reliable access method for people with ALS. In short, bilateral innervation means that relatively equal distri- butions of right and left-brain hemisphere innervation govern the function of a specific facial part (such as the eye muscles). The 7 extraocular muscles, even though bilaterally innervated, are small and thus fatigue quickly causing the AAC user to have false positive selections when eye tracking. Research has shown that it is reasonable that with bulbar disabilities, eye movement abnormalities should be considered (Eye Movement Abnormal- ities in ALS) and may need to be addressed with disease pro- gression. SGD ACCESS METHODS AND PROS AND CONS RELATED TO ALS

Access Method

Pros

Cons

Stage of ALS

Used in early stage of ALS

Positioning of device for access and ability to isolate their fingers causing false positive selections. Slow access to language and access, higher incident of false positive strikes. Very slow access to language. Calibration issued and positioning of device, direct sunlight washing out cameras For most ALS pa- tients the ability to maintain good head movement and/or position- ing degrades early in the disease progression

Touchscreen

Immediate access to their language/ AAC device.

Switch interface Ability to access different areas of the body depend- ing on muscle control

Used in middle and late stage of ALS

Used in mid stage and late of ALS

Eye Tracking

Eyes remain intact until late stage of ALS but will eventually fail as an access point Allows for direct select access with gross movement of head move- ment

Head Mouse

Used in early to middle stage of ALS

WHAT IS A BRAIN CONTROLLED INTERFACE (BCI)? In short, a Brain-Computer Interface (BCI) is a system that enables communication and control between the brain and an external device, such as a computer, without requiring any phys- ical movement or action. BCI technology typically involves the use of sensors (electroencephalogram = EEG’s) to detect and in- terpret neural activity, which is then translated into commands

BACK TO CONTENTS

that can be used to control external devices. BCI’s can be exter- nal or internal to the brain and both options present their own strengths and limitations. INVASIVE V. NON-INVASIVE BCI The benefit of an invasive approach collects brain signals which have the benefit of providing high-quality (stronger) sig- nals but have the disadvantage of requiring surgical implanta- tion of a probe under the scalp with a permanent connection point outside the brain. While the resulting signals provide more accurate readings of brain signals, there is the high cost of the procedure and the risk of infection, as well as the possibility that scar tissues may form which can reduce the effectiveness of the probe. In a paper from the New England Journal of Medicine, “Neu- roprosthesis for Decoding Speech in a Paralyzed Person with Anarthria” ((https://www.nejm.org/doi/full), scientists were able to demonstrate positive results for language formulation with an invasive BCI procedure. The results they reported from their study included the ability of the patients to decode sentences via cortical activity in real-time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, they detected 98% of the attempts by the participant to produce individual words, and they were able to classify words with 47.1% accuracy using cortical signals that were sta- ble throughout the 81-week study period. Although these types of invasive clinical studies are starting to show positive results, the real-world applications of this technology are still limited to lab-based environments.

Currently, more companies and researchers are relying on non-invasive types of brain signal sensing (EEG). EEG is a non-in- vasive method for measurement and recording of the electrical activity of the brain with no surgical intervention. EEG is typi- cally performed using small sensors, called electrodes, that are placed against the scalp to receive signals from the brain. Elec- trodes might be used “dry” (without gel), or signal collection can be enhanced using a special paste or gel between the scalp and electrode. As shown below, the electrodes can be placed any- where on the head to allow researchers to monitor brain activity and utilize those outputs to view anything thing from affective states to motor functioning to mental fixation (and many more). Using this EEG data BCI technology can provide an alternative means of communication and control for patients with ALS, es- pecially late-stage. By using sensors to detect neural activity in the brain from the occipital lobe, BCI systems can interpret the patient's intentions and translate them into commands that can be used to control external devices, such as a speech-generating device (SGD), environmental controls, or robotic devices. The occipital lobe (see Figure 1) is the brain region respon- sible for processing visual information. In the context of BCIs (Brain Computer Interfaces) for patients with ALS, the patient is presented with visual stimuli on a screen or other display. These visual stimuli can be placed in conjunction with words or com- mands that the patient may wish to communicate.

Figure 1: The occipital lobe

In this illustration, shows an implantable EEG shows the placement of the electrocorticography electrode on the participant’s speech motor cortex and the head stages used to connect the electrode to the computer.

BACK TO CONTENTS

use SSVEPs to detect and interpret the patient's intentions; this access method will be discussed later in this article. ANATOMY OF BCI Overall, the anatomy of a BCI is complex and involves multiple components that need to work together seamlessly to provide a reliable and effective means of communication or control of an external device. BCI systems involve a combination of hard- ware and software components. The image below demonstrates the process and how these systems work together to identify a person’s visual intentions and how they are captured and then turned into a language output system. The anatomy of a BCI can be broken down into several key parts, including: • Brain signal acquisition: This component involves the use of sensors or electrodes to capture signals from the brain. There are several types of sensors that can be used for this purpose, including electroencephalography (EEG), magne- toencephalography (MEG), and functional magnetic reso- nance imaging (fMRI). • Signal processing: The signals captured by the sensors need to be processed and analyzed in order to extract meaningful information. Signal processing algorithms can be used to filter out noise, enhance signal quality, and ex- tract features that are relevant to the task at hand. • Feature extraction and selection: This component involves the identification of specific features or patterns in the brain signals that are relevant to the intended application of the BCI. For example, in a BCI designed for communi-

These visual signals create corresponding electrical signals in the patient’s occipital lobe "visual evoked potentials" (SSVEPs) which are then identified via EEG. SSVEP (Steady State Visual Evoked Potentials) stands for Steady-State Visually Evoked Po- tential, it is a type of brainwave activity that occurs in response to visual flickering stimuli at a specific frequency. SSVEP is a com- mon non-invasive method used in current neuroscience and cognitive research to study visual processing and attention. SSVEPs are electrical signals that are generated in the oc- cipital lobe in response to visual stimuli. By presenting the ALS patient with a sequence of letters or words on a screen and monitoring the SSVEPs that are generated in response, the BCI can determine which letter or word the patient is thinking of or mentally fixating on. With this technology, it is now possible to

Anatomy of a BCI (Brain Computer Interface)

BACK TO CONTENTS

cation, the relevant features might be associated with the intention to move a specific muscle or focus attention on a particular visual stimulus. • Classification: Classification is the technical term to de- scribe the way machines (i.e., computers) group things into meaningful categories. Once the relevant features have been identified, a classifier can be trained to recognize and interpret them. Machine learning algorithms are common- ly used for this purpose and can be trained to recognize specific patterns in the data that correspond to different intended actions or states. • Feedback and control: The final component of a BCI is the feedback and control mechanism. This involves providing feedback to the user based on their brain signals and using those signals to control a device or perform a specific ac- tion. The feedback can take the form of visual, auditory, or tactile cues, and the control mechanism can be anything from a robotic arm to a computer cursor, virtual keyboard or AAC (Augmentative and Alternative Communication) device. This complete BCI feedback system translates the patient's intentions into commands that can be used to select the de- sired letter, word, phrase, or access command on the screen. A brain-controlled interface can allow the patient to communicate without the need for physical movement or speech, which can be difficult if not impossible for patients with advanced ALS. BCI technology works when a person views a visual flashing stimulus, such as a flickering light, the brain generates electri- cal activity that can be detected using occipital lobe electroen- cephalography (EEG). In the case of SSVEP, the stimulus flickers at a fixed frequency, typically between 5 and 60 Hz. The brain's electrical activity synchronizes with the frequency of the stimu- lus, resulting in a characteristic pattern of brainwave oscillations at that frequency. These responses can be detected as distinct peaks or frequency components in the EEG signal. By analyzing the amplitude, phase, and other characteristics of these com- ponents pALS will then be able to activate language options or other Internet of things (IoT). Below is an example of a partici- pant from a Cognixion study being exposed to a stimulus at 7Hz and responding accurately via EEG.

Historically, SSVEPs have had a wide range of applications dedicated to laboratory settings. They are commonly used in neuroscience research to investigate visual perception, atten- tion, and cognitive processes. It has also found applications in brain-computer interfaces (BCIs), where users can control ex- ternal devices or interact with computer systems using their brain activity. The robust and reliable nature of SSVEP responses makes them useful for designing efficient and accurate BCIs. THE FUTURE IS NOW FOR SGD’S AND BCI Cognixion has taken on the challenge to provide patients with ALS (pALS) with access to language throughout the disease progression with pure BCI access. With many pALS, their access to their SGD via eye tracking, switch, or touch becomes difficult if not impossible due to the progression in the late stages. This is not due to the lack of technical success of their current technol- ogy but more of an issue with lack of bodily access, again their eye gaze failing. BCI access is a quite different access method than eye track- ing. With eye tracking, one must physically look and track to- wards the desired stimuli using the eye and the camera system of the tracking system (with some desired acceptance input such as a “dwell” option). With BCI as an access method, one must visually gaze (mentally fixate) at the flashing stimuli, thus triggering the SSVEP. This mental fixation will then activate the desired acceptance of what the individual is accessing. With electrodes attached to the occipital lobe, Cognixion’s BCI SGD with EEG stimuli presented can associate letters, words, phrases, or symbols to language (see below). This means that stimuli will be presented to the person allowing them to men- tally fixate on letters, numbers, words and/or phrases allowing them to continue to communicate when other access methods have failed. This innovative system will allow individuals with an inability to functionally eye-track to still have access to language (see image below).

TRAINING AND RETRAINING BCIs often require some form of training and retraining for the user. For example, in a BCI designed for communication, the user may need to undergo training to learn how to control

BACK TO CONTENTS

their brain signals in a consistent and reliable manner. This may involve practicing specific mental tasks, such as imagining the movement of a particular body part or paying attention to one (and only one) of multiple visual stimuli. In some cases, the BCI itself may also need to be retrained. This may involve updating the signal processing algorithms or adjusting the classification model to better accommodate changes in the user's brain signals. For example, if the user ex- periences changes in their brain activity due to medication or other factors, the BCI may need to be retrained to recognize and interpret these changes accurately. Additionally, brain signals used to control the BCI may change over time, and the BCI must adapt to these changes to maintain accuracy and reliability. Re- training can take various forms, depending on the specific BCI and the intended application. Overall, retraining is an important aspect of BCI development and use and plays a critical role in ensuring that BCIs remain ef- fective and reliable over time. By incorporating strategies for re- training and adaptation into the design of BCIs, researchers and clinicians can help to maximize their potential to improve the lives of individuals with neurological conditions or injuries. With this powerful technology the need for retraining is needed to ensure the highest level of accuracy for word or phrase selec- tion. REFERENCES: Arwa Rekik, Saloua Mrabet, Imen Kacem, Amina Nasri, Mouna Ben Djebara, Amina Gargouri, and Riadh Gouider Eye Movement Abnormalities in Amyotrophic Lateral Scle- rosis in a Tunisian Cohort https://www.ncbi.nlm.nih.gov/pmc/ articles/PMC9291663/ Xintong Guo, Xiaoxuan Liu,Shan Ye, Xiangyi Liu, Xu Yang, and Dongsheng Fan Eye Movement Abnormalities in Amyotrophic Lateral Sclero- sis https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9026966 David A. Moses, Ph.D., Sean L. Metzger, M.S., Jessie R. Liu, B.S., Gopala K. Anumanchipalli, Ph.D., Joseph G. Makin, Ph.D., Pengfei F. Sun, Ph.D., Josh Chartier, Ph.D., Maximilian E. Dougherty, B.A., Patricia M. Liu, M.A., Gary M. Abrams, M.D., Adelyn Tu-Chan, D.O., Karunesh Ganguly, M.D., Ph.D., et al. Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria https://www.nejm.org/doi/full

BACK TO CONTENTS

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7

www.closingthegap.com

Made with FlippingBook Ebook Creator